If you’re an Ansible user and encounter the following error:
unix_listener:"..."too longforUnix domain socket
you need to set the control_path option in your ansible.cfg file to tell SSH to use shorter path names for the control socket. You should have a look at the ssh_config(5) man page (under
ControlPath) for a list of possible substitutions.
In the company I work for we’re using RabbitMQ to offload non-timecritical processing of tasks. To be able to recover in case RabbitMQ goes down our queues are durable and all our messages are marked as persistent. We generally have a very low number of messages in flight at any moment in time. There’s just one queue with a decent amount of them: the “failed messages” dump.
It so happens that after a botched update to the most recent version of RabbitMQ (3.5.3 at the time) our admins had to nuke the server and install it from scratch. They had made a backup of RabbitMQ’s Mnesia database and I was tasked to recover the messages from it.
This is the story of how I did it.
Since our RabbitMQ was configured to persist all the messages this should be generally possible. Surely I wouldn’t be the first one to attempt this. 😐
Looking through the Internet it seems there’s no way of ex/importing a node’s configuration if it’s not running. I couldn’t find any documentation on how to import a Mnesia backup into a new node or extract data from it into a usable form. 😞
My idea was to setup a virtual machine (running DebianWheezy) with RabbitMQ and then to somehow make it read/recover and run the broken server’s database.
In the following you’ll see the following placeholders:
Now what? Why don’t I try to give it the Mnesia files piece by piece again?
rabbit_* files in again and fix their permissions
All our queues were back and all their configuration seemed OK as well. But we still didn’t have our messages back yet.
So I tried to copy more and more files over from the backup repeating the above steps. I finally reached my goal after copying
recovery.dets. Fixing their permissions and starting RabbitMQ it had all the queues restored with all the messages in them. 😂
Now I could use ordinary methods to extract all the messages. Dumping all the messages and examining them they looked OK. Publishing the recovered messages to the new server I was pretty euphoric. 😁
Sometimes well-intentioned features have unintended side effects. Case in point: WordPress’ maintenance mode. Whenever you update plugins WP will automatically enter maintenance mode, which displays a nice message to your visitors that the site will be back online shortly. It will automatically go out of maintenance once the updates are done.
Well, sometimes unexpected things happen: you are stuck in maintenance mode. WP will effectively lock you out … even the admin section will not be accessible. *ugh* This is the moment you start panicking … luckily if you wait 10 minutes or delete the .maintenance file manually you’ll be able to access your site again. *phew*
I recently finished introducing custom CA infrastructure in two instances. Each having two sub CAs, two Servers and a bunch of users. The “create your own CA” part was quite easy after I found a dated but still accurate tutorial. In hindsight it is quite silly why I didn’t do this before.
On the server side I had to make it work on:
Each expecting its own Format/Packing of certificates, keys and certificate chains. :/
On the client side I had to produce installation and configuration howtos for Windows and OS X and a bunch of popular browsers and email clients. Then there is the “user education” part … this is still in progress, but its looking good.