Backing up the blogs

Although this blog has been mostly dormant for quite some time now, my two other blogs–the Matthew Picture of the Day and the Peter Picture of the Day–have been plugging along, barely missing a beat. I do have all the photos on my home computer, of course, which I back up twice (continuously via Time Machine and also to CD-ROM). But I’ve been a bit cavalier about backing up the blog. I’ve periodically done manual backups before major upgrades, but I’ve sort of naively trusted in Bluehost not to lose my content. A year ago or so, Bluehost had a substantial hiccup and MPOD was offline for several hours. They did rebuild everything, and all was well, but the episode did give me pause. Also, I remember Photopoint.

So it was time to get serious about backing up the blogs. These were my criteria:

1. The backup must be stored by a different hosting company than that which hosts the blog. After all, I want to be able to recover if Bluehost has a catastrophic failure.
2. The entire site must be backed up. As of now, the MPOD is at about 800MB and the PPOD is about 500MB (until the debut of the PPOD, I uploaded only the small size photos to the MPOD, but now I upload the full-sized originals to both, which is why the MPOD archive is not much larger). This is particularly important because PhotoQ, the photoblog plugin that I use, stores the photos in two directories of its own creation within wp-content. So the backup solution must be able to see and archive these.
3. The backup must be automatic. I don’t want to be required to start each backup and I especially don’t want to have to use my home computer as an intermediary in transferring the archived files.

With this, I evaluated four automatic backup plugins. Rather, I installed and tried three, all of which failed in one way or another, before I found one that worked. My present solution is XCloner, stored on Amazon S3. Here are my experiences with the four plugins:

The first one I used was Automatic WordPress Backup. I do have to give this one credit: it was the first one I heard about that used Amazon’s S3 service to store the data. So it’s a pioneer of sorts. It has its own website and a slick demo video. Like many folks, though, I’d start a manual backup, it would finish, but nothing would have happened and there would be nothing stored on S3. Eventually I found the log file and discovered that it was running out of memory. This problem I could actually fix: I had inadvertently left about 10000 spam comments in the WordPress database, and clearing those out allowed the plugin to finish. But seriously, 10000 comments is not really all that many for a high volume blog. So the first problem with this was that it didn’t look like it could scale. (Note further that the flashy promotional videos and postings on the blog were not done by the programmer, and that the programmer is working on a different plugin now with similar functionality.)

The second problem was that AWB backed up the database and a few set directories, but not arbitrary directories introduced by other plugins. So only the database structure of the photoblog was backed up, not the photos.

And it also seemed that, despite all the flashiness of the website, it was quickly becoming abandon-ware. There hasn’t been a new version since August 2010. I had posted a question on their product blog asking if there was a way to backup other directories, but it was forever stuck in moderation and has never been posted. I think they had expected more people to buy premium support (which is no longer for sale). They were also upset that WordPress’s rules prohibited them from automatically inserting a link to them in the footer of each of your blog pages. And they also insisted that, instead of being controlled through the ‘plugins’ or ‘settings’ menus in WordPress, they needed to add their own menu to the main sidebar. And, I think it’s lost compatibility somewhere around WordPress 3.2.

The next plugin I tried was wp Time Machine. It has a very simple interface and does back up all files. It’s designed so that you just type in the minimal information needed and then click “generate archive,” so it’s a little counter-intuitive that you need to first click ‘Show Plugin Options’ to switch from the default Dropbox to Amazon S3. The instructions are found within the plugin interface, but it sends you to a blog post to get the details of setting up the cron job needed to run the backups automatically.  It also automatically includes, in the files it saves on Amazon S3, an instructions file for recovering after a crash.

I initially had file compression turned on while making the archives, and when adding the photo files to the archive, it would ungloriously quit running after about 330MB, leaving an incomplete and corrupt archive file. Turning off compression I was able to get the PPOD site successfully backed up–I saw the full file archives appear in the Amazon S3 console–but the plugin page display just indicated that the backup was in progress. (Turns out, JPEG files are already pretty well compressed and ZIP won’t make them much smaller.) In fact, several times I left it for hours thinking it was still working on the backup, because that’s what the screen indicated, while in fact it had hung up somewhere. Trying both PPOD and MPOD, I variously got failures to finish building the archive file, a finished archive file that failed to transfer to Amazon S3, and successful transfers to S3 that failed to indicate completion in the plugin dashboard. Perhaps it runs better via cron job, but it seemed like this plugin had troubles with lots of files.

The third plugin was pressBackup. They sell their own cloud storage service, but their plugin is free to use with Amazon S3. It uses a wizard to walk you through the setup process in a completely intuitive manner. It just asks you what service you’re using, how often you want to backup, and how many previous copies you want to keep stored. But the problem: they allow you to backup the database, themes, plugins, and uploads. Not arbitrary files and directories. So, like Automatic WordPress Backup, it wouldn’t save the photos. That, and its easy configuration was perhaps too easy: although you can have a daily backup, there’s no way to set the time at which the backup will execute; it will always execute at the same time as your first backup. I sort of imagine that servers are more idle and bandwith is more plentiful in the middle of the night, which is when I would want to schedule backups. But you don’t need to configure your own cron jobs.

Which brings me to XCloner, which I’m currently using. It is tremendously flexible and the configuration is both more involved and less obvious than with other products. It has a PDF manual, which seems to be a few releases behind. It did take a bit of work for me to get it up and running, but once I figured it out, configuring it for subsequent blogs was completely straightforward. But it was far from obvious which configuration options needed to be set and which didn’t need to be.

A few notes: you need to create your own directory structure, preferably in the blog’s top level directory, for it to store the backup files (which will then get transfered elsewhere). In order to transfer to Amazon S3, it needs to be run from a cron job; a manual backup will only create and store the backup file. There are quite a few settings that one doesn’t need to bother with (for example, cron configuration names). And the plugin itself has a username and password, that it wants you to configure right away–I suppose this is useful for a multi-author blog. If you want to do things like doing a nightly backup of the database only to be stored locally and a weekly backup of the whole site transferred to an FTP site, then XCloner can do that. It didn’t have any trouble with the large number of files I needed to back up, but if it did, then it has an option for an incremental backup that works on a smaller number of files at a time.

So in the end, there are no circumstances in which I’d recommend Automatic WordPress Backup. If you don’t want to mess with cron jobs, and the limitations to pressBackup don’t deter you, I’d recommend it first. If you need more flexibility–say with when the backup executes–and if the size of your site doesn’t trip up wp Time Machine, I’d recommend that. And if you have having things configured for you and want lots of settings available for tweaking–or if the other options won’t work for you, then XCloner will certainly get the job done, once you figure it out.