Backup - a delicate.

Backup - a delicate matter.

Virtually all agree that backups should be done. But, nevertheless, this problem pops up again and again. A recent poll showed two interesting points: First half of us are not doing backups at all, and secondly the author does not even occurred to me to include in the survey item "once a day. What is wrong with a simple, at first glance, the task - to pack their files and put the archive into a warm and dry place?

The main problem is that Bacup not apply to those things that you can do without thinking! If you try to blunt wrap up the entire contents of the screw, then first you have nowhere to these files (daily:)) hold, and secondly your machine will do archiving clock itself, the beloved, instead of carrying out your tasks. And when you start to think (which is already not easy), it turns out that the data on the HDD very different, and Bacup them desirable, too, in different ways (that definitely complicates the situation). As a consequence, any decision not to do backups at all (disguised as a "set aside for later"), or put to the first available tool and somehow managed to quickly set up, in the hope that this will be enough.

So, no, and can not be, such a solution for backups, that fits all. Some people need to backup their texts under WINDOWS on a network drive, somebody needs to backup the entire server to tape, someone uses instead of traditional backups Venti, and somebody should just make regular backups of their web project and submit themselves to GMail . This, in turn, greatly complicates the selection of suitable software for you.

So IMHO the best way to help someone with Backup - to tell what, why and how you backup. I have encountered two different scenarios, and I want to talk about their decisions. It should be noted that in both cases for me, the most critical factor was reliability - why you need the backup, from which can not be guaranteed to correctly restore your data?

Backup a large web project

Make a safe backup of a large project is not easy - so that after restoring from backup, he continued his work is guaranteed and no problems (except for the absence of new data that have been added since the last backup, of course). The fact is that the disk files, and database is not always located in "holistic> (consistent) state. Given some transaction in the database (which is not always clearly formulated as SQL-transaction, by the way), creates temporary files are written logs ... 

And in a big project is often a lot of other "entry points" in addition to HTTP: processing incoming mail, RPC-requests , cron-task, work different TCP / UDP services. Catch all of this huge edifice in a consistent state, and yes even hold it all the time that the backup is performed - a problem for external software simply impossible!

In this regard, our decision is that the project itself must support the ability to create an integrated backups. The approach is roughly the same as with the test - that the project was possible (and convenient) to test the code originally written with the requirements of testing. 

Realizes this is trivial "head-on" - all applications of the project supported a unified system of locks, and backup at the time the project is blocked.

But this led to a new problem - in fact it does not back up fast, especially a large backup of the database ... but if the project is a long time to block once a day, our users, we thank you for it just will not tell. The problem we have decided to reorganization of the database so that all tables have been divided into two types: static and dynamic. With "static" tables are performed only SQL-query SELECT and INSERT, and "dynamic" can be changed arbitrarily. Of course, all volumetric data were made in static tables. This has reduced the time to block the project for the backup to a few seconds: archiving (tar, no compression) project files, create a dump dynamic tables, and remember id last recording in static tables.


Then the lock is released, and already without haste, dump the static tables (up to memorized id), tar compressed and went to the store. Of course, static spreadsheets and project files can (and should) to backup incrementally.

Despite the universality of the approach (we use it in all his projects), our script for backup will be of little use to others, because confined to the use of our framework.

Backup conventional dedicated server

After weighing their claims to the software for backup server, I came to the following list:

• Software for backup should be simple and reliable. The dependence of the number of bugs the size of the software all know, and I only bugs in the backups to a shortage of perfect happiness!
• When the backup server completely impossible to take into account all the nuances installed on it projects and ensure that they guarantee the integrity after restore from backup. Therefore, the approach described below is not used instead, together with those described above backup of specific projects. But nevertheless, there are things that need to be and backup server - for example, the integrity of the database (it can not be complete in terms of a specific project, but it should be complete in terms of MySQL-server).
• Backup should be performed quickly so as not to interfere with the server to perform its primary function. (On the server, an acquaintance admin I once discovered that his cron-task archived screw once a day. Tar.bz2 is performed for two hours, with nothing else at this time on the server makes it virtually impossible!) In particular, based on previous item, it means that lock the database should not all the time backup server, but only during backup database files themselves.
• Format files. Putting such a critical thing as backups in the dependence on non-standard file formats are very reluctant. Standard utilities are more than enough to back up and encrypt backups (eg tar + gzip + gpg).
• The system should allow sufficient flexibility to handle backups - Encrypt, pour on other servers, etc.
• The interface must be sufficiently informative for debugging and configuration backups, but it should not be spamming me every day with letters on each server "all right, backup is made, I want to receive mail only if the error occurred.