Designing Code Structure for Server Deployment

vinmar

I’m not sure when our offshore team decided to start using subversion. Was it at the start of (or during) their current project with us or have they always used it? One thing’s for sure, they certainly never considered using it as a deployment tool and that’s causing me one helluva headache today.

The ‘data/’ directory, whose name alone shows at least some foresight as to what kind of stuff would wind up in there, is parallel to such esteemed colleagues as ‘application/’ & ‘html/’. But this is exactly what we have repository ignore earmarks for, right? Yeah, sure – but this directory (including several subdirectories) are in the repository. So moving it away or writing additional data to it results in a modified version of the codebase. 😦

Exactly What Code Is Running On This Server?

If you can’t answer this question, you have a real problem in your IT landscape. Currently, all developers have ssh access to all systems and push changes around as they develop them (exactly because there is no repository management to pull their changes down to the live system). A few times, I’ve committed my changes only to find out that a dependency which is present on my local (and uptodate) system hasn’t been deployed yet to production. The proverbial chicken-and-egg problem.

Divide And Conquer

My solution is to start migrating directories like ‘data/’ and ‘resized-user-images/’ out of source code control. Of course, moving away such critical application resources can have tidal wave like effects on synchronization, backups, and cronjobs. So you have to prepare in fine detail, construct a plausible smoke test for ensuring correct operations after the migration, and most importantly have an accurate rollback plan ready to go in case you figure out you missed some crucial detail half way through the procedure.

Once this is done, I’ll examine the various cronjobs for unexpected writes and updates to files that are normally under version control. We’ll revisit this topic after getting the production environment running under source code management and we can easily check the status of the currently deployed codebase.

Ensure Your Test Systems Get First Class Treatment

If you have a test system, get it equipped with all the backups and cronjobs running on production. Even if you can’t bring over all the scripts (because synchronizations are happening between multipled machines and you only have one test server), whatever you’re able to get running brings your test environment closer to production. In turn, you’ve got a better test bed you’ll have for making any future changes.

Slow And Steady Wins The Race

Do not go all out and attempt to fix everything in one big-bang release. Even if the next day seems error free, I guarantee that monthly cronjob will hit you hard and you’ll be completely unprepared for it (and rolling back after a month will make you wished you hadn’t ever considered trying to get a grip on your environment in the first place).

Instead, after each change ensure that the live server still operates as required and eventually replace the current free-for-all with a simplified update script which will easily let everyone deploy a specified revision and know exactly which version of the code is currently running on any given system.

How are you handling your code deployments? Drop us a line and share your ideas!