Deploying during business hours

I recently had to defend a habit I created of doing our production deployments at the start of our day.

Most companies deploy during off hours, or over the weekend. Having myself suffered through many early morning deploys I decided to try a new strategy about a year ago, and it was very successful. Let me lay out my reasoning.

Developers hate to get up at 2am: And we should really care about what developers like and dislike. Not because it's a nice thing to do, but because developers can do massive amounts of damage (purposefully or not) when they are forced to do something they don't want to. Take Zeck for example. Zeck is tired and really wants to go to sleep, but the testing team is taking longer then estimated because the documentation of some new feature is non-existent. After 2 hours of waiting for something to happen a defect is opened against Zeck. Turns out Zeck is not handling exceptions and his code failed a down scenario test (yes some testers actually test this). Rather then think about how to implement this correctly, Zeck just returns what is required to pass this one test (the back end will never actually go down right?) and kicks the defect back to testing. Is this example convoluted? Only slightly. But regardless, what have we accomplished? Exposure of the deploy to the customers was limited. Zeck and the rest of the team don't come in the next morning. Zeck is a little less happy with his job and thinks about what life at Google is like this time of year.

Users are the best testers: Even if your testers are complete sadists when it comes to finding flaws with your developers code, nothing compares to the breaking power of actual usage. And they are going to find most of these defects the morning after a deploy, while your developers are sleeping in, or taking comp time for the previous night. Even if you take the comp hours ahead of time (or not at all), people are going to be worn out from the previous night. A caffeine loaded sleep deprived developer can be the most dangerous of all coding animals. Especially when you are working with live data, which often cannot be restored with out losing new data.

Creatures of habit: If your developers are human (and they are human) then they have a routine. Wake up, drink coffee, write code, go home, or something to that effect. When you introduce a unexpected change (while you may know you have to wake up tomorrow at 2am, your body does not), you throw off the routine, and impair your ability to get in the zone. Now this point is very abstract, but if you have done any reasonable amount of coding you know what I mean. Granted some people can always find the zone, they are a sort of enlightened species that walk among us. If you know someone like this please send me their contact info. It's uh, for my ... research.

It's tough to deploy every two weeks: So you read an article about agile development while getting the oil changed on your Prius. But you just can't imagine doing a deployment every two weeks! Once a month is painful enough. However if you remove the pain (see above) from deployments it's easy to deploy twice a month, even four times. The more often you deploy, the less risk you introduce of something breaking, which means less testing. It's like compound interest, sort of. But if you really want to make it work you have to reduce the fixed pain the night deploy brings.

Bottom line: Happy and rested developers fix code fast, and properly. They also tell funnier jokes. How can you argue with that?