We use it to automate or semi-automate a broad range of tasks that can't be accomplished within the context of a single app or system.
For example, we have a pretty extensive pre-flight deployment checklist for our applications that are in PCI scope in order to ensure we meet compliance requirements and reduce human error to provide a high degree of reliability to our customers. This checklist includes a number of sanity checks to ensure we are deploying the correct code, that it has been thoroughly tested, that it has received the necessary sign-offs, that contingency plans are in place, and that we have appropriate monitoring in place.
Some of these steps are automated such as launching monitoring dashboards and some steps cannot be automated and must be done by a human such as verifying that appropriate contingency plans are in place. Our runbook walks our developers through this process step-by-step and documents that all steps were followed for future audits. Further it provides a low barrier to entry so developers that are new to the team can execute deployments of our mission-critical applications without extensive training. When our process changes, for example when migrating our applications to kubernetes or introducing new monitoring, we can make these changes without requiring developers to learn a new process.
Another example where we've found runbooks invaluable is doing OS upgrades for services in our datacenters. These operations have resisted full automation due to slight variations in system configurations, however scripting them with runbooks allows us to troubleshoot issues, build fixes into our runbooks, and resume execution where we left off.
3 comments
[ 1.3 ms ] story [ 20.5 ms ] threadFor example, we have a pretty extensive pre-flight deployment checklist for our applications that are in PCI scope in order to ensure we meet compliance requirements and reduce human error to provide a high degree of reliability to our customers. This checklist includes a number of sanity checks to ensure we are deploying the correct code, that it has been thoroughly tested, that it has received the necessary sign-offs, that contingency plans are in place, and that we have appropriate monitoring in place.
Some of these steps are automated such as launching monitoring dashboards and some steps cannot be automated and must be done by a human such as verifying that appropriate contingency plans are in place. Our runbook walks our developers through this process step-by-step and documents that all steps were followed for future audits. Further it provides a low barrier to entry so developers that are new to the team can execute deployments of our mission-critical applications without extensive training. When our process changes, for example when migrating our applications to kubernetes or introducing new monitoring, we can make these changes without requiring developers to learn a new process.
Another example where we've found runbooks invaluable is doing OS upgrades for services in our datacenters. These operations have resisted full automation due to slight variations in system configurations, however scripting them with runbooks allows us to troubleshoot issues, build fixes into our runbooks, and resume execution where we left off.