Ask HN: How Do Your DevOps/Ops Teams Handle OS/Container/App Patching?

3 points by dogismycopilot ↗ HN

4 comments

[ 4.7 ms ] story [ 17.8 ms ] thread
How do your teams minimize the amount of time spent patching? It is an enormous time sink for our devops teams, even using industry-standard open source software basically everywhere.
Designate one person to do this in your team as opposed to multiple people. Take turns to manage it. I am not sure why is it so big a time sink unless you are doing it like twice or thrice a week. Usually we do this once or twice a month
Well, let's say that you have 2 physical locations, 10 racks each, and a cloud provider, the OS, the networking equipment, the smart PDUs, the iDRAC and firmware, kernel (reboot), containers, VMware hosts/vsphere, Openshift, a few windows boxes because why not, database, apache, and all the downstream. A "high" CVE has 3 weeks to be fixed.
The app patching should be done as part of your standard CI/CD process, with appropriate control gates managed by humans at the Dev versus QA versus Prod environment interfaces. But that should really just be a button click, after human discussion has occurred and the appropriate level of consensus and approval is given.

Containers should be patched in a similar fashion. But the tooling might be somewhat different for containers versus apps.

You also need a CI/CD process to patch the OS on your servers, but again the tooling might be different again for OS versus containers versus apps.