DevOps Engineering: Who does your automation?
Before being introduced to the concept of DevOps, we at Twilio struggled with what to call the role. It requires a different kind of thinking. To us, DevOps is about automation to improve productivity, reduce risk, and achieve scale. DevOps engineers should always be striving to deprecate themselves. It has been more then 2 year since we founded Twilio and we've never had a sysadmin or an ops person. This has forced us to write better software automation from day one. However, having someone that can provide leadership on DevOps issues (and fundamentally understands distributed systems/CAP/etc) to the whole team is extremely valuable.
Configuration management tools like Chef and Puppet are the beginning but a more complete view of infrastructure and service automation (e.g., http://redeyemon.sourceforge.net/) seems inevitable in the long term.
We are looking to hire a Lead DevOps engineer. Here is the detailed job description if you know someone that might be interested: http://twilio.jobscore.com/jobs/twilio/devops-engineer/bSHjYWq2Sr37U6eJe4aGWH
22 comments
[ 8.6 ms ] story [ 90.0 ms ] threadCall it what you will, 'DevOps' is just what you'd expect a good sysadmin to do in a sufficiently large org. That there is on going friction between developers and sysadmin ("it works on my box" is a valid excuse?) is perhaps a larger sign of the culture and people involved.
I work in a large organization (5000+ employees) as a unix/linux sysadmin. You have to automate deployment, monitoring, configuration management because there just isn't time to do anything else. We have a group (about 4) out of 30+ that have an overall view of what you might call devops. The rest specilize in development or sysadmins. We meet once a week and review all progress on the dev/ops issues.
I like the term OpSec (http://en.wikipedia.org/wiki/Operations_security) someone who looks after Systems/Security ;) Since a good system admin should also know a great deal about security. Maybe Dev/Op/Sec.. haha, but really ;)
p.s. I found that job posting to be full of PR fluff. It would be nice to see job posting actually post what their looking for in a distilled form.
I just think people need to specialize in their field (+ a general knowledge of how the overall systems works).
Almost how a carpenter, plumber, and electrician each specialize but probably have a general idea how everything works. Do you really want a plumber building the foundation of your house? Just seems a little silly to me. Each domain is so large that I would rather have deep expert knowledge in a area than a general handyman who might cut corners or waste time with solutions he doesn't know about.
This is called "second line support".
I'm sad that people are not taking the time to learn exactly what DevOps means and thusly commandeering the name for whatever they think they need.
Now I know that "DevOps" means "those who forget history are doomed to repeat it". Whenever you come across something in IT you think is new, take the time to find out if mainframe shops were doing it in the 1970s... Back then sysadmins wrote assembly language to automate systems tasks and it was just an everyday part of the job!
For instance: I wanted to do a few projects in Rails. I made sure that my admins had links, guides, and all the info they needed when I started the project. By the time I was ready to launch, they'd already had Ruby, Passenger, and all my dependent gems installed. Deploy was a breeze.
Later, a different coworker did a project in Django, but didn't breathe a word to our sysadmins. When he and the client agreed that it was launch time, our admins were scrambling to figure out what version of Python we needed, how to set up the environment, and what the deployment best practices were. Everyone was angry and the deploy was late, but the developer didn't see any consequences.
Kills me.
In the old-world way of things, the Architects of the system would take all of that into account - the sysadmins were just there to keep the systems up and running and install the software per the developers instructions.
What's new, or seemingly new, because of the rapid advance of technology and the internet in general, is the rapid develop/deploy cycle, which ends up requiring a much tighter coupling between development and operations... the two almost blend together. The lines are getting fuzzy.
Obviously the real world is different. New code is deployed that has bugs, there are unpredicted events, there are complex failure scenarios that are difficult to automate etc.
A DevOps engineer may write automation software in conjunction with developers to automate the operational aspects of business logic. Thus, it's a partnership between the DevOps engineer whose metrics are driven by availability/reliability/scalability/security and the developer who is trying to attain some business objective.
I, too, cringe a little when I hear the term, perhaps because I've always viewed myself as a "pure" sysadmin. That is, this is the kind of work which continues to bring me joy after 2 decades. Application development, on the other hand, despite an ability to code, does not.
CM tools (current and their predecessors like cfengine) as they tend to be implemented strike me as a way of avoiding administering the whole system, by focusing on a single component (servers) and reducing them to a lowest common denominator.
It also fits the pattern of trying to solve problems entirely in a custom software solution, a trait I associate with developers, not sysadmins. This is the source of my inference of DevOps' true leanings.
The reality is that the system consists of considerably more than fungible servers[1]. There's the underlying hardware, the peripheral hardware like disks, database software[2], network hardware and configuration, and service providers.
These are all traditional sysadmin areas, with plenty of room for professional growth and overall benefit. So much so, that I have difficulty imagining why any SA (not a developer in SA's clothing) would wish to do more dev at the expense of increaaed mastery of the infrastructure.
By way of example, something I wrote on LinkedIn, in response to why I'm not a fan of puppet:
[1] Virtualization and "cloud" notwithstanding, as anyone who's experienced an outage with a hosting provider or an i/o perofrmance issue with a cloud server can attest.[2] Which can be like an encapsulated OS itself
I use chef because I needed a way to consistently deploy software in exactly the same way every time to multiple servers. A repeatable deployment process that scales. Why did I pick chef over puppet or cfengine? Because I also wanted to learn ruby and the entire thing is in ruby, win. Plus we have a ruby on rails team in the office, if ever I needed a hand, they were there or if they wanted to write recipes to deploy software thats not an issue either.
DevOps is just a new buzz word. My title is System Administrator. I have a depth of knowledge in the infrastructure of all our technologies and how they are interconnected. I know how to deploy, maintain, scale and do it all over again. My knowledge in programming in second to that but is necessary to automate repetitive processes. DevOps I guess is being classed as a way to bring the IT team together, making sys admins and developers aware of each others "agendas" when it comes to running apps on servers. I as a sys admin have to understand what each of our applications does throughout its life cycle, from the browser right down to the database. Because if something goes wrong, I'll be the first to figure out where that happened and relay this back to the developer. By understanding core functionality of the code I can make it clear to the developer, "look you are executing a query on the database thats scanning millions of rows because its not using an index, its occurring in this controller in your code". We are seeing this term DevOps be classified now as more and more startups are emerging, because its essentially alot easier to execute in a startup environment where everyone is talking to each other to roll things out, everyone is trying to solve problems together. In large companies you'll see a massive separation and segregation along with processes to keep people in their place and slow results.
I hope that all makes sense. DevOps as a process is not new, the classification of it as this term is.
http://www.ccpgames.com/en/jobs/job-details.aspx?jobid=101