HN Explorer

Ask HN: Was hired to improve company's devops, founder won't listen to my ideas

3 points by WAthrowaway ↗ HN
Three weeks ago I joined a fast growing startup on the West Coast to improve their dev operations and processes. The company is seeing crazy growth but they are still doing things like pushing directly to master and using prod deploys to test things out. I was hired to help improve this and implement things like staging environments, autoscaling of resources, and other DevOpsy stuff. Since joining I've received considerable pushback on my ideas. Obviously I am the new guy and don't expect to just dictate people around nor is that my intention. However what really gets me is that the founder just straight up does not seem to trust me. Every suggestion I give - even things as industry standard such as the use of a staging environment - is met with suspicion and doubt and the insistence that I hadn't done enough proper "research" on the idea. I find this highly offensive considering I've been working in DevOps for almost seven years and before that was a SysAdmin.

So after two weeks, I got the approval to create a staging branch. I wrote up a huge document on the staging workflow, covering edge cases such as hotfixes and rollbacks. I put time on people's calendars and walked them through it including the founder. Yet he still continues to resist the change and pushes directly to master, ignores the new processes I put in place, and generally just does what he wants. He also keeps harping that I need to "earn his trust." I find this baffling - I mean, I was hired for a reason right? Why would they hire me if they didn't trust me?

I am starting to get frustrated, especially since the founder has pushed breaking changes directly to master and then I got flack since I am now the "systems" guy and am responsible for the health of the app. I'm trying to think of the best way to approach this diplomatically. Do any HNers have some pointers or advice to give? I've worked with difficult clients before but I've never faced such a huge lack of trust.

7 comments

[ 3.2 ms ] story [ 28.3 ms ] thread
amacalac
If you're using Github or Bitbucket, consider putting a block on a direct push to master, and guiding them to a Pull Request / Code Review based procedure of merging to master.

If logic is breaking, look to encourage units tests on new code, and back port unit test coverage as time goes on.

WAthrowaway
Yes these are all things I have proposed but I just get stonewalled by the founder and the two senior engineers. I am debating just implementing the block without their approval - because it would seriously help the company - but I don't want to seem aggressive or confrontational?
officialchicken
One thing stands out in all of that: he's still pushing to master. Change starts at the top with leadership and then some accountability. Either start looking for a new gig or really embrace "continuous improvement" and start with much much smaller fights.
WAthrowaway
Yea I am looking around but I really like the product and I thought this would an opportunity to put my skills to the test since it all essentially greenfield. But if I can't get approval from the guy who hired me then it seems moot.
throwaway5250
Assume nothing will change, because it probably won't. Given that, consider the pluses and minuses of staying vs going. If the money is awesome, you might want to stay, at least for a while.
Lenad
A process I like to use and that might be helpful to you is what I call "Technical Initiatives".

The idea is to allow everyone to suggest and implement technical improvement by providing a process to do so. For each idea, the owner fills a spec to describe it. The main ideas are: - Be transparent on what we want to do and our issues, the doc should be public and comments-friendly - Follow an advice process by proactively asking people to improve or challenge the idea - Always have by a trial with time boundaries and goal

A template could look like this:

- Owner: Name

- Title: to easily speak about it

- Reviewers: people who should definitely have a look

- Summary

- Why: Describe what are the issues with the current process, the bugs or similar.

- What is the change

- How to implement the change

- Duration of the trial

- What should we've achieved at the end of the trial

In your situation, it might be something like:

- WAthrowaway

- Staging env

- Reviewers: Founder, Senior Eng #1...

- Setup a staging environment

- We encountered critical issue #1 on October 1st, issue #2 on October 2nd

- A staging environment would allow to catch this bugs before it is on production

- 1. Create a staging env 2. Create a staging branch 3. block push on master 4. PR staging / master

- 1 month

- Critical issues happen on staging and not on master

PS: It's inspired by http://www.reinventingorganizationswiki.com/Decision_Making

mhkool
My wife is a management consultant and she has hundreds of anecdotes about owners who do not want to change. If I were you I would be looking for a new job since you will most likely not gain the trust of the owner.