Ask HN: What were the things you did that made the biggest impact at your work?
It could be a tool that you have created, a process that you have started or anything else which created a big and lasting impact on your workplace.
For me, it was probably starting holding regular tech book review sessions.
Over time the discussion around both architecture and implementation details got way more structured, quicker and more satisfying to all parties. Seems obvious, but building up mental model of software development through a structured approach is better than just letting it happen naturally during work hours.
215 comments
[ 2.8 ms ] story [ 277 ms ] threadAutomation
The automated checker was implemented using cPython.
...and as a safety executive at a Widget factory, I have to be really careful who I am aiming my very rare criticisms toward; at times it seems as though people who work in the beer factories that don't put widgets in their beer cans have much lower safety standards than I do, and they get upset at some of the criticisms directed at them.
...that is why we are planning to use Ruby or Scala next time; the other factory workers don't understand that lol.
...I also implemented a "you're working too slowly" message for people who didn't follow their documentation with good authentication practices.
Most people either write no docs or they write a doc that assumes the reader already knows what the thing is.
I write docs that assume you're a new employee and explain the thing in simple, plain English before building up the technical details.
Notably, these kind of docs make it easy to handoff maintenance and further development to someone else so you can move on to new projects.
How did you do these so people actually participated? My company is very deadline driven so people don’t really care about learning. Every attempt at regular learning meetings quickly died because people feel they don’t have time. Do you have a specific format?
I have led regular meetings about coding style in another company. These went pretty well but in the current company I have failed at instilling curiosity for learning things for its own sake
“What are you doing?” you ask.
“Can’t you see?” comes the impatient reply. “I’m sawing down this tree.”
“You look exhausted!” you exclaim. “How long have you been at it?”
“Over five hours,” he returns, “and I’m beat! This is hard work.”
“Well, why don’t you take a break for a few minutes and sharpen that saw?” you inquire. “I’m sure it would go a lot faster.”
“I don’t have time to sharpen the saw,” the man says emphatically. “I’m too busy sawing!”
-- The 7th habit of "The 7 Habits of Highly Effective People."
I had to convince both sides: 1. the management so that they would give us explicit permission to take time from our "actual work" time 2. the devs to actually start and keep it going
management caved in after... - I researched our competitors and their tech, highlighted features which would be very hard/impossible to implement given our architecture and what we needed to learn - dozens of comments about what we lose implementing feature Y in the "old" and "known" way vs using tech/service X - made comparisons to different professions which always have to stay on top of their game to win
all while repeating the message that we cannot have a world-class product without a world-class tech team. I think framing this as a "loss" instead of "gain" helped a ton.
With management buy-in, we were creating a "learning" card for everyone in the tech team each sprint. These cards were "committed" so there was no getting around of finishing them.
The dev team needed some motivation as well. Coding features as quickly as possible has obvious and visible advantage - you get praised, you might get a bonus, you might be promoted. Learning on the other hand.. less so.
To keep a long story short, I convinced them that the more they know the better their market value is by showing various stats. After the initial nudge, it was just intellectual curiosity that kept the meetings going.
Format wasn't anything special: - pick a book/article/system by vote - one person spends a couple of hours going through the chosen 'thing' - prepares a list of items to talk about - people vote about what should we talk about (before the meeting) - we have a presentation/discussion. each meeting ends with - list of practises/ideas we would like to implement. the list is committed to our documentation repository and followed up on - list of things we would like to learn in the future
Unfortunately, I find very long BTD loops to be personally offensive though, to the point it makes me emotional, which harms my ability to make my case. Despite knowing this, it's almost impossible to stop it from coloring my arguments. I fear that I have such a large amount of contempt for those who don't share my view on this that it leaks out whatever I do. I even know, intellectually, that this is wrong - ignorance does not deserve contempt, but compassion, after all. But my traitor heart doesn't want to listen. :(
When was this, and how does it look now?
Having come off of a project that supported 400k registered and 100k active users on a single server, this project isn't yet at that scale and they were anticipating massive scale issues. This mistake was a real bonanza for an engineering team wanting to learn the latest tech; a massive failure for the product that really needs fast UX iterations to succeed.
If you want to talk more about it, you can email me.
Back in the late 00s we horizontally scaled a monolithic project with 5mil daily users and another 8mil daily users. Peak was 10k and 15k requests/sec of dynamic per-user webpages.
The hardest part was the databases. Our solution was standard for the time but pretty ugly nowadays: sharding MySQL by user id along with heavy use of memcached.
Would have killed for some of the modern scalable databases. Sometimes I wonder if some people using microservices are chasing challenges since much of the actually necessary hard parts are already figured out.
I don't know how true / substantial this effect is, obviously. Nonetheless, seems like good self awareness shown in this comment. That's more than a lot of people can say
Alternatively, I'd like to work for an organization that's on the same page here. Effectiveness in software (or anything that depends on iterating on a design) relies more on a fast loop than anything; I'm tired of fighting this battle.
It can be annoying, but at the same time, it forces them to take a break to think about things they're currently working on. When you have nothing to think about and your current work is simple and you're waiting minutes, you tend to get bored and frustrated though.
And also remember that some people would simply rather get paid to do nothing, and a long BTD cycle helps that.
And yet some organizations want to run all the linters, all the tests, for all the languages, orchestrated by Jenkins on a huge Amazon cluster. This is bad.
[1] This is very cool - an Apple IIe emulator running Logo in the browser. https://www.scullinsteel.com/apple2/#logo
[2] Inventing on Principle has improving the BTD loop at its core https://vimeo.com/36579366
i.e. It might not be a bad feedback loop, you just need more of them that are faster.
Imagine trying to read a novel but after every paragraph you had to stop and wait for 5-10 seconds.
Maybe a different way to look at it is: if you could shorten the BTD loop to a minute or two, why _wouldn't_ you want that?
You should try writing software for SP routers :)
Encountering these in my very first programming job made me learn make(1).
The thing is, I'm not very bright and my work "style" involves a fair share of trial and error. Obviously, all this waiting prevented me from doing my thing, so I eventually rewrote the whole build system just so I can do more trial-and-error.
I've always considered my dependency on a fast BTD loop a weakness, though. I guess the majority of developers can just imagine what the code does in their head and only make the changes when they know what they're doing.
The ole eyebrow started twitching when I read this. I once had a 6 month contract with a RoR shop where they had such bad performance issues they started developing in production mode so it wouldn't reload everything constantly. This allowed you to make some changes quickly, but 90% of everything required you to restart the web server and it literally took minutes for it to load everything. They had close to 200 rubygems, and I've always guessed that was the core of the issue.
When I suggested we need to fix that issue, the senior did-an-internship-here-last-summer developer got extremely passive aggressive with me (I didn't renew that contract). This guy thought he was amazing, I remember him arguing with me about a bug where they were storing decimal numbers as float in MySQL. I seriously had a 30 minute long conversation where he told me repeatedly I was wrong until I finally got exasperated, made the changes, and it fixed the bug. The guy was couldn't believe it fixed the issue and spent another couple of hours looking for why I was wrong.
I've never touched RoR again, that experience was horrific.
Sometimes you can't help the slowness, but I know I seriously dislike header-only libraries in C++ specifically because of their effect on the BTD loop. I have no idea how people can tolerate working like that, it's a slow hell for me.
Making sure that upcoming new features and achievements are showcased to the whole company through weekly demos.
Now each team has their own dashboard. E.g support team shows a graph of open/assigned/outstanding tickets, with SLA breaches highlighted. Sales team shows a sum up of data anomalies in their CRM (missing data, inconsistent states).
The rationale is that anything that shows up in red needs immediate action from a human being to avoid blocking the team, and we strive for the right balance between always green and always red (which are both useless states).
Currently, I'm having more fun building software tools that are actually useful to people. Who knows, maybe a few years down the road I'll get bored by this, and dedicate my time to other activities ;)
So there were 2 kinds of big categories of things that happened. One was that we really slashed the amount of time from first master branch build to deployment. If things went well we could release to production in under an hour. This radically changed everything about our process. We started building smaller stuff because we could try things out in production very quickly. Code review times dropped a lot because we had smaller units of code. Planning meetings went faster because we were talking about smaller things so discussions didn’t go sideways. The team seemed to really like it.
Second class of thing that happened was that people from other teams would drop by our slack channel and see us rolling out code by talking/clicking to a bot, and they’d want to know more. And then we’d show the speed improvements and how it integrated into the rest of the prod architecture so that you could make the migration to using it too. So it kinda organically became the defacto release architecture at the company over time.
I’m a little bummed out now because it was super fun to see this all go down, it was almost accidentally the most far-reaching change I’ve instigated at work and the impact was great.
https://www.amazon.com/Unicorn-Project-Developers-Disruption...
The gist of it is that reducing your time-to-production is one of the best investments you can make. It changes everything about how you build things.
[EDIT] to be clear, quick painful dangerous releases are no better, and (pretty much necessarily) come with the same pain in onboarding, jumping between repos/projects, undocumented "oh sometimes you have to re-install that to get it going" works-on-my-machine bullshit, getting a good feedback loop going, et c.
I wrote a software firewall to prevent the exploitation of these insecure websites as well as a monitoring system to identify any websites that got compromised in other ways.
It moved us from spending 80% of our time recovering endlessly defaced websites to 5%, pushed us on to newer and bigger projects, and made several hackers and at least one pen tester actively curse the systems in their exploit code.
Realizing that my career is my own, and that I owe nothing to anybody. Nobody is doing _you_ a favor by hiring you. At most, it's a calculated risk.
Furthermore, realizing that I shouldn't suffer for management decisions. If you've been putting out fires for a whole year, either the whole team has been consistently doing a very poor job (unlikely, though possible), or the fault is out of your reach (quite likely IMO). In either case, it's totally acceptable to jump the ship to another team.
A company only keeps you around if they can justify getting more value from you than what they are paying to keep you around. Add more value and document that value you're creating for job security.
Therefore most feature code should be shipped to master with feature flags.
In general I don't like this approach. Branches are troublesome when you hire mediocre developers who don't communicate and coordinate between each other. Merge conflicts are often a result of laziness, poor architecture and planning, and weak communication skills. In my experience, smart developers do not create problems with feature branches.
It also means we can put an extra layer of QA testing between dev branch and master, which works nicely as a means of forcing communication between devs and QA too, and helps with onboarding (because the product featureset is huge).
Automated tests are great, but having a person read the feature ticket description and testing steps and then check that the functionality actually makes sense before it hits the master branch...
One more tool in the box, and it's a great culture correction mechanism too. Devs who don't test their work before submitting it get it sent back for fixing and feel less productive, so doing things thoroughly the first time feels quicker than rushing it. Doesn't even require management to monitor rates of reopening tickets, etc: people just don't like having to redo things.
Because the test was specially designed to weed out people that didn't know how computers/compilers worked, it was surprisingly effective at removing 90% of candidates. The people that got in were of the highest quality.
- I was the one-person support for a while, and created our support process, user guides, etc. Any way that I could automate the process to speed up response time and free myself up to do other small tidbits of work, I tried.
- I was the one-person QA team, and documented known corner cases, golden paths, feature-specific regression testing guides, etc. When another employee began helping with QA, it made it much easier to onboard them to the process. We’ve since hired someone to build test automation programs and they’re referencing the same guides that are years-old now to guide their automated tests. I thought that was pretty cool.
Initiatives: More and more initiatives and idea discussion - because of this everyone started thinking everyday to build innovation and became adaptable to take risk.
[0]: https://github.com/kbariotis/templates
1/ carefully rearranging headers in a large C++ product, cutting build time from hours to 30 minutes. Think levelization a la Viega (see https://www.youtube.com/watch?v=QjFpKJ8Xx78).
2/ setting up JSP->java->class compilation to detect syntax errors at build time rather than at web server startup, eliminated finding such errors in the field completely.
I would say this comment section should be renamed "Don't make a big impact at work without clearly getting the credit for the value added"
Before you do any more impactful things at work to benefit someone else's equity, do the following
- set up an internal blog location where you can write up, each friday, what you and your team have achieved
- add simple videos of good stuff in action (screen casting is easy now and always stick your voice track on top)
- tell people, with links.
Stake your claim.
* reduced build times. Boss was angry at me spending time on it and it did take a while but after we started shipping weekly instead of quarterly...
* ignore my boss anytime he gets annoyed I automated legacy things. It bit me in the ass once, but paid out 10x several times.
* internal tools and automation scripts
Multiple usages:
Enter one keyword, and look it up in Wiki, Redmine, two extra ticket systems, mails with customers with fewer clicks.
Or enter a customername, and find him in any if these systems using other search forms.
Also offering short links to often used functions, like selfhosted password generator etc.
Saved me lots of clicks and copy and paste when debugging or investigating issues. The company had >1000 customers, product was an ecommerce software based on php/mysql
Received a small bonus when the bosses found out. I do not know if my former collegues still use it, I left the company.
I think a lot of the tensions that often exists between people from different backgrounds, especially when deadlines and budgets are tight, is lack of understanding what the other person is doing, what their job is.
So I started doing little talks and workshops around topics like, web applications, databases, what the different components of the system are and how they interact with each other, different types of "bugs", computational complexity, etc.
While, most of these weren't directly applicable to those peoples daily work, it helped them understand a bit better why for example task A takes much longer than B, although from a laypersons perspective they look the same. Also people appreciate learning new things when broken down to the appropriate level and removal of jargon.
Similarly I let interested tech people learn about things such as UX, product development, marketing, finance, legal, etc.