Ask HN: Testing Code Changes Against Production

2 points by psp219 2y ago ↗ HN

I’m currently reaching out here to see if anybody has any advice on testing code prior to pushing to production. I write unit / integration tests to make sure code is as correct as possible, but I don’t have any way of actually testing it against real, production traffic to know for sure.

Writing a unit / integration test for every potential edge case doesn’t seem scalable - wondering if there’s a way to test code changes directly against production traffic without exposing users to potentially breaking changes?

A canary comes to mind but serving real production traffic would expose users to potentially breaking changes?

How do you guys handle this at your companies? Would appreciate any thoughts.

2 comments

[ 3.3 ms ] story [ 12.8 ms ] thread

ssss11 2y ago ↗

I suppose it depends what type of code you’re developing but in general I’d expect a dev environment and unit tests, then a test environment and integration tests, a staging environment and UAT (or UAT in Test) and then production.

Test or staging would usually have a copy of old prod data.

It sounds as though you can’t predict the prod traffic though. Is there no sample traffic or even stats on types and volume of traffic over a period of time you can take to then feed to your test environment?

ericalexander0 2y ago ↗

There is no one size fits all answer. It depends on the tolerance for failure in the system. If testing in production stays within the tolerance, then it's likely more efficient to test there. If not, then it's more efficient to test pre-prod.