There may be an anxiety factor which negatively impacts live coding interviews which wouldn't manifest as often in demonstrating similar skillsets in a game. It would be interesting to see how much of a difference something like this makes.
If someone is going to argue that Factorio is a useful tool for technical interviews, it's only reasonable to expect that they will present a visible correlation between in-game performance and real-life performance. Otherwise, how is this anything other than a novel just-so story?
Even if something isn't measurable, you can provide anecdotal evidence that a correlation exists.
If you're going to say "Methods X and Y pale in comparison to Method Z for evaluating W" and expect anyone to draw conclusions from it, one would hope for even just a little bit of evidence.
How can this not have a huge bias in favor of gamers?
Might work for them because that's the kind of culture fit they need, but I doesn't look all that generalizable for other companies.
You only have to understand how to move your character using WASD, the concept of Inventory and pointing at different places on the ground with a cursor. Failing to understand this simple set of rules would disqualify a person from most jobs ever invented.
But its biased against people who can't use mouse and keyboard :)
But on serious note,
People who played the game could just hide the fact that they know the game mechanics. And act like they are catching up faster then normal person would.
Isn't that a common criticism of every interview style? Whether it's whiteboard questions, take-home projects, soft skill questions, lots of people will lie and say they've never seen the question before.
At this point, you would be foolish to admit that you’ve seen a question before. Everyone is faking it, and if you be honest, you’ll just keep getting harder and harder questions from the interviewer. The only time you should not fake it is if you’ve been asked that question at the same company. I think they would keep track of that and if you didn’t mention it, they would look deceitful on your part if they catch you.
> you’ll just keep getting harder and harder questions from the interviewer.
Well, there is a counter strategy that works even in this situation.
The counter strategy would be to just know way more interview questions than the interviewer, and know the answer to every question that they have, so that they run out!
When I was full time applying for jobs, a couple years ago, it got to the point with me. It turns out that people interviewing others, only have a couple questions that they use for interviews, and it is not that hard to know all of them.
>People who played the game could just hide the fact that they know the game mechanics. And act like they are catching up faster then normal person would.
This is a universal problem with all interview techniques. Interested in any ideas for how to fix it.
It's even a problem with "personal interviews," because some people have had more experience with optimizing one-hour first impressions than others.
Most interviews in other industries try to figure out how much a person knows at that point in time. Software interviews try to do the impossible and try to figure out how quickly someone learns. That’s because this can be gamed, as you mentioned. I know people that have memorized 800 leetcode questions and got jobs at all the FAANGs, even though they were average developers. For the record I’ve never gotten into FAANG (although I have gotten into FAANG-adjacent) so my evaluation should be taken with a grain of salt.
The best interview is one that can’t be gamed. Just like Hollywood auditions. Give the people the script and see how well they do. Tell the interview candidate you are going to give them 3 questions and they pick the one they want to do, even if they know it already. This strips away all pretense that they are figuring it out on the spot. At least then you are on a level playing field.
It's not about understanding the set of rules, it's about applying it in real time while playing. It adds a huge mental burden on the interviewee if they aren't familiar with it already. And the older you get, the harder it becomes.
>Failing to understand this simple set of rules would disqualify a person from most jobs ever invented.
I think you dramatically over-estimate most people's literacy with games and movement in a virtual space, let alone most people's hand-eye coordination. How many people can touch type these days?
You literally only place your fingers on the WASD keys and you move your character in a 2D (!) virtual space. With Minecraft it would be a completely different story, but Factorio is really accessible.
> You only have to understand how to move your character using WASD, the concept of Inventory and pointing at different places on the ground with a cursor.
You make it sound so easy. Once I gave my PS4 controller to my dad to play a driving game. Even though he's a better driver than I am, it did not go well.
Using a controller for someone who has never used a controller for and mentally mapping the act of driving to it is probably more difficult then handing a mouse and keyboard to someone interviewing for a software development job.
Game controllers are different beasts altogether. It's always fun watching a hardcore PC gamer struggle with a DualShock (or the asymmetrical weird thing they use on Xbox) for the first time. I know what you mean, but Factorio is really much slower and much easier to pick up than a driving game on a controller. I'm not saying it's a great choice for a one-hour interview, but anyone should be able to do whatever they want in Factorio after a couple of hours.
Yeah it's not about "failing to understand", or "being able".
It's about having advantage if you are already familiar with similar things. It's about those few extra things that you don't have to spend a single thought on anymore, giving you more headspace to think about the problem at hand.
And it's also about feeling less stressed because you are in a rather familiar environment.
Can you really not see how such an interview would bias towards Gamers?
I don't think it would have a huge bias towards gamers in general. the real issue would be people who already happen to be good at factorio. if you play with experienced people, it's easy to pick up on a lot of efficient patterns without really understanding why they work.
I have zero interest in sports and am clueless to the rules of baseball or football or basketball. I hated when I got questions that assumed I had any such knowledge.
Granted this was way back when when interviewers like to toss brainteaser questions to cargocult Google/Microsoft/etc...before they started doing leetcode to cargocult Google etc.
If you're actually D tier, and you're good at Factorio, why are you a D tier? You clearly have the mindset for problem-solving. What is holding you back?
Software development as a career is a lot more than just learning, design, applying logic. We still have to manage projects at a people/organizational level, interact with team members with various backgrounds, understand what people are asking of you.
There's a lot to being a successful developer that is probably so easy for many individuals that they might not consider that a person might be IQ smart, but unable to handle such a task at a professional level.
I'm in a similar boat to the the GP: I do really well on logic tests and programming challenges, but I struggle with the the soft skills necessary to be a good team member.
I'm not a programmer, I only know enough C to do simple embedded micro designs. Electronics is my jam, so I probably could get much better if I took the time.
It might, if they were actually suggesting this. Title was intended to race to the top of Hacker News.
>What's the takeaway from this? I don't know. We certainly can't switch to using Factorio as an interviewing method - you might as well just give a candidate a take-home assignment.
I don't know that I agree. I don't really play games so I would be a bit lost here, but if you needed me to design and start coding a large distributed system I would happily oblige while he watched...
I don't think there is any silver bullet way to do technical interviews. The idea that you can extrapolate a few hours of exposure to a person into a reliable predictor of future performance is somewhat silly to me. Any technique you use is going to be biased for people that are good at "X", where "X" is a tiny subset of what you need in an employee.
Obviously, you still need to do interviews, but keep some perspective when deciding. Don't overweight any one thing.
I see what you mean, but in my experience there's this tendency where you think you're testing for "X", but you aren't really.
A whiteboard FizzBuzz might be actually testing for memorization, deliberate interview prep, extrovert tendencies, etc. And not for any kind of actual technical skill. Failing the FizzBuzz could also just mean high social anxiety.
I heard good advice for life and interviews (be they for work, marriage, etc).
For life, it's that you aren't your weaknesses but your strengths. Don't worry about filling the skill hole in juggling for instance, put that time into your strengths or hobbies. Be a better you, not a more complete someone else.
For hiring though, if you've assured basic competence you're now more interested in weaknesses than strengths. You don't need the world's best person in the role, you need someone who won't bring down the group - either in their incompetence, malaise, nasty behavior, or whatever.
So when being interviewing for a role don't worry about being the best, shoot for 60% and you're golden - but do your best to avoid showing even a single red flag.
An amusing thought experiment but obviously it would have catastrophic consequences in real life... Pretty much anyone can get good at the game by watching Let’s Plays or reading guides.
I agree with the sentiment, but the ability to study (read guides) and demonstrate understanding/retention of the material by actually doing something (playing Factorio) is a good signal.
Honest question: is that fundamentally any different than building skills by imitating StackOverflow solutions or studying principles and patterns to build knowledge? I would view those as respective software development analogues, though there's definitely room for disagreement, particularly for very deep problem and solution spaces.
The conclusion of the article somewhat contradicts the title:
"What's the takeaway from this? I don't know. We certainly can't switch to using Factorio as an interviewing method - you might as well just give a candidate a take-home assignment. At the very least, we can do better than whiteboard interviews."
So it's not a description of how to use Factorio for interviewing, beyond a simple assignment of Factorio tasks to seniority levels. It does however make an interesting comparison between Factorio and software mechanisms.
I seriously think that "personal projects" are very underrated when it comes to hiring. If I see that someone has a lot of personal projects and has lots of time in Factorio they probably know how to code. I'm not sure you can rank them as Junior/Senior too effectively as these two things don't require much leadership, but you can basically skip the trivial phone interview questions at this point.
often in multiplayer factorio playthroughs you get one player who becomes the defacto "lead dev". it takes some nontrivial organization to get subgoals completed in a way that people aren't blocking each other. a common mistake is two players building separate factories too close together, which leads to some very messy solutions and limits how much the throughput can be upgraded.
Why wouldn't I want to take every possible factor into consideration? If I have two people, one of whom has a github with lots of personal projects and the other one does not, I'm naturally going to get a better sense of the github developer in terms of their overall capabilities.
That doesn't automatically make him the better candidate but it definitely gives me more information to act on.
If people don't want to or can't work on personal projects then that's fine, but I'm not going to ignore the people who are passionate about building their own stuff.
Not unreasonably. Like it or not, at else being equal someone with more current experience is probably the better employee.
There's this feeling that employers should just take the next person who fits some quota because to pick and choose is to be biased which has somehow become a bad phrase. My employer literally pays me to be biased. My job skills include having appropriate and helpful biases.
I doubt jobs for urban planners ask what cities you design in your free time. I doubt jobs for lawyers ask what cases you've argued in your free time. Why do we ask what software you've built in your free time for software engineering interviews?
Because in software personal projects are easy to do. It just takes drive and skill, which are both relevant skills. Passion as also valuable, it tends to show that people are willing to learn and have interest in the type of work that they will be doing.
Artist and even urban planners regularly have a portfolio, I don't see how this is any different.
It also takes the privilege of time. For example, a single childless person likely has more time for personal projects than a single parent of 3. This does not mean one is a better candidate than the other.
> Passion as also valuable, it tends to show that people are willing to learn and have interest in the type of work that they will be doing.
You can have both of those things without "passion".
Why not ask? Programming is a craft, and having a showcase could demonstrate initiative, capabilities, and inventiveness. They're concrete samples of your work.
Aside, there seems to be a swath of people who seem bizarrely resentful of personal projects. Yes, we get it, not EVERYONE in the universe has sufficient free time. Bla bla bla, maybe cut down on Netflix.
> Aside, there seems to be a swath of people who seem bizarrely resentful of personal projects
I wouldn't say anyone is resentful, just rightfully annoyed. It's not bizarre. I just don't think we as an industry should require everyone dedicate every waking moment to working. It's not healthy. Surgeons don't do surgery in their free time and are not asked about it in interviews.
> Yes, we get it, not EVERYONE in the universe has sufficient free time. Bla bla bla, maybe cut down on Netflix.
I think the implication of not having enough free time is that a person does not have enough free time to do things like watch enough Netflix to free up enough time to work more.
I would argue that if you consider your personal project to be a form of work, then something is wrong. If the principal reason that you're working on your own personal projects is to be able to pad out your resume, then I agree with you.
I don't think surgery is a good analogy. Engineering is a field where you're actually producing something, such as a product, that you can share and be proud of.
It's a great tool for junior developers. If someone hasn't had a dev job before, then a personal project -- even something simple like a twitter bot -- is almost a requirement in my eyes. IME, it is a strong signal for capability at that skill level.
Most higher-level devs don't build personal projects. I've worked with plenty of ridiculously capable people who work 9-5 and go home to their family. If anything, diligently keeping set hours is probably a stronger signal for a good senior candidate than personal projects are. A good senior candidate should be doing the thing they want to do already (thus, no need for side projects), or be capable of getting everything they need done for their job in 8 hours or less.
isn't the best technical interview inherently just doing the job itself? find some aspects of the job that's representative of it and have the candidate do it - if they can, then you're hired.
the issue is with technical jobs it's difficult to surmise what exactly is "representative" to begin with. I imagine the only reason candidates aren't just asked to work in the actual job for say, 8 hours instead of doing say, 7 irrelevant 45min interviews is because of intellectual property.
I wonder if Google or another large company has ever tried to re-interview all employees with more than say, 4 years of experience and correlated their interview performance with past job performance.
When I was responsible for recruiting interns, I always wanted to use TIS-100 as an interview tool, or more likely a modified version of TIS-100 with different instructions/architecture to not give an advantage to people that have "played" it before.
It would test their ability to read documentation and apply what they've read to problem. It also doesn't rely on them memorizing frameworks/API's etc. And it even in the early "levels" you can ask them to optimize for program size or cycle-count. Unfortunately no one would let me do it.
It’s unclear if this article is satire or not. Given that you will be spending 20 hours of at least two people’s time (one candidate, at least one employee), the space of alternative interviews is quite large. You could, for example, spend 20 hours observing them perform an actual software task. This piece doesn’t consider alternatives at all, merely asserting that this particular 20 hour game is the best that can be done. I see no reason to believe that comparing the play of a practiced factorio player to that of a brand new player would carry much signal at all. That means that a company carrying out this type of interview would need to ask candidates to study by playing the game.
This winter break I spent an embarrassing amount of time on Satisfactory on Steam, which is essentially a nicer (and probably easier) modern refresh of Factorio.
While I played, I came to similar conclusions to the article. Building factories in this game is very very similar conceptually to building software. I ran into exactly the same problems that I ran while I build software like:
- If I don't plan ahead enough the scope and layout of the factory, I end up with a spaghetti mess that is very difficult to rectify (technical debt)
- If I plan too much ahead, I overwhelm myself without even starting. The amount over-engineering and over-preparation becomes counterproductive and demoralizing.
- Starting new factories is fun. Maintaining and extending a factory that is starting to show serious design issues is a chore. I always tend to want to scrap the factory and start fresh.
- While designing a factory, modularization is key. You can go with a "monolithic" factory where you provide all possible materials as input, and try to build everything as output. It is very efficient transportation wise, and can centralize all management, but it can and it will become an unmaintainable mess. You can also design factories as "microservices", where each factory is a very compact, clean and scoped. It will only produce nails, or rubber, or copper wire. When you need to increase production of that item, you just duplicate the module (horizontal scaling). It seems fantastic at first, but the issue is now transportation. Dozens of micro-factories have to communicate with themselves to combine and produce more complicated items. The physical distance makes planning transportation a logistic and construction nightmare. So you have to find the right compromise between monolithic and micro-services.
I think I agree with the article that you can extract a lot of information about how good a person can be at software development by the way he plays Factorio/Satisfactory. Not so practical though :)
If you enjoyed Satisfactory you might also want to try out Dyson Sphere Program. It doesn't provide the veritical building but it is a different take on the factorio/satisfactory style of gameplay.
I tried DSP, very fun game but the performance is terrible. i7-9700K and RTX2080 and I get 20 FPS before using even 1/10th of the first planet. Can't recommend it currently.
I've only played the demo of factorio, but on the same computer (AMD Ryzen 5, RX540) DSP runs much better for me - there's a noticeable slowdown on save now that I've covered about half my starting planet but other than that it's stutter free.
Interesting, while i haven't built a sphere yet I have factories going on multiple planets and haven't noticed any performance issues. I'm on a a 2700k with a 970GTX. Not suggesting you don't have issues, but perhaps there is a fix?
Satisfactory is really great, highly recommended from me as well. I think the verticality really adds a lot of fun over Factorio where you are restricted to keeping everything on the same level. Can make some really confusing spaghetti though...
The loss of 3D was one of the biggest things I missed when moving on from industrial mods for minecraft (the inspiration for factorio).
I recently got an RTX card and played around with the minecraft RTX demos. I decided to look further into behavior packs, and apparently the scripting API is much more capable than it previously was. The community of people making content is basically nonexistent, but I now think it's just because people have moved on.
Everything is in place to have added blocks, oregen, ticking tile entities, power generation/distribution, multiblocks, etc. It would be nice to see a behavior pack that picks up the torch of gregtech/industrialcraft.
Unfortunately, most of the unofficial GT versions have lacked proper vision. That is, vision in a way that is not aligned with GT proper, and what really draws players of industrial games in.
Anyway, here is a promising example. I hope to see more industrial content for minecraft for windows 10.
Yeah, I love both factorio and satisfactory, but factorio ended up being the one I dumped more hours into.
Satisfactory ended up hiding too many details as you move around - the first person view and the very large structures made it really hard to untangle the mess.
Satisfactory felt like writing regexes - Basically write only, hard to read and parse afterwards.
That said, man the first person view is fun when you're actually doing the building. Just not nearly as fun once things have gone wrong and you need to debug.
Don't know if I agree on the height issues... every machine has ladders, they give you a tower fairly early on, and you can always walk on conveyors
As for debugging, I've found that planning a build on paper, keeping in mind production ratios, eliminates the need to experiment and debug, and then building simply becomes figuring out how to route conveyors
I've never had issues troubleshooting by getting to a high place and looking at the big picture, but to be honest since I started planning my builds on paper it barely even comes up
If that's the analogy, then I don't think it captures the core difficulty of software, which is unknown unknowns. That is, you write something that depends on an external system working a certain way, you follow its spec and depend on it working that way, and then it just ... doesn't, and you have to find exactly where it deviates, possibly making up some complicated kludge.
To capture that, there would have to be e.g. some hidden logic about which direction the output comes out that you have to deal with and work around until you understand it.
IANAProgrammer, to some extent I create my own GIGO issues which are like small instances of poorly defined behaviour. Sometimes you'll realise your whole world has ground to a halt and it's a conveyor backed up because one item arrived at an inserter and blocked it.
I play a bit of DSP but have at least 3 different mitigations which are akin to input sanitising. However, if you just stick a filtered inserter by a conveyor to grab errant materials then it can comeback and bite you ... though I've found either I can route them back to the proper channel or I can get away with the technical debt by throwing enough storage at them and leaving it until the thing needs refactoring.
You can have too much realism! I like that every green-circuit board works, rather than getting failures of systems that use that item with some random errors (that increase of your closer to the sun, say). I think that would be too frustrating.
I think this part is captured by Satisfactory "Tier" system. As you progress in the game, you unlock Tiers which allow you to build new materials and structures. The issue often is that you optimize the hell out of your factories to build something that is needed at you current Tier, and when you unlock the next one, you suddenly have to refactor everything to build the new items, that require different ratio of materials and inputs.
There's not a lot of unknown unknowns in factorio, but one thing which it really captures about software development is your own decisions coming back to bite you, or emergent complexity from notionally simple rules. As your factory grows you'll find yourself wishing you'd built things differently because suddenly something needs to expand beyond what you expected or some part of it is right in the way of a train network. Train networks are really tricky to make work effectively, it's very easy to have deadlocks appearing if you make a mistake, and you might not notice until you suddenly don't have any power because the trail with the coal is stuck waiting at a fouled up junction or something like that. There's a lot of opportunity to debug problems as they appear even in something you designed entirely yourself (and in a multiplayer game it's even more so because you might be detangling someone else's build).
A lot of the process of learning the game is basically learning patterns to reduce this. At the higher level of megabase design it's basically about designing a modular system so you can scale things out as efficiently as possible (and there's lots of possible variations of this).
I think this could potentially be the Biters - the enemies in the game that will periodically attack your base. You can guess that they will attack your base, but you are never sure quite when and it's always a concern in the back of your head.
I haven't played Satisfactory since the water update, so I have no idea if my approach still works, but IMO the easiest way to design an efficient factory is to not bother trying to balance anything, and scale out in only one direction as needed. Feed everything along a central stack of belts, and multiplex inputs/outputs via splitters/combiners so any individual machine line will always have throughput.
It doesn't make for a very exciting build, but it's very rote and dependable.
Factorio is very much like programming on a good team, automation, expansion, refactoring, tooling used to solve your problems.
Satisfactory is cool, but at no point do you get the "ok so I built this entire factory and now I can just replicate it modularly" - no, just place every, single, block, by, hand.
To me factorio is a rethinking of the RTS game, satisfactory is like a really nice minecraft mod.
I think its possible, and I might switch over my opinion if such a thing existed - it'd still be hard to optimize it to build worlds of "factorio level" complexity.
Now, granted, they might change their minds -- but it's hard to imagine how blueprints would work. Factorio is played on a 2D plane, with few obstacles in general and none that can't be removed; Satisfactory is played in a 3D world with immutable obstacles.
The game is tuned towards not needing blueprints. You don't need large quantities of anything, just a small amount of every item -- the complexity scales up, but the scale itself kinda doesn't. Yes, some players will attempt to turn the entire output of the map into turbo-motors, producing exactly the right amount of every ingredient, and will build their factory in the sky to avoid dealing with the terrain--
And yes, blueprints could be useful in this specific case. But that's not most players.
Factorio is (kind of) infinitely scalable. There is no max map size, and the game engine's performance is so incredibly good that it can support absolutely insanely massive factories. You can go as big as you can imagine, and the game will keep up.
It's also ridiculously stable, in the ~thousand hours I've played I've never had a crash or a game breaking bug.
Meanwhile satisfactory puts you on a fixed-size map and will slow down to a crawl once you build too much. This limits a lot of the things you can do in the game.
The simple answer is scale. The recipes in satisfactory and factorio are pretty similar in complexity, but the production chains in factorio are quite a bit larger. Also a lot of complexity comes in with scaling production up in factorio, since you largely have to scale horizontally, but in satisfactory, most of the scaling is vertical. Later, you can do more horizontal scaling, but even into the mid-game, you're pretty constrained by the amount of resource locations you have access to.
I play both. Satisfactory has more game in the sense of doing things besides designing a factory. Factorio is close to pure factory building, whereas it's like 70% in saisfactory.
Satisfactory has more elements of a traditional open-world action game (that just happens to borrow factorio's factory system).
I think Satisfactory is “nicer” in that it’s prettier and more user-friendly, has higher production values, etc. Not to speak for the parent comment but I took “nicer” to be a very deliberate choice instead of “better.”
It is true that Factorio is currently a better game and I suspect that will be the case even when Satisfactory is complete. I don’t really think it’s intended to be a 3D Factorio (even if that’s the easiest way to describe it): it seems that the design is to be less intricate as a construction management game, but with more details in the world design and player exploration: there are elements of No Man’s Sky or Astroneer that are absent in Factorio. So I am still curious to see where the devs will take it.
Yes that's what I meant. I don't know factorio very well, "nicer" meant just that Satisfactory is more focused on being a pretty game with cool graphics and world exploration, and less "hardcore" on the automation stuff.
I've got to the "I probably need to leave my starter solar system in order to increase my DS production rate" stage, and honestly I think that's my limit. I find I can't recall where everything is (I think that can be fixed to some extent) and I'm not about to start writing documentation for a game!
The latest update in Experimental (in Early Access on April 13) adds Drones for point-to-point delivery without needing connecting belts or rails or whatever. They're pretty low-bandwidth so you can't use them to transport raw materials at any scale, but if your "microservices" are sufficiently focused at a higher tier, you can tie them all together pretty easily with Drones now.
At least, in theory. Drones are T8 tech, and I'm not _quite_ there yet to start putting that theory into practice. But that's the stated intent, at least.
Also dipped my toes into Satisfactory this winter - after spending over 2,000 hours in Factorio.
Games like these challenge your ability to manage complex systems. Remembering which parts of your system processes which data (aka, "materials"). Finding and addressing bottlenecks in production lines. Maintaining and upgrading components, while creating new production lines using the techniques you learn. Managing time spent refactoring old systems vs replacing them with new ones. Etc, etc, etc. Planning properly - as you mention - is extremely critical to building a good factory.
However, despite over a decade of software development experience, and some time in Factorio, my first several play throughs in Satisfactory where an efficiency disaster, and even my recent ones were an ugly mess of spaghetti for the first few days of gameplay.
It took several playthroughs for me to grok the mechanics well enough to build a somewhat efficient factory. If my first couple tries were reviewed in a job interview I highly doubt I'd get the job.
There are strong similarities between these games and the mechanics software development, but like any new system, it takes time to create a true intuitive understanding of the mechanics and demonstrate them in front of others. If you're going to interview someone for a job, you're better off testing their ability to play the game that they'll be playing on a daily basis if they get hired: "Software Development".
Software Development, the game!
Want to play a game where you will never run out of new content? Where you are constantly challenged by new issues that'll haunt your dreams and make you lose sleep? Software Development is the game for you! With an ever changing landscape, where components and frameworks are updated daily! That's right, DAILY! This MASSIVE-multiplayer-online game never turns off. There are actively hundreds of thousands of players right now!
And you know what the best part is? Software Development is not "Pay-to-Play" like all those other games that try to steal those valuable dollars out of your pocket.
No, in Software Development YOU get PAID to play. That's right! All you have to do is find a company, nail a job interview, and enjoy playing the game you love while they hand you buckets of REAL LIFE MONEY that you can spend on other games that you have to pay to play. And also, food and rent and stuff...
Factorio runs on Windows, Mac and Linux; Satisfactory only runs (currently) on Windows. Factorio is the better option, though Satisfactory is a great game non-the-less.
The major difference between Factorio and software is you can't refactor in Factorio. Your options are to spaghetti or start fresh. You can't really pick and choose good pieces of your layout because of physical restrictions and the game is a giant math equation, and you want inputs to match outputs at every step of the way - if you don't have the space to do this you don't have the space.
It's like if in the software world you could greenfield, waterfall, or spaghetti. There is no agile in Factorio, which is broadly accepted as most commonly the best way to develop a piece of software.
Why not? You clear out the part you don't like, and build anew
No, code refactoring does not fit perfectly in this analogy. And your ability to do so is somewhat dependent on how you build. But I think all those factors are simply part of the puzzle in refactoring a factory
It's exactly like refactoring. You can replace a component that does something with a different structure for accomplishing that. Of course, if the system was tangled, then you may need to refactor other components first (read: relocate them to give room). The structure of the base around something is going to place limits on the refactoring. If the base is so horrible that you decide to gut it entirely, it's just like a rewrite.
The efficiency of "refactoring" in Factorio is like refactoring code using notepad while typing with your toes and you're only allowed to compile the source once you're done. Which is to say: on a completely different level!
You can absolutely refactor in factorio...in fact you have to refactor as you get access to better technologies. Just like software, refactorability is largely dependent on how well you design your base at the beginning. If you leave plenty of room for expansion and compartmentalize different production areas, it makes it extremely easy to upgrade with minimal disruption to the factory. And also like software, if you don't put any thought into the design at the beginning, it will probably be easier to start from scratch than try to redesign on the fly.
Ultimately how well your base is designed is completely based upon what things you need to produce in the future, what their inputs are, what their outputs are, what those outputs will be inputs into, and in which quantities you need to produce all of this stuff.
These are all things a new player has no clue about. All you're really saying is an experienced player that has done enough researching and equation balancing (X miners feeds Y furnaces and a line of belt can support Z miners) ahead of time should be able to design a base that doesn't require too much demolishing. And presumably, not overdesign in unnecessary base components.
It assumes perfect knowledge.
The closest to refactoring in Factorio is the ability to use bots to tear down and build blueprints. But you get that ability long after it would first be useful - that would be around when you get plastics, which easily triples the size of your base.
Ultimately how well your program is designed is completely based upon what features you need to develop in the future, what their parameters are, what their return values are, what those return values will be arguments into, and in what scales you need to run all of this stuff.
These are all things a new programmer has no clue about. All you're really saying is an experienced programmer that has done enough researching and equation balancing (X algorithms feeds Y threads and a processor can support Z cores) ahead of time should be able to design a program that doesn't require too much changes in the future. And presumably, not overdesign in unnecessary micro optimizations.
It assumes perfect knowledge.
The closest to refactoring in programming is the ability to use IDEs to remove dead code and do code generation. But programers learn about IDEs long after they would first be useful - that would be around when you get to integrations, which easily triples the code base of your program.
Everything you mentioned is also true about programming
This is the reason why I don’t like playing factorio. I’d rather make real software and get paid or build something usable vs a game version of it. I spend a lot of time already doing it as my job, and it stresses me out in a similar way.
I like factorio because I don't have to have 17 meetings to get alignment on changing 3 lines of code. "I objectively need more electricity, so I will refactor the nuclear reactor" is the only agreement needed
I honestly stopped after the winter holidays for this reason. At one point, it felt a bit too similar to actually working... but I enjoyed it very much until the "spaghetti factory point of no return" :)
I found refactoring demo... GildedRose to work well. It is super fun for people and they can talk through it. It helps surface how they are thinking about code, definitely not the something that can answer all the questions but reduces tension a lot with candidates.
Click-Bait title, and judging by comments here everyone went for the click-bait.
The author doesn't actually use factorio for interviewing, and thinks it world be a bad idea to do so.
A more accurate title would be "Comparison between Factorio and Software Engineering Concepts" - but that wouldn't have gotten it to the top of HN so fast, nor gotten so many comments so quickly.
I believe we may be at a point where it is understood that headlines are created with the goal of attracting readers to the article, and it may no longer be necessary to point this out for each article we come across.
That's not even the case. The point of headlines is to attract readers to the comment section on facebook, reddit, twitter, HN, etc. Some small percentage of people who have become engaged in the discussion seeded by the title then click through and provide a bit of ad revenue.
What’s the limit? What if I write an article with a title like “Breaking: Google to deprecate its search engine, will get into selling beanie babies instead” but in reality the article is about the many different types of mayonnaise one can buy? I think it’s reasonable to expect the title to somewhat reflect the content of the article.
There's a difference between editorializing a headline vs having an article titled "X is the best", where the content is literally "Well X obviously isn't the best, that would be ridiculous. Also we have no idea what the best is."
When the headline is so obviously clickbait, it's probably more interesting than the actual article... And the comments here are likely more interesting as well, just based on the title.
So I no longer worry about people who haven't read the article. I just enjoy the discussions.
software engineering and factorio are cybernetics and systems theory. That's the overlap - dealing with causal loops, buffers, queue theory, feedback, all that.
It really pisses me off that we as an industry don't broadly know our roots and the actual theory that we are putting into practice when we work. The "coding is engineering or art" debate? Easy; it's systems engineering.
The closest we get as an industry to teaching what's under the hood is to deal with computer science, which is great, but data structures and complexity heuristics are only a small piece of the puzzle.
It's no great surprise that there's such a big problem with cargo culting when most of us don't explicitly know what it is that we're doing.
The fact that all of the game's functionality mirrors real-world situations when building system shows that you could just ask someone to build a scaled down system rather than asking them to play this game.
Any game sufficiently complicated as this will be akin to asking someone to write some example code in a language they've never used before, so you wouldn't be able to tell much anyway.
Just drop them into a highest difficulty game and see how they are at crisis management. And at prioritizing. You could even see what they do at peace times.. Using it for personality analysis sounds like a bad idea.
I have used and continued to use whiteboard coding in interviews. However I emphasize to the person that I am interviewing that there is no expectation that what they write on the whiteboard compile, or even be a real language. I'll usually do a quick demo and joke that my own white boarding looks like the bastard child of Python and FORTRAN. The point of the exercise is to have a design conversation and see if the person can think about algorithms or solving problems, not write valid code with a marker.
Do people still have the same level of disdain for this technique or am I misunderstanding the objections people have to whiteboard coding.
I don't think anyone is expecting syntactically valid code (or at least that would be huge red flag for me if I interviewed). You do get bonus points the more "real" and consistent your whiteboard code is though.
I think people dislike whiteboard interviews because there is a significant subset of people who do not perform well under pressure, combined with the fact that a lot of software development is not terrifically algorithm heavy. This means that it's a (by nature) uncomfortable task and does not adequately represent the value the developer can bring to the business. In my current role, I would much rather have someone who can show strong knowledge of dependency injection and unit testing on my team over the person who understands how to balance a binary tree from memory.
I really only use algo/code/sql whiteboarding for junior devs now, and most mid/senior devs get asked about some hard code bug they've solved, what their preferred software patterns are, and we whiteboard some architecture.
The problem is that algorithms were discovered by mathematicians/computer scientists doing proofs for 40 hrs / week. It isn't something that can be reasoned about. It must be learned and memorized. This forces people to study that which is already embedded in the tools they use to perform their job.
If you ask people to whiteboard out a system at a high level that accomplishes some task, then that can be reasoned about and better tests their understanding.
See that's what I do mostly. Most often it's "here's a table of data I have, how would you compute this report." Not things like inverting binary trees or implementing hash maps.
I have a similar pet theory that Baba is You has incredibly high predictive power for skilled application security people. It would be pretty heinous to hire based on that, of course.
357 comments
[ 5.2 ms ] story [ 310 ms ] threadSo really, this article is saying "live, self-directed coding is the best technical interview", which isn't very original.
There might also be anxiety around playing a game you don’t know well, which wouldn’t manifest in a language you were familiar with.
Honestly, this article is pretty superficial.
I don't see this mentioned in the article.
Do only statements that are measurable have value in your opinion?
If you're going to say "Methods X and Y pale in comparison to Method Z for evaluating W" and expect anyone to draw conclusions from it, one would hope for even just a little bit of evidence.
But on serious note, People who played the game could just hide the fact that they know the game mechanics. And act like they are catching up faster then normal person would.
Well, there is a counter strategy that works even in this situation.
The counter strategy would be to just know way more interview questions than the interviewer, and know the answer to every question that they have, so that they run out!
When I was full time applying for jobs, a couple years ago, it got to the point with me. It turns out that people interviewing others, only have a couple questions that they use for interviews, and it is not that hard to know all of them.
This is a universal problem with all interview techniques. Interested in any ideas for how to fix it.
It's even a problem with "personal interviews," because some people have had more experience with optimizing one-hour first impressions than others.
The best interview is one that can’t be gamed. Just like Hollywood auditions. Give the people the script and see how well they do. Tell the interview candidate you are going to give them 3 questions and they pick the one they want to do, even if they know it already. This strips away all pretense that they are figuring it out on the spot. At least then you are on a level playing field.
I think you dramatically over-estimate most people's literacy with games and movement in a virtual space, let alone most people's hand-eye coordination. How many people can touch type these days?
You make it sound so easy. Once I gave my PS4 controller to my dad to play a driving game. Even though he's a better driver than I am, it did not go well.
Can you really not see how such an interview would bias towards Gamers?
I don’t think anyone is arguing that McKinsey has a bias towards gamers.
then again, I guess that's also true of software.
I have zero interest in sports and am clueless to the rules of baseball or football or basketball. I hated when I got questions that assumed I had any such knowledge.
Granted this was way back when when interviewers like to toss brainteaser questions to cargocult Google/Microsoft/etc...before they started doing leetcode to cargocult Google etc.
There's a lot to being a successful developer that is probably so easy for many individuals that they might not consider that a person might be IQ smart, but unable to handle such a task at a professional level.
I'm in a similar boat to the the GP: I do really well on logic tests and programming challenges, but I struggle with the the soft skills necessary to be a good team member.
Because these actually have very little in relation.
>What's the takeaway from this? I don't know. We certainly can't switch to using Factorio as an interviewing method - you might as well just give a candidate a take-home assignment.
But turning off the locals for your factory is probably not what you want to do.
Obviously, you still need to do interviews, but keep some perspective when deciding. Don't overweight any one thing.
Or don't even try and make X not even intersect with anything you need.
It still puzzles me how many people opt "don't even try".
A whiteboard FizzBuzz might be actually testing for memorization, deliberate interview prep, extrovert tendencies, etc. And not for any kind of actual technical skill. Failing the FizzBuzz could also just mean high social anxiety.
They're conceptually similar to software bugs - inevitable, often surprising and impossible to fix all at once.
For life, it's that you aren't your weaknesses but your strengths. Don't worry about filling the skill hole in juggling for instance, put that time into your strengths or hobbies. Be a better you, not a more complete someone else.
For hiring though, if you've assured basic competence you're now more interested in weaknesses than strengths. You don't need the world's best person in the role, you need someone who won't bring down the group - either in their incompetence, malaise, nasty behavior, or whatever.
So when being interviewing for a role don't worry about being the best, shoot for 60% and you're golden - but do your best to avoid showing even a single red flag.
"What's the takeaway from this? I don't know. We certainly can't switch to using Factorio as an interviewing method - you might as well just give a candidate a take-home assignment. At the very least, we can do better than whiteboard interviews."
So it's not a description of how to use Factorio for interviewing, beyond a simple assignment of Factorio tasks to seniority levels. It does however make an interesting comparison between Factorio and software mechanisms.
That doesn't automatically make him the better candidate but it definitely gives me more information to act on.
If people don't want to or can't work on personal projects then that's fine, but I'm not going to ignore the people who are passionate about building their own stuff.
There's this feeling that employers should just take the next person who fits some quota because to pick and choose is to be biased which has somehow become a bad phrase. My employer literally pays me to be biased. My job skills include having appropriate and helpful biases.
Artist and even urban planners regularly have a portfolio, I don't see how this is any different.
It also takes the privilege of time. For example, a single childless person likely has more time for personal projects than a single parent of 3. This does not mean one is a better candidate than the other.
> Passion as also valuable, it tends to show that people are willing to learn and have interest in the type of work that they will be doing.
You can have both of those things without "passion".
Aside, there seems to be a swath of people who seem bizarrely resentful of personal projects. Yes, we get it, not EVERYONE in the universe has sufficient free time. Bla bla bla, maybe cut down on Netflix.
I wouldn't say anyone is resentful, just rightfully annoyed. It's not bizarre. I just don't think we as an industry should require everyone dedicate every waking moment to working. It's not healthy. Surgeons don't do surgery in their free time and are not asked about it in interviews.
> Yes, we get it, not EVERYONE in the universe has sufficient free time. Bla bla bla, maybe cut down on Netflix.
I think the implication of not having enough free time is that a person does not have enough free time to do things like watch enough Netflix to free up enough time to work more.
I don't think surgery is a good analogy. Engineering is a field where you're actually producing something, such as a product, that you can share and be proud of.
Most higher-level devs don't build personal projects. I've worked with plenty of ridiculously capable people who work 9-5 and go home to their family. If anything, diligently keeping set hours is probably a stronger signal for a good senior candidate than personal projects are. A good senior candidate should be doing the thing they want to do already (thus, no need for side projects), or be capable of getting everything they need done for their job in 8 hours or less.
the issue is with technical jobs it's difficult to surmise what exactly is "representative" to begin with. I imagine the only reason candidates aren't just asked to work in the actual job for say, 8 hours instead of doing say, 7 irrelevant 45min interviews is because of intellectual property.
I wonder if Google or another large company has ever tried to re-interview all employees with more than say, 4 years of experience and correlated their interview performance with past job performance.
It would test their ability to read documentation and apply what they've read to problem. It also doesn't rely on them memorizing frameworks/API's etc. And it even in the early "levels" you can ask them to optimize for program size or cycle-count. Unfortunately no one would let me do it.
While I played, I came to similar conclusions to the article. Building factories in this game is very very similar conceptually to building software. I ran into exactly the same problems that I ran while I build software like:
- If I don't plan ahead enough the scope and layout of the factory, I end up with a spaghetti mess that is very difficult to rectify (technical debt)
- If I plan too much ahead, I overwhelm myself without even starting. The amount over-engineering and over-preparation becomes counterproductive and demoralizing.
- Starting new factories is fun. Maintaining and extending a factory that is starting to show serious design issues is a chore. I always tend to want to scrap the factory and start fresh.
- While designing a factory, modularization is key. You can go with a "monolithic" factory where you provide all possible materials as input, and try to build everything as output. It is very efficient transportation wise, and can centralize all management, but it can and it will become an unmaintainable mess. You can also design factories as "microservices", where each factory is a very compact, clean and scoped. It will only produce nails, or rubber, or copper wire. When you need to increase production of that item, you just duplicate the module (horizontal scaling). It seems fantastic at first, but the issue is now transportation. Dozens of micro-factories have to communicate with themselves to combine and produce more complicated items. The physical distance makes planning transportation a logistic and construction nightmare. So you have to find the right compromise between monolithic and micro-services.
I think I agree with the article that you can extract a lot of information about how good a person can be at software development by the way he plays Factorio/Satisfactory. Not so practical though :)
Satisfactory is freakin incredible and it is easily the best new game I have played in the last 10 years.
I could never get into Satisfactory with the 3D graphics, but I absolutely love Factorio.
I recently got an RTX card and played around with the minecraft RTX demos. I decided to look further into behavior packs, and apparently the scripting API is much more capable than it previously was. The community of people making content is basically nonexistent, but I now think it's just because people have moved on.
Everything is in place to have added blocks, oregen, ticking tile entities, power generation/distribution, multiblocks, etc. It would be nice to see a behavior pack that picks up the torch of gregtech/industrialcraft.
Unfortunately, most of the unofficial GT versions have lacked proper vision. That is, vision in a way that is not aligned with GT proper, and what really draws players of industrial games in.
Anyway, here is a promising example. I hope to see more industrial content for minecraft for windows 10.
https://mcpedl.com/advanced-machinery/?cookie_check=1
Satisfactory ended up hiding too many details as you move around - the first person view and the very large structures made it really hard to untangle the mess.
Satisfactory felt like writing regexes - Basically write only, hard to read and parse afterwards.
That said, man the first person view is fun when you're actually doing the building. Just not nearly as fun once things have gone wrong and you need to debug.
As for debugging, I've found that planning a build on paper, keeping in mind production ratios, eliminates the need to experiment and debug, and then building simply becomes figuring out how to route conveyors
I've never had issues troubleshooting by getting to a high place and looking at the big picture, but to be honest since I started planning my builds on paper it barely even comes up
If that's the analogy, then I don't think it captures the core difficulty of software, which is unknown unknowns. That is, you write something that depends on an external system working a certain way, you follow its spec and depend on it working that way, and then it just ... doesn't, and you have to find exactly where it deviates, possibly making up some complicated kludge.
To capture that, there would have to be e.g. some hidden logic about which direction the output comes out that you have to deal with and work around until you understand it.
I play a bit of DSP but have at least 3 different mitigations which are akin to input sanitising. However, if you just stick a filtered inserter by a conveyor to grab errant materials then it can comeback and bite you ... though I've found either I can route them back to the proper channel or I can get away with the technical debt by throwing enough storage at them and leaving it until the thing needs refactoring.
You can have too much realism! I like that every green-circuit board works, rather than getting failures of systems that use that item with some random errors (that increase of your closer to the sun, say). I think that would be too frustrating.
A lot of the process of learning the game is basically learning patterns to reduce this. At the higher level of megabase design it's basically about designing a modular system so you can scale things out as efficiently as possible (and there's lots of possible variations of this).
It doesn't make for a very exciting build, but it's very rote and dependable.
Also - build the entire thing high up in the sky.
Satisfactory is cool, but at no point do you get the "ok so I built this entire factory and now I can just replicate it modularly" - no, just place every, single, block, by, hand.
To me factorio is a rethinking of the RTS game, satisfactory is like a really nice minecraft mod.
Now, granted, they might change their minds -- but it's hard to imagine how blueprints would work. Factorio is played on a 2D plane, with few obstacles in general and none that can't be removed; Satisfactory is played in a 3D world with immutable obstacles.
The game is tuned towards not needing blueprints. You don't need large quantities of anything, just a small amount of every item -- the complexity scales up, but the scale itself kinda doesn't. Yes, some players will attempt to turn the entire output of the map into turbo-motors, producing exactly the right amount of every ingredient, and will build their factory in the sky to avoid dealing with the terrain--
And yes, blueprints could be useful in this specific case. But that's not most players.
Which is ironic considering a big part of the original inspiration for Factorio was factory building mods for minecraft.
It's also ridiculously stable, in the ~thousand hours I've played I've never had a crash or a game breaking bug.
Meanwhile satisfactory puts you on a fixed-size map and will slow down to a crawl once you build too much. This limits a lot of the things you can do in the game.
Satisfactory has more elements of a traditional open-world action game (that just happens to borrow factorio's factory system).
It is true that Factorio is currently a better game and I suspect that will be the case even when Satisfactory is complete. I don’t really think it’s intended to be a 3D Factorio (even if that’s the easiest way to describe it): it seems that the design is to be less intricate as a construction management game, but with more details in the world design and player exploration: there are elements of No Man’s Sky or Astroneer that are absent in Factorio. So I am still curious to see where the devs will take it.
Ha, I have a dsp.yml file for exactly that :D
At least, in theory. Drones are T8 tech, and I'm not _quite_ there yet to start putting that theory into practice. But that's the stated intent, at least.
Games like these challenge your ability to manage complex systems. Remembering which parts of your system processes which data (aka, "materials"). Finding and addressing bottlenecks in production lines. Maintaining and upgrading components, while creating new production lines using the techniques you learn. Managing time spent refactoring old systems vs replacing them with new ones. Etc, etc, etc. Planning properly - as you mention - is extremely critical to building a good factory.
However, despite over a decade of software development experience, and some time in Factorio, my first several play throughs in Satisfactory where an efficiency disaster, and even my recent ones were an ugly mess of spaghetti for the first few days of gameplay.
It took several playthroughs for me to grok the mechanics well enough to build a somewhat efficient factory. If my first couple tries were reviewed in a job interview I highly doubt I'd get the job.
There are strong similarities between these games and the mechanics software development, but like any new system, it takes time to create a true intuitive understanding of the mechanics and demonstrate them in front of others. If you're going to interview someone for a job, you're better off testing their ability to play the game that they'll be playing on a daily basis if they get hired: "Software Development".
Software Development, the game!
Want to play a game where you will never run out of new content? Where you are constantly challenged by new issues that'll haunt your dreams and make you lose sleep? Software Development is the game for you! With an ever changing landscape, where components and frameworks are updated daily! That's right, DAILY! This MASSIVE-multiplayer-online game never turns off. There are actively hundreds of thousands of players right now!
And you know what the best part is? Software Development is not "Pay-to-Play" like all those other games that try to steal those valuable dollars out of your pocket.
No, in Software Development YOU get PAID to play. That's right! All you have to do is find a company, nail a job interview, and enjoy playing the game you love while they hand you buckets of REAL LIFE MONEY that you can spend on other games that you have to pay to play. And also, food and rent and stuff...
Download now at, the internet.
It's like if in the software world you could greenfield, waterfall, or spaghetti. There is no agile in Factorio, which is broadly accepted as most commonly the best way to develop a piece of software.
No, code refactoring does not fit perfectly in this analogy. And your ability to do so is somewhat dependent on how you build. But I think all those factors are simply part of the puzzle in refactoring a factory
These are all things a new player has no clue about. All you're really saying is an experienced player that has done enough researching and equation balancing (X miners feeds Y furnaces and a line of belt can support Z miners) ahead of time should be able to design a base that doesn't require too much demolishing. And presumably, not overdesign in unnecessary base components.
It assumes perfect knowledge.
The closest to refactoring in Factorio is the ability to use bots to tear down and build blueprints. But you get that ability long after it would first be useful - that would be around when you get plastics, which easily triples the size of your base.
These are all things a new programmer has no clue about. All you're really saying is an experienced programmer that has done enough researching and equation balancing (X algorithms feeds Y threads and a processor can support Z cores) ahead of time should be able to design a program that doesn't require too much changes in the future. And presumably, not overdesign in unnecessary micro optimizations.
It assumes perfect knowledge.
The closest to refactoring in programming is the ability to use IDEs to remove dead code and do code generation. But programers learn about IDEs long after they would first be useful - that would be around when you get to integrations, which easily triples the code base of your program.
Everything you mentioned is also true about programming
The author doesn't actually use factorio for interviewing, and thinks it world be a bad idea to do so.
A more accurate title would be "Comparison between Factorio and Software Engineering Concepts" - but that wouldn't have gotten it to the top of HN so fast, nor gotten so many comments so quickly.
So I no longer worry about people who haven't read the article. I just enjoy the discussions.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
I've replaced the title with representative language from the article body.
https://hn.algolia.com/?dateRange=all&page=5&prefix=true&sor...
It really pisses me off that we as an industry don't broadly know our roots and the actual theory that we are putting into practice when we work. The "coding is engineering or art" debate? Easy; it's systems engineering.
The closest we get as an industry to teaching what's under the hood is to deal with computer science, which is great, but data structures and complexity heuristics are only a small piece of the puzzle.
It's no great surprise that there's such a big problem with cargo culting when most of us don't explicitly know what it is that we're doing.
Any game sufficiently complicated as this will be akin to asking someone to write some example code in a language they've never used before, so you wouldn't be able to tell much anyway.
Has anybody ever played less than 4hr of factorio?
I have used and continued to use whiteboard coding in interviews. However I emphasize to the person that I am interviewing that there is no expectation that what they write on the whiteboard compile, or even be a real language. I'll usually do a quick demo and joke that my own white boarding looks like the bastard child of Python and FORTRAN. The point of the exercise is to have a design conversation and see if the person can think about algorithms or solving problems, not write valid code with a marker.
Do people still have the same level of disdain for this technique or am I misunderstanding the objections people have to whiteboard coding.
I think people dislike whiteboard interviews because there is a significant subset of people who do not perform well under pressure, combined with the fact that a lot of software development is not terrifically algorithm heavy. This means that it's a (by nature) uncomfortable task and does not adequately represent the value the developer can bring to the business. In my current role, I would much rather have someone who can show strong knowledge of dependency injection and unit testing on my team over the person who understands how to balance a binary tree from memory.
I really only use algo/code/sql whiteboarding for junior devs now, and most mid/senior devs get asked about some hard code bug they've solved, what their preferred software patterns are, and we whiteboard some architecture.
If you ask people to whiteboard out a system at a high level that accomplishes some task, then that can be reasoned about and better tests their understanding.
Considering the average number of per-month applicants for software developer positions is probably less than one, you ought to have that much time.