Isn't the joel test less about pass/fail and more about communicating and encouraging certain things that make a better environment for developers? I don't really care if they "pass" the joel test or not, but I do find the answers to the questions helpful. I sure as hell wouldn't want to work in a place with no source control.
I agree. Also, I don't know if I would consider a 12/12 the only way to "pass," if you want to think about it that way. A 10/12 is a pretty good score, but I think it is more about the transparency and knowing what the job offers and what it doesn't.
"A score of 12 is perfect, 11 is tolerable, but 10 or lower and you've got serious problems. The truth is that most software organizations are running with a score of 2 or 3, and they need serious help, because companies like Microsoft run at 12 full-time. "
That middle part is where the tests run. On Careers for example we have a decent set of Selenium tests that run through the core functionality of the site on every dev build (Can I Login? Can I buy something? Can I post a job? If I have a profile and I go to the profile page is it there? type stuff).
Selenium can handle a lot of gui issues with different browsers. There are also tools to compare images (could be used with screenshots).
Yes ultimately you need human eyes and on the product. It also helps to have someone with a QA mentality, who has a developed intuition for breaking things and finding edge cases. But rigorous use of automated testing, code review, and dogfooding greatly reduces the need for dedicated QA.
I don't think I'd be speaking out of school to mention that a) StackExchange and Fog Creek are different companies with their own products, processes, and engineering cultures and b) engineering best practice is a moving target especially since the way we deliver software has changed so much since the days of "gold master which costs $X million and 12 months if we screw up anything on it." If all of your software is web apps delivered over the open Internet, and you have traffic like egads, then you can use a different set of engineering practices to ensure that the proportion of customers inconvenienced by bugs is essentially the same as before without needing a dedicated testing phase/team/etc.
There are projects in Fog Creek which are organized around that philosophy, too. I think the Trello guys have written about it fairly extensively. The first thing Google came up with was http://www.joelonsoftware.com/items/2012/01/06.html -- search for "bugs."
I was thinking similarly (about the moving target). My ideas about dedicated testers have shifted over the last few years with a pronounced emphasis on shorter deployment cycles and unit tests, and I was surprised to see the up-to-date schedule bit in there at all!
Show me a project with an up-to-date schedule and I'll show you a project that has a tool for declaring success in the face of failure.
In my, albeit short, experience I have found that the "do you have testers?" and "do you fix bugs before writing new code?" to be overkill for most consumer-facing web applications.
They are very important if the software is mission-critical or if there is some sort of physical cost to shipping patches. But if you can continuously update the live site its not that big of a deal if a bug gets out. It will be noticed soon, often by your users, and then you fix it and that is it.
This works if the developers and other stakeholders at the company are testing the product themselves and are responsive and can triage bug reports quickly.
Having an up-to-date schedule can often be unrealistic as there can be shifting priorities and developers should be flexible to move around and respond as necessary.
>But if you can continuously update the live site its not that big of a deal if a bug gets out. It will be noticed soon, often by your users, and then you fix it and that is it.
You consider this acceptable? "Don't worry, if there's a bug, our users will tell us, no need to test!"
Yes. Not all bugs are the same in terms of end-user impact and how they affect the overall performance of the system. You can expend a lot of resources trying to track down every single last glitch in many cases you should. However, if you have the luxury to be able to push updates and fixes at any time and you a free non-critical service, its better to just have a culture of fixing things quickly and allocating resources to more high-priority things like adding new features.
basically the Joel Test is out of date in a few areas. It reflects a Microsoft Excel development-centric view of the world. The way Excel was built and shipped in 1993 no longer reflects the way most software is built and shipped today.
Somewhat off-topic but why do companies feel the need to write:
"Stack Exchange, Inc. does not discriminate in employment matters on the basis of race, color, religion, gender, national origin, age, military service eligibility, veteran status, sexual orientation, marital status, disability, or any other protected class."
Presumably they are legally required not to discriminate in these matters, in which case it's rather pointless to announce it. They don't specifically state that they wont kill you and harvest your organs - but I assume that they don't (although Oracle might)
Or if these are areas where companies are allowed to discriminate then they are hardly likely to say so on their job ads.
It has nothing to do with legalities and everything to do with communicating the values that are part of the corporate culture. And it sends the message that if you have a problem working with/for people of a different color or sexual orientation, don't bother applying.
What really upsets me about both FogCreek (and StackExchange now) is the requirement of "permanent legal right to work in the US".
Joel used to write a shit ton of material about how they will save no effort to attract the best developers they can find, on how they need to impress candidates, on how they will bring the guy on a limo, etc, etc, etc... but if you are a foreign student graduating on a top school from the US, or if you already have a ton of experience but need a sponsor for your H1B, they will not even look at your application? For what? Saving the extra $5-10k incurred in the legal process?
IMO if you are truly, actually, really serious about hiring the best of the best, you'll hire them wherever they live. Moving to the USA shouldn't be required or necessary, particularly in software development.
This is why the Internet was invented.
Otherwise you're just hiring "the best people we could convince to move to {location} for {competitive salary}".
Orthogonal problems. In my case, they wouldn't need to convince me to move. They just need to help me find a legal way to stay.
Yes, they could just as well try to have people working remotely, but then it's another set of arguments that need to be taken. Joel never wrote about how working remotely is "just as good" as working in a office environment geared towards developers - quite the opposite. So I could understand why FogCreek/SE would require people to work locally, but I still don't get why they can't put some effort to sponsor visas.
I believe the visa issue is thorny, not just because of the money, but because of the logistics and lawyering and so forth.
It'd be easier for everyone involved to just acknowledge, like GitHub does, that remote devs can be extremely kick ass.
To be clear, Stack Exchange and Fog Creek are different animals in this area. But Joel is, at his heart, an "everyone must sit down to lunch at the same table or the entire company is doomed" kind of guy. Me, not so much.
The logistics and lawyering and so forth is something that you will have to deal once. After that, the process is entirely reproducible.
In fact, a company the size of FogCreek or SE should be able to just hire one immigration lawyer and get this done with.
In fees to the government, a H1B application costs less than $1k (~$2.5k if you get the "expedited" process to bring the response time from USCIS from 3 months to 15 days), so the $5-10k range should actually include more than enough hours from a half-decent office that can deal with getting the paperwork and filling all the requirements from the government.
The one thing that they could have used as an excuse not to do it: the fact that all the Googles and Microsofts around take all the quota of H1B applications and they are pushed out of it. That would've been valid until 2008, but since 2009 the quota hasn't been filled.
34 comments
[ 4.7 ms ] story [ 86.4 ms ] threadhttp://www.joelonsoftware.com/articles/fog0000000043.html
"A score of 12 is perfect, 11 is tolerable, but 10 or lower and you've got serious problems. The truth is that most software organizations are running with a score of 2 or 3, and they need serious help, because companies like Microsoft run at 12 full-time. "
If you are competing with microsoft in web services you can probably afford to score rather less!
Maybe the testers are unsuspecting users in the A/B pool of Facebook-style deployment where A = stable and B = mystery meat.
I'd personally rather work at a place that has tests vs. testers.
Yes ultimately you need human eyes and on the product. It also helps to have someone with a QA mentality, who has a developed intuition for breaking things and finding edge cases. But rigorous use of automated testing, code review, and dogfooding greatly reduces the need for dedicated QA.
There are projects in Fog Creek which are organized around that philosophy, too. I think the Trello guys have written about it fairly extensively. The first thing Google came up with was http://www.joelonsoftware.com/items/2012/01/06.html -- search for "bugs."
Show me a project with an up-to-date schedule and I'll show you a project that has a tool for declaring success in the face of failure.
They are very important if the software is mission-critical or if there is some sort of physical cost to shipping patches. But if you can continuously update the live site its not that big of a deal if a bug gets out. It will be noticed soon, often by your users, and then you fix it and that is it.
This works if the developers and other stakeholders at the company are testing the product themselves and are responsive and can triage bug reports quickly.
Having an up-to-date schedule can often be unrealistic as there can be shifting priorities and developers should be flexible to move around and respond as necessary.
You consider this acceptable? "Don't worry, if there's a bug, our users will tell us, no need to test!"
More eloquently stated: http://www.quora.com/Quality-Assurance-QA/When-is-not-having...
Ideally, most of the testing should be automated and engineers should be writing test cases as they build features.
It's just that all absolute statements about what you must do are wrong - except this one!
"Stack Exchange, Inc. does not discriminate in employment matters on the basis of race, color, religion, gender, national origin, age, military service eligibility, veteran status, sexual orientation, marital status, disability, or any other protected class."
Presumably they are legally required not to discriminate in these matters, in which case it's rather pointless to announce it. They don't specifically state that they wont kill you and harvest your organs - but I assume that they don't (although Oracle might)
Or if these are areas where companies are allowed to discriminate then they are hardly likely to say so on their job ads.
If you're interested in some applicable law, this is a good summary:
http://www.eeoc.gov/facts/qanda.html
Joel used to write a shit ton of material about how they will save no effort to attract the best developers they can find, on how they need to impress candidates, on how they will bring the guy on a limo, etc, etc, etc... but if you are a foreign student graduating on a top school from the US, or if you already have a ton of experience but need a sponsor for your H1B, they will not even look at your application? For what? Saving the extra $5-10k incurred in the legal process?
Not cool, FogCreek. Not cool.
This is why the Internet was invented.
Otherwise you're just hiring "the best people we could convince to move to {location} for {competitive salary}".
Yes, they could just as well try to have people working remotely, but then it's another set of arguments that need to be taken. Joel never wrote about how working remotely is "just as good" as working in a office environment geared towards developers - quite the opposite. So I could understand why FogCreek/SE would require people to work locally, but I still don't get why they can't put some effort to sponsor visas.
It'd be easier for everyone involved to just acknowledge, like GitHub does, that remote devs can be extremely kick ass.
To be clear, Stack Exchange and Fog Creek are different animals in this area. But Joel is, at his heart, an "everyone must sit down to lunch at the same table or the entire company is doomed" kind of guy. Me, not so much.
In fact, a company the size of FogCreek or SE should be able to just hire one immigration lawyer and get this done with.
In fees to the government, a H1B application costs less than $1k (~$2.5k if you get the "expedited" process to bring the response time from USCIS from 3 months to 15 days), so the $5-10k range should actually include more than enough hours from a half-decent office that can deal with getting the paperwork and filling all the requirements from the government.
The one thing that they could have used as an excuse not to do it: the fact that all the Googles and Microsofts around take all the quota of H1B applications and they are pushed out of it. That would've been valid until 2008, but since 2009 the quota hasn't been filled.