Be took this approach, for what it's worth -- they sold two models of BeBox, and all developers were given the slower one.
They pointed this out very proudly in their launch pitch for BeOS, a pitch which focused as much on performance / concurrency advantages of BeOS over the then-lagging MacOS as it did on the novel API features of the OS.
Don't know whether that anecdote counts as for or against the idea, but it's certainly been tried...
I'd say for. A fully loaded Be desktop could easily weigh in at under 300MB. That would count as a "lightweight" Linux distro. BeOS was a lean machine.
I still have my BeBox, if only because back in the day I got a hell of a lot of good things done with it (reverse engineering of strange file systems, mostly) and without question I can still kick out some fully functional, performant, code with the thing. Old computers may be slow, but there is a hell of a lot of bloat in our computer worlds these days.
The reality is your programmers should instead have a low-level understanding of what their code does, and how it performs. They should then be using profilers and monitoring tools to ensure it's behaving inline with their expectations. One of my strengths is always being able to give a rough estimate as to how long a task should take before I've written it. The only surprise usually is if I find ways to make it even faster than I assumed was possible, by finding clever techniques to use.
I would add that in addition to having that low-level understanding, they should also care about what their code does and how it performs.
That's where it tends to break down for me. For a lot of people in McProgramming jobs, they don't care, and rely on Moore's law to handle the rest, or mark it off as technical debt that can be paid off at some unknown time in the future (and that time never seems to arrive).
I'd say give them a virtual machine image to run with certain (slower) specs, and ask them to periodically try the app out on that. Their main dev machine needs to be fast though.
If you can get past the "condescending excuse to save on hardware costs" bit there's actually a glimmer of truth to this. People are generally more motivated to solve problems they can directly experience and therefore understand. Question is: can you get developers to develop on a fast machine but consistently use an emulator/testing environment that's slower? If they are motivated and disciplined I would say yes. But it's worth a thought.
Yeah, it would make sense to test things on a slow machine, but having a slower machine is like increasing a lumberjack's output by giving him a dull axe.
While I agree with you, the thing with developers these days is that they will see an ad that says "We give our developers the best and fastest machines to do their work", and they won't care about the reasoning behind using slow machines.
I would bet that there's even a segment of developers who actually rely on Moore's law to overcome performance issues in their apps at release time.
In a game post-mortem, one of the ways the team was motivated to maintain acceptable performance was to add a picture of their perf-dev in the corner of the game in varying states of angriness. The lower the framerate, the angrier he looked. It helped personalize the frustration gamers would feel when a developer introduced a feature that caused poor performance.
With a desktop/web app, this is harder to do, but the idea could still have merit - surface perf data so developers can see the clock-cycle impact of their features.
There is a glimmer of truth to this idea, assuming developers are being lazy about performance, but I still think it will reduce productivity.
My experience: I work at a high frequency fund. We do not skimp on performance - I rewrote Java's Math.exp for speed, and I rewrote our data store reader (already optimized to be read sequentially) to use multiple cores. Nevertheless, slow performance == downtime. I should be writing code right now. Instead, I'm waiting for a simulator to finish testing my strategy. I've probably wasted hundreds of hours this year waiting for tasks to finish, in 5-15 minute chunks.
I think there is an easier solution. If your developers are being lazy about performance get new developers. If that really isn't an option, pay them more if the code runs faster.
If that really isn't an option, pay them more if the code runs faster.
The problem with this, is that any benchmark can be gamed. However, if your software runs daily batch processes, you can keep track of processing time and take the average time per transaction. Measuring actual performance in the production environment would be much harder to game.
Since the problem is lazy programmers, we are probably discussing regime of doing sums in php rather than in the db. In this regime, legitimate performance improvements are usually easier than gaming benchmarks.
It's just better policy to ensure that some engineer like Wally thinking, "I'm gonna code me a minivan!" actually benefits the company if the plan is executed.
Why is programming management so stone-age? I realize it's been often compared to herding cats, but are things really so bad that a simple problem gets a "lazy programmers" and "get new ones" response? And from a management point of view, in any other domain it _would_ be a simple problem.
Imagine complaining that your architects don't make your houses energy-efficient enough. So you get new architects?! No... you talk to them, you set standards, you check their designs or have managers check them... you solve the problem with the many management tools developed over the years.
But when it comes to programming, we barely see this at all, at least not in small companies. Why?!
If I had a dollar for every equivalent of "what do you mean slow, it's not when I try with 3 of <something that the app should process in thousands> on my overclocked XXX with 16 GB of RAM and 4 hard disks in 10000 rpm and RAID" I've heard or read I'd be very rich now.
The only competition they have is the other bad developer's all-time favorite "but it works and it's fast when you delete the database and start from that."
There are more bad than good developers (and managers!) in the world. They won't care to make anything faster than what's
being "acceptable" for them. Case study: Adobe Reader.
Having a slow machine available to test your code on is a nice option, but I'd be very annoyed if I was forced to use it as my development environment.
Then again, it depends on what you're doing. I have a pretty slow netbook that I use when I travel. Working in Ruby or Java can be painful, but working in Lua feels exactly the same as on my Macbook Pro.
Not good enough. We have automated performance testing on older machines here. There was a case where something that ran blazingly fast on the developer's machine completely bombed the automated performance test on an older machine. In fact it ran several orders of magnitude slower. It turns out that the older machine had a smaller and less associative cache. On the developer's machine there were no memory accesses in the innermost loop, but on the older machine it was hitting memory several times in the innermost loop.
This is a case where the rated MIPS difference was less than a factor of 4, but the performance difference was orders of magnitude.
The converse is true-- you can get your developers to write slow and bloated code by giving them top of the line machines. Fast machines are good for quick builds, but taking the trouble to test the code somewhere else is usually too much.
edit-compile part of the development cycle should take place on the best machine possible (or machines -- see http://linux.die.net/man/7/icecream), but the run/test part of the cycle should take place on the target machine.
Let's see, I spend extra time compiling, my debug version runs a lot slower while I try to find the bug, and I get laughed at by people working at other companies. Not to mention crap machines might not even meet the specs of the development tools.
Thank $DIETY that this guy doesn't run a shop for embedded programmers.
We solved this problem for web development for orbs.com by doing the following:
Everyone has fast dev machines.
We access the site through a local domain name like orbs-sunny.com referring to my personal computer; however, this is routed through Nginx with throttling on to simulate a slower external connection.
If we need a faster connection for certain types of testing like uploading 100MB files, we use orbs-sunny-fast.com. This goes directly to our machine and skips nginx.
It is not perfect in that it doesn't simulate a slow computer; however, it's good enough to make everybody pay attention to speed.
We did this because speed related issues were often ignored at the time of original development. We found that the slowdown on execution hasn't slowed development since our app is supposed to be responsive anyway. If it's not responsive, it's much better to fix it while your head is in that space. This results in faster overall development.
I remember reading a quote that claimed that the reason Windows UAC is so messed up is because all developers run as admins on their Windows machines, so they never bother to make their programs run without admin permissions. Opposed to, say, Linux, where the default is to run without root priviliges, and only run as root for the things you really need.
"Robert Garrett, from Princeton, competed in several events. The discus was unknown in America so he had a blacksmith fashion one like those in Greek art. It weighed 20 pounds -- much too heavy to throw. Once in Greece, he found that a modern discus only weighed five pounds. He practiced with one for a few days. Then he dethroned the Greek champion. Garret came home with four gold medals."
I am sort of confused. If you ask them to improve performance do they just ignore you?
I know with the current project I am on, we have a very specific cpu and memory usage requirement. This specific requirement is silly in what the customer demanded, but it is a requirement and it is our job to make it happen. There is no option to ignore it.
Don't give your developers an option. Set specific goals based on your target environment. The product is not ready to ship until it meets those goals. At the same time you have to be reasonable. Your goals must be based on what the applications are actually doing. If they are very graphics heavy, you may need faster hardware than if its just a standard office style app. Giving them slower hardware will never make impossible things possible.
I don't have anything measurable, but I do work on a slow, glitchy VM with a bloated application on an old machine in some of my projects.. It takes in the several minutes for me to test any changes to the codebase.
It's possible that under the right conditions this might work (though I doubt it..), but in my situation the code has been steadily degrading due to refactoring being too painful of a wait, and having the time for a coffee break between any code change shakes any semblance of concentration on what you're working on straight out of you.
You might be better off deploying to a slower test machine and have it run some time-tests there.
My guess is the answer is probably yes, if you give your developers slower machines, they will start optimizing things for performance.
However, I suspect they won't spend the time optimizing what you want them to. Good developers HATE being inefficient. Sitting around waiting for something to compile gets your out of the flow of writing and testing code.
So instead of spending all their time optimizing the app, I suspect they'll spend a disproportionate amount of time optimizing the build process.
Not really the outcome you want...
Now, as you suggest, if there's a way to slow down only the running of the app, and not the rest of the development process, that would be useful and probably would cause the sort of changes you're looking for.
47 comments
[ 4.7 ms ] story [ 108 ms ] threadOh wait, no you can't.
They pointed this out very proudly in their launch pitch for BeOS, a pitch which focused as much on performance / concurrency advantages of BeOS over the then-lagging MacOS as it did on the novel API features of the OS.
Don't know whether that anecdote counts as for or against the idea, but it's certainly been tried...
The reality is your programmers should instead have a low-level understanding of what their code does, and how it performs. They should then be using profilers and monitoring tools to ensure it's behaving inline with their expectations. One of my strengths is always being able to give a rough estimate as to how long a task should take before I've written it. The only surprise usually is if I find ways to make it even faster than I assumed was possible, by finding clever techniques to use.
I would add that in addition to having that low-level understanding, they should also care about what their code does and how it performs.
That's where it tends to break down for me. For a lot of people in McProgramming jobs, they don't care, and rely on Moore's law to handle the rest, or mark it off as technical debt that can be paid off at some unknown time in the future (and that time never seems to arrive).
I would bet that there's even a segment of developers who actually rely on Moore's law to overcome performance issues in their apps at release time.
With a desktop/web app, this is harder to do, but the idea could still have merit - surface perf data so developers can see the clock-cycle impact of their features.
My experience: I work at a high frequency fund. We do not skimp on performance - I rewrote Java's Math.exp for speed, and I rewrote our data store reader (already optimized to be read sequentially) to use multiple cores. Nevertheless, slow performance == downtime. I should be writing code right now. Instead, I'm waiting for a simulator to finish testing my strategy. I've probably wasted hundreds of hours this year waiting for tasks to finish, in 5-15 minute chunks.
I think there is an easier solution. If your developers are being lazy about performance get new developers. If that really isn't an option, pay them more if the code runs faster.
The problem with this, is that any benchmark can be gamed. However, if your software runs daily batch processes, you can keep track of processing time and take the average time per transaction. Measuring actual performance in the production environment would be much harder to game.
Imagine complaining that your architects don't make your houses energy-efficient enough. So you get new architects?! No... you talk to them, you set standards, you check their designs or have managers check them... you solve the problem with the many management tools developed over the years.
But when it comes to programming, we barely see this at all, at least not in small companies. Why?!
The only competition they have is the other bad developer's all-time favorite "but it works and it's fast when you delete the database and start from that."
There are more bad than good developers (and managers!) in the world. They won't care to make anything faster than what's being "acceptable" for them. Case study: Adobe Reader.
Then again, it depends on what you're doing. I have a pretty slow netbook that I use when I travel. Working in Ruby or Java can be painful, but working in Lua feels exactly the same as on my Macbook Pro.
This is a case where the rated MIPS difference was less than a factor of 4, but the performance difference was orders of magnitude.
Paul Grahm - sentence 7 http://www.paulgraham.com/13sentences.html
:)
Thank $DIETY that this guy doesn't run a shop for embedded programmers.
Everyone has fast dev machines.
We access the site through a local domain name like orbs-sunny.com referring to my personal computer; however, this is routed through Nginx with throttling on to simulate a slower external connection.
If we need a faster connection for certain types of testing like uploading 100MB files, we use orbs-sunny-fast.com. This goes directly to our machine and skips nginx.
It is not perfect in that it doesn't simulate a slow computer; however, it's good enough to make everybody pay attention to speed.
We did this because speed related issues were often ignored at the time of original development. We found that the slowdown on execution hasn't slowed development since our app is supposed to be responsive anyway. If it's not responsive, it's much better to fix it while your head is in that space. This results in faster overall development.
Sunny
I remember reading a quote that claimed that the reason Windows UAC is so messed up is because all developers run as admins on their Windows machines, so they never bother to make their programs run without admin permissions. Opposed to, say, Linux, where the default is to run without root priviliges, and only run as root for the things you really need.
From http://www.uh.edu/engines/epi1137.htm:
"Robert Garrett, from Princeton, competed in several events. The discus was unknown in America so he had a blacksmith fashion one like those in Greek art. It weighed 20 pounds -- much too heavy to throw. Once in Greece, he found that a modern discus only weighed five pounds. He practiced with one for a few days. Then he dethroned the Greek champion. Garret came home with four gold medals."
Get punchier mixes by listening to wimpier speakers while you mix.
I know with the current project I am on, we have a very specific cpu and memory usage requirement. This specific requirement is silly in what the customer demanded, but it is a requirement and it is our job to make it happen. There is no option to ignore it.
Don't give your developers an option. Set specific goals based on your target environment. The product is not ready to ship until it meets those goals. At the same time you have to be reasonable. Your goals must be based on what the applications are actually doing. If they are very graphics heavy, you may need faster hardware than if its just a standard office style app. Giving them slower hardware will never make impossible things possible.
It's possible that under the right conditions this might work (though I doubt it..), but in my situation the code has been steadily degrading due to refactoring being too painful of a wait, and having the time for a coffee break between any code change shakes any semblance of concentration on what you're working on straight out of you.
You might be better off deploying to a slower test machine and have it run some time-tests there.
However, I suspect they won't spend the time optimizing what you want them to. Good developers HATE being inefficient. Sitting around waiting for something to compile gets your out of the flow of writing and testing code.
So instead of spending all their time optimizing the app, I suspect they'll spend a disproportionate amount of time optimizing the build process.
Not really the outcome you want...
Now, as you suggest, if there's a way to slow down only the running of the app, and not the rest of the development process, that would be useful and probably would cause the sort of changes you're looking for.