The study followed 49 undergraduates who had taken Java classes. I would argue that, because of this, the students were probably more experienced with static type systems.
A good point. Also, 49 programmers is very little. But even with thousands of developers using all sorts of languages, it's still moot to draw any conclusion from. It's very subjective and dependant on people's experience/taste/State-of-mind..
For me, switching from static (java) to dynamic typing (common lisp) has saved me a lot of development time. Or did it? Maybe it was Emacs? Maybe it was functional programming? Maybe it was the REPL? Maybe experience?
I think in a lot of cases programmers are happiest with their new hotness simply because they became better and more experienced programmers in the process of learning the new hotness, not because the new hotness is 'good' necessarily.
Using students (especially undergrads) is a typical problem of programmer productivity studies. N=49 and 27h of experimentation time per subject isn't a lot either. Doing a study that can draw actually useful conclusions may just be prohibitievely expensive.
only 20x? What happens when you reevaluate it with the python code not having any pieces that are C under the covers? (my understanding that a lot of python idioms for fast code basically turn into ah "use the approach that pushes as much work into C as possible")
Not in the same way at all.
Bytestring or hmatrix may have their primops in C, but the control flow stuff code for libs like Repa or Vector are pure Haskell (and in fact current state of the art work on Repa has it matching if not beating C code).
The crucial difference is that in python, these faster python idioms are for using C as the control code
Not to mention being able to rule out large classes of errors at compile time, and ease of refactoring large code bases. You mean I get all that for free? Where do I sign up?
Obligatory language argument: If by ease of refactoring you mean having to change everything whenever anything changes just to avoid a little messiness. :)
The paper's title is casting doubts on the positive impact of statically typed languages as they had a negative impact on development time. (though not quality)
In the experiments the existence of the static type system has neither a positive nor a negative impact on an application's development time (under the conditions of the experiment).
What's the point of dynamic languages if they don't even make the initial implementation faster? They're certainly harder to maintain afterwards.
We measured two different points in the development: first, the development time until a minimal scanner has been implemented, and second the quality of the resulting software measured by the number of successful test cases fulfilled by the parser. In none of these measured points the use of a static type system turned out to have a successful impact. In the first case, the use of a statically typed programming language had a significant negative impact, in the latter one, no significant difference could be measured.
Here's my pet unprovable theory. Development time is dominated by a programmer's understanding of a problem. Static languages force programmers to say a lot about their understanding of a problem.
Dynamic code can be very fast, but in lisp, after algorithms, that means actually going and finding all of the calls to elt and replacing them by calls to nth. With a static language the compiler has a much better chance of specializing the code for you.
In my own experience: if I write dynamic code I'm still thinking of types. Especially in Python.. What's mutable, what isn't... This is what my professors at school say too. Less typing (pressing keys) doesn't make you think faster!
Seconded. Even though i write python code, i am still thinking of types. Infact, it's one of my gripes with the language. There are some tired mental states, when i don't think/forget to think about types, but just write and call functions. It's at those times that i find having to go back and read the original function for a type or having to go to REPL for testing the type painful. I would be happy to have a compiler tell me what type is expected instead. I guess more experience will lead me to infer this from the actual error message itself.(ex: Nonetype has no function iter)
You should strive to not rely on type-checking, and trust and rely more on duck-typing. If at some point you need to ensure something is not None, test it and act accordingly in the alternate case. If something absolutely must not be None (or must have a specific method), ever, use assert() so that things blow up upfront where you can infer (or even read) the reason, and not deeper in the code where the actual error message, as you mention, has to be parsed according to the whole call stack to make sense of it.
Sometimes you may need to type-dispatch when you want to do smart functions. A typical example is a function that takes a sequence/iterable/generator of strings or a string. Since a string is iterable in python, you have no choice but to check for BaseString.
Duck typing requires more mental effort to track the variable types. And in python some type errors are detected only in runtime, like when you call a function with a wrong parameter count.
An interesting study, but really hard to compare. It's like attempting a discussion of the effectiveness/efficiency of Engligh vs French (using conjugations or sex).
And in particular, trying to compare the effectiveness of English vs. French by using 49 undergraduate students who weren't proficient in either language.
Well I think the article hints at the real benefits from type systems. They took longer in the beginning understanding the type system, but made up for it later on by being able to finish in the same amount of time. Static systems be balling for refactoring.
Hopefully for the next experiment they have them tackle a longer project, and have the programmers tackle short problems in multiple languages to 'rank' them.
What about maintenance? I've never heard a static-typing proponent claim they can develop faster. It's always that they can maintain the software with much less work and have less overhead at run time.
I don't know. I have an equal (equally small :() amount of experience with both. And, amusingly, I've disliked both.
What I found after using Java, Gosu (basically Java but less stupid), Python and JavaScript fairly significantly is that all of them have their own problems. In that particular set, I'd probably value a static type system a little more than a dynamic one, but it gets eclipsed by other language characteristics (e.g. support for functional programming).
In fact, thanks to being scarred by Java, for the longest time I really liked dynamically typed languages more. However, I later realized that it was the lambdas and the higher-order functions and the clever abstractions and the terseness that I liked more than the dynamic typing.
Once I learned Haskell (and later OCaml), I've been converted to liking static typing quite a bit. The beauty in these two languages is not just that the type system catches more bugs than Java's while being less of a burden, but also that they type system can actually make the code more expressive.
My favorite example of this is the read function from Haskell. It is the opposite of toString--it goes from a String to some value. However, the really awesome thing is that you do not need to explicitly specify what type to parse to--the type inference can figure it out for you! Imagine being able to use "5".fromString() the same way you use 5.toString(). In a dynamically typed language, you would have to somehow specify what to parse to explicitly. So your fromString code would something like Integer.fromString("5"), which is less nice and throws away the toString/fromString symmetry.
I have about the same experience with dynamic (Perl, Scheme, JavaScript, Python) and static (Java, Gosu, Haskell, OCaml) languages and I like good static typing most.
I lived almost entirely in Ruby from about 2003-2010. Going back to statically typed languages for iOS development has reminded me now nice it is to let the computer catch all my trivial mistakes.
On the rare occasion you really need the kind of anything-goes dynamism of Ruby/Python/JS you can just roll your own dispatch tables.
Studying individuals is the wrong experiment. Static typing benefits tooling most when you have larger groups of programmers who much collaborate, often asynchronously. It's easy to hold a mental model of the code you write in your head and minimize mistakes, it's harder to hold a mental model of a larger program consisting of code written by many programmers.
I'd like to see them hand a pre-written codebase of say, 10,000 lines to a bunch of students, and ask them to make large changes/new features to the codebase.
>Studying individuals is the wrong experiment. Static typing benefits tooling most when you have larger groups of programmers who much collaborate, often asynchronously.
Absolutely.
And also: static typing benefits is not about "development time" but about maintenance, adding stuff, refactoring etc.
If anything, dynamic typing would be expected to lead to faster development time, which is also why it's used in most prototyping.
Lastly: small sample. Did enough of them use an editor/IDE that could take advantage of a static type system?
i agree totally with both parents -- i wanted to comment the same...
static typing would more benefit:
* large groups of programmers
* large codebases
* maintenance (which is usually taking more resources then the initial development of a large code base)
aditionally i've come to believe that not all static typing is created equal. most allow `null` to be returned instead of adhering to the type, then some have exceptions and not many languages type side effects (like IO). i'm basically saying that C/C++/Java/C#'s kind of typing is not Haskell's kind of typing, and that the potential gains from Haskell's type system are much bigger on the long run (while it also comes with a steeper learning curve).
Did enough of them use an editor/IDE that could
take advantage of a static type system?
Just because the language is static, that doesn't mean it benefits from an IDE.
For instance, Scala has fairly good integration on top of Eclipse and IntelliJ IDEA, however every time I try doing some work in Scala, I end up cursing and screaming, because these Scala IDE plugins are slow, incomplete and unstable and get in my way. And I can't blame their authors, because Scala is a difficult language to deal with.
Another example would be C++, which has been around for a very long time and is one of the most popular programming languages ever. And yet even Visual Studio has problems with its refactoring/IntelliSense support.
And then there are the Smalltalk environments, still around, still kicking ass.
Here's the thing ... there are static languages, and then there are languages designed for usage within an IDE ;-) Java, C# and Smalltalk are like that, while Scala, Haskell and C++ aren't.
And good on them. But that doesn't mean it's easy to write large websites in dynamically-typed programming languages, it just means that they are very good at what they do.
- Multi-site development at least in 3 countries;
- Sometimes up to 100+ developers
- CI systems
- Source code of several MB of source code
- Enterprise like infrastructures for Fortune 500 companies
- Different skill sets from the "just out of the university" to the "top developer"
Maybe YouTube beats this, but Google only has Phd guys able to crack out crazy algorithms/data structures in minutes. Not typical in most software houses.
I agree dynamic languages result in smaller codebases, but that's only because they tend to be so much more concise than most statically typed languages.
Also, as for large and complex projects done by distributed teams, I don't have to point much further than Django or Plone to prove dynamic typing works well in that context.
The plural of anecdote is not proof. Heck, it's not even data.
Heck, the examples you give aren't even anecdotes. They're just name-dropping. One would have to be pretty familiar with the codebases in question and the history of their development in order to be able to give a clear assessment of what, if any, impact dynamic typing might have had on them.
Fair enough. They prove large projects built by distributed teams are possible in at least one dynamic language.
There was a time I was familiar with both codebases (I have some catching up to do) and that's why I mentioned them. Both projects carry heavy heritage and are experiencing huge pressures to evolve and both are doing very well (from what I hear on the dev lists).
Sure, but even then one merely has an observation that there are these two large projects that are written in dynamic languages and are being successfully maintained.
That doesn't imply that they are being successfully maintained because they are written in dynamic languages, or despite being written in dynamic languages. And of course there's also the possibility that static vs. dynamic is a wash and doesn't really have an impact at all. Or that the potential impact of going with static or dynamic is heavily influenced by other factors - does the static language have type inference, does the dynamic language support duck typing, stuff like that.
Long story short, correlation does not, in and of itself, imply causation.
In any case, it's mostly an apples to oranges comparison. Observing from my heavily biased toward web applications experience, there aren't as many Python codebases as there are Java ones at least in part because you can do a lot more with a small Python codebase than you can do with a similarly sized Java codebase.
By limiting the damage a single crappy programmer can do.
Everywhere people talk about 'large' teams maintaining a large codebase in this thread, substitute 'mediocre' teams stuck with a poor bloated codebase that is the vehicle for their ambitions. It's just not worth anybody's time to understand its unique needs in detail, especially since any improvement you make today risks being messed up tomorrow.
I can understand using static typing if you already have a large codebase that's statically typed, but is there any reason you'd start a project with a statically typed language?
Yeah the benefits of static typing are front-loaded at the start of a project. I might rewrite in a statically-typed language for performance if it ever needs it, but I wouldn't start statically-typed.
My lisp interpreter above allows me to tear out a lisp function and replace it with a C function, while leaving the unit tests untouched.
My big problem with this study is that the project sizes and teams are too small. Static typing really starts to get it's power as the team and project get bigger. This is especially true as the skill levels from larger teams start to spread from amazing all the way down to terrible. In smaller teams you can often work with higher quality developers, but with larger corporate systems, you don't have that luxury. You have to deal with all kinds of experience levels, etc. You need to be able to modulize the project, etc.
From my personal experience, dynamic typing usually works well with 1-2 developers and continues to be useful until about 4-5 developers (static typing too for that matter). After that, static typing usually wins pretty consistently.
My personal preference is static typing. I find it makes the code more readable and catches a lot of oopses early, at compile time. Otherwise you just end up writing a ton of test cases and hope to catch the same issues at runtime. So you have less protection (you have to catch it with a test), and you just end up spending more time writing tests anyways.
I do understand the benefits of both, I just prefer static typing because of the way I work. Both work and have their places. Like everything else, you need to use the right tool for the right job AND the right people.
I don't think the size (or the revenue) of the company matters as much as the size of the projects that you're working on within those companies.
Someone mentioned maintenance, so I will also ask how old the codebases that you're working on are? For dynamic typed languages, as the code grows older (with more bug fixes and add-ons and one-offs) then it gets harder and harder to maintain as 100% error free (let alone debug).
I will also throw in the fact that doing software library version upgrades on dynamically typed systems is a pain in the ass. Some code branches have system level calls changed, and unless you have very good unit test coverages (which may or may not need to be re-written on a library version upgrade) then you may or may not catch the problem during the upgrade. With a statically typed language, a static analysis tool can tell you exactly where and how the library or interface upgrade affects your codebase.
"I will also throw in the fact that doing software library version upgrades on dynamically typed systems is a pain in the ass."
This is very true, but you gain a lot with dynamic languages in other areas. The last company I worked for had about 300 programmers in the office I was in. Roughly half did Java and the other half did Ruby. I was one of the few that moved between both ruby and java projects. I did not see these great benefits of static typing that people always insist exist for large teams at large companies.
Have you also used dynamic languages extensively as part of large teams at your employer? Dealing with lots of people over long periods of time is messy, no matter what language/tool you use. I think the claim is just that static typing gives you better results than dynamic typing. All relative..
49 undergraduate students working independently on a small project over the course of a single week using a previously unseen language with no consideration of the performance or maintainability of the resulting code. I can't see how there are any meaningful conclusions to be drawn from this study other than "inexperienced programmers find learning new dynamically typed languages easier".
Not at all. When learning any new programming language there's going to be a period where you're working by trial and error. Removing compile-time typing generally means less syntax to learn which makes the feedback loop tighter which I would imagine gets you the point of familiarity quicker. The study seems to back that idea, but not much more.
It could also be the case that it is faster to write text parsers in dynamic languages as opposed to static ones. There are so many different variables to come to a meaningful conclusion. What about working on very large programs with other people?
I don't think you can show much of anything in 27 hours of development time. Neither the advantages nor the drawbacks of static typing will have any impact on productivity for such a small one off, single developer, throw away project.
It seems the only question this experiment could possibly claim to answer is whether or not the additional finger typing required by static languages affects the coding speed of these particular 49 undergrads.
The conclusion contains a paragraph that matches my initial reaction to the headline and opening. "It seems that the experiment at least does not contradict with a common believe that is often articulated in the area of type systems: that dynamically typed systems have a benefit in small projects while this benefit is reduced (or even negated) in larger projects." (It goes on, but I can't seem to copy actual text out of it, I typed that).
My feeling about this has always been that dynamically typed languages will save time on initial implementation and small projects, and statically typed languages will win on large projects and things that require more maintenance. They acknowledge this, and didn't find anything to contradict it.
Their own graphs seem to show that effect too. In some areas, dynamically and statically typed languages were equal, while in others dynamically typed languages had lower times. I don't actually see where the conclusion that a "static type system had no effect on development time" came from, it appears that it had an overall (if not very large) negative effect. Just like I would expect for a small (27 hours) project.
I didn't crunch the numbers though, maybe I'm just eyeballing it wrong. (again, I can't seem to copy-paste out of it, and I don't feel like reentering all of them)
Maintenance... Dynamic would be faster for me to write in from 'scratch' (I find it weird it has NO effect on dev time, but then again the study isn't really broad), however after your sprint(s) changing things is really much easier with static typing. That's what I feel at least, especially with Haskell (and Hoogle) (which unfortunately I don't get to use that often).
Meh, dynamically typed systems are within an order of magnitude of statically typed systems, even on benchmarks that greatly benefit from the static type information.
The study may or may not be flawed, but what's really interesting to me is the reaction. We need more science in our computer science, which means more experiments and more results like this. We should also be open to the truth that we use the tools we like because we like them rather than because they're technically superior, even though we pimp them ad nauseum as though they are.
I once read an article about a technique Intel had developed for improving cooling of processors by changing the shape of the fan. I related this to some of my co-workers. One of them proceeded to tell me that this can't possibly work, backing up his argument with "reasoning" based on off-the-cuff remarks about the way air and physics "must" work. The fact that Intel had actually done this seemed to have no effect on his eagerness to continue the "debate" about this scientific fact.
I have a hard time believing that I get no benefit from using Haskell over Smalltalk, but if a body of science were to appear that cast doubt on that belief, the appropriate thing to do is change the belief, not stand around debating from imagined first principles why all the science is wrong and can't be so. Shut up, design an experiment and go prove it!
Perhaps there's little of this kind of actual science in our computer science because it would mean asking hard questions and accepting difficult truths. "The prisoner falls in love with his chains."
I second that.It's one of the reasons i quit my day job to do a Master's degree...though i chose a naive enough college and course that am now back to a programming job where i am surrounded by the same type of crowd again.
Oh. Sorry, for that snarky reply(tired i guess). I have a master's in cognitive science(from india). And currently a python programmer here. Mainly because there aren't that many jobs(in cog. sci.) , here and i was not focused enough to get published in a journal during my master's. As for types, well, one ex: is a situation where i was trying to defend my choice of Dvorak kb saying usability and got a dismissive, snorty laugh as a response.
Software development is so complicated that there is always an endless supply of objections to fire at any study at odds with one's beliefs. And that is exactly how all these discussions go. All we're doing is repeating shibboleths.
The most interesting studies would be ones that changed somebody's beliefs. That doesn't happen very often in our field. Does it ever?
I've only scanned the comments, but the chief reaction seems to be exactly what you want for a scientific approach--people are arguing about the scope and methodology of the study. A paper like this is not some broad-reaching conclusion--it's very specific and based on some potentially flawed methodology. You want people to qualify exactly how specific it is and talk about potential flaws in its approach. That's how you improve the general knowledge.
Also, I suspect there are a couple of reasons such studies are uncommon in computer science. For one, CS isn't really a science; programmers and computer scientists are not trained in the scientific method or experimentation (beyond their general education); almost no CS papers I've read have contained empirical studies. If anything, they are closer to math papers than science papers!
Additionally, this sort of study is basically sociology. (Or something similar.) These sorts of fields are considered a little shady by hard scientists, and CS people tend to empathize more with the latter. I think this explains the immediate attack on methodology.
All that said, having more studies done about these questions would be great. I'm just not sure who's the best to do them. Maybe HCI researchers? I can't help thinking that the really intense PL people I know wouldn't be very interested in doing this.
"Almost no CS papers I've read have contained empirical studies."
You aren't reading the right papers then. At least in Software Engineering, you can't get into the main conferences (ICSE and FSE) without a pretty significant empirical study.
That's a broad brush, and though it's probably a good categorization, I don't find this distinction to be all that useful as a boundary. For instance, the field of artificial intelligence and cognitive psychology branched at one point, so much of my work in cognitive architecture and algorithmic modeling necessitates user studies. One would be hard-pressed to bucket AI into software engineering though. Likewise, in machine learning, I've seen a push from classical data-driven to modern "data-informed" approaches to analyzing these results. Computational linguistics (NLP) and computational narrative are yet additional examples of fields that often requires user studies or other empirical data.
More to the point, the distinction of what is and isn't computer science has become even more blurry in the research community because research in itself has become more inter-disciplinary. There seems to be little to gain from attempting to "bucket" research into distinct taxonomies.
The only problem is, people have been doing these experiments for 30 years, and do you know what the net effect it's had on the world of programmers: none at all. Saying "no no, this time really listen to this study" seems to be having no effect.
There are a lot of causes for this, not least of all the things you mention (nobody cares about science, people like or dislike based on non-scientific evidence).
But it's also because people know that all these studies are flawed. As much as I hate to be the "nitpicker" who takes apart studies, EVERY SINGLE STUDY on programming doesn't even come close to real-life scenarios. In fact, one of the few studies that people actually believe is the "some programmers are 10x better" study, and that was actually fairly well conducted - many students were given identical tasks, and a fairly large amount of time to do them.
But take a look at the Dynamic vs. Static argument. For years, the Dynamic-fans have been saying "Quicker to program, so it's better", while the Static-fans have been saying "Quicker to program, but harder to maintain, hard to use with large teams". So now we have a study that doesn't even come close to addressing most of the issues that have been argued for years! Of course this isn't going to convince anyone.
"In fact, one of the few studies that people actually believe is the "some programmers are 10x better" study, and that was actually fairly well conducted - many students were given identical tasks, and a fairly large amount of time to do them."
Can you by chance point me to this paper? I'd like to add it to my paper collection, since most of the studies I've seen concerning programmer variability use members of the workforce. I'm not aware of the one involving students, but such replication studies are easy to miss.
I'm not sure how that link helps me. I've already seen many of those studies. Which one satisfies, "students were given identical tasks, and a fairly large amount of time to do them"?
That is the specific study I am searching for to add to my list of papers. Did you give me this link because you were referring to Humphrey (A Discipline for Software Engineering), or something else? I can track down Humphrey, but it will take me a few days, since it's a physical book.
My apologies. I misread what you were looking for. Mostly by not actually reading what you wrote. :(
Perhaps the previous author was thinking of the Prechelt "An Empirical Comparison of .." paper? http://page.mi.fu-berlin.de/prechelt/Biblio/jccpprtTR.pdf . Section 5.7 has "work times" for Java and C/C++ programmers using well-observed times. However, that is not for a "fairly large amount of time."
There is no such study. It's a well-regarded and popular myth.
Laurent Bossavit does a masterful job of researching the origins of this myth (and others) in his new book The Leprechauns of Software Engineering, available on LeanPub [1].
Not at all, that study was not well conducted, it compared people just learning to program with people that already had experience, in a time when programming was much different from what it is today... Just go read the damn paper...
I don't know how science can help you here. In the case of the improved fan design, well that's easily testable.
However, I don't want to go down the rabbit hole and argue social vs natural sciences.
I find it dumbfounding that in this day and age, people can still create rather arbitrary social experiments with only 49 people and then think they can draw grandiose conclusions from their "data".
Unfortunately, user studies are difficult to conduct in computer science mainly because it is difficult to get users that you can study. Like it or not, 49 people is actually a pretty big study relative to what is out there. For better or for worse, I have seen much smaller studies accepted/published by top tier conferences.
The problem is that there are so many soft factors affecting the outcome that it's very hard to tell if even a great scientific study will apply to your case. The number of people on your team, their experience with the technology being used, their relationships between each other, the politics of the working space and thousands other factors may have much bigger effect on the productivity and quality then the choice of a typing system, and it may be pretty impossible to control for them in the study. I agree that it's good to approach things scientifically, but at the same time it's also good to question how much science applies to the problems being solved.
Counterexample: JavaScript. It is the best demonstration I know of that our choice of languages and tools is mostly based on historical accident rather than any technical criteria.
Agreed with your overall premise, but this particular study doesn't meet the criteria to have any conclusions drawn from its results.
The number of programmers is too small and skills not representative (49 undergrad programmers). The problem (writing a simple scanner / parser) isn't one that will really benefit from a decent type system. All you need need are ints, strings, and arrays and you're good to go.
Reasoning from basic principles is a valuable tool to evaluate conclusions. Sure, it has strong weaknesses (hidden assumptions which are wrong, insufficient imagination about what could happen, rationalizing one's biases). But it is useful, when you dont have complete trust in the quality or the scope of the experiment. For instance, claims about quantum computing solving NP complete problems are legitimately held in doubt because of theoretical reasons. Also, whenever there are short term positives hiding a long term negative, like say unsustainable financial or ecological behavior, the negatives might be only seen by a chain of reasoning and not by direct experiments.
I agree in the sense that we see so many cases in the other direction - reasoning full of holes being trusted over empirics. The interesting thing is in any given situation, how much trust to give to the different tools that we have to evaluate a claim.
No study can control all the variables enough to convince a fanboy that his favorite language isn't the greatest ever. However, the current state of PL research is as close to astrology as you can get. Why are we working on type systems and modules and concurrency primitives et. al. without a scrap of evidence that any of it contributes to programmer productivity? There's no science there.
In fact, everyone here should flip this around and ask: can you design a practical experiment to compare productivity of dynamic vs. static languages? Will others find it convincing? Probably not. PL advocates are no different than religious missionaries. They have no objective proof of any of their claims. And both will murder the natives if they don't convert.
Well, one thought that comes to mind is that the topic is essentially a subset of ergonomics, so perhaps experimental protocols should take a few more cues from ergonomics research.
For example, controlling variables has to be done by constructing artificial systems from the ground up. You can't just pull two commercial products off the shelf and then pretend you're examining the impact of only one of the hundered different things that differs between the two.
Similarly, if we wanted to compare the impact of dynamic vs. static typing, we'd have to make sure that that is the only variable. Which means you basically have to construct a new programming language from the ground up, so that you can easily create new dialects of it that differ in only one very specific characteristic.
PL research is not necessarily for "contributing to programmer productivity." Formalization of various programming language concepts into a type system[0] allows researchers to apply analysis and verification techniques. The simplest example I can think of is the Maybe type. Instead of having null pointers and, if you forget to check for NULL, getting a runtime error (segfault, NullPointerException, whatever), instead you would fail to compile since the "null-like" pointer would be a Maybe type, not the concrete value. You can't use it without unwrapping it. Concretely, the code:
int foo(int *p) {
return *p + 17; //well, probably something more complicated.
}
if p is NULL, this code doesn't work. In a language with a more expressive type system, we would write:
foo :: Maybe Int -> Maybe Int
foo p = case p of
Just x -> Just (x + 17)
Nothing -> Nothing
(Note: there are more succinct ways to write this example in Haskell, but I'm trying to illustrate the code.)
In this case, we have demonstrably covered every case. That's what an expressive type system gets you: you can completely preclude certain classes of bugs like null pointer dereferences by having a sufficiently expressive type system. It's the same in more relevant PL research: people are attempting to formalize systems so that whole classes of bugs can be removed at compile time. It's not about user case studies or anything like that: those can come later, when features look like they'd be useful to integrate into languages.
It looks like the featured article gives a case study about a pretty bad type system. A sufficiently expressive type system doesn't get in the way--it aids the programmer, not hinders her. Heck, in Haskell and ML, you don't even have write down types--the compiler will infer them for you. (It is Haskell practice to type-annotate toplevel functions anyway).
[0] When I say type system, I mean a static type system. For the purposes of this discussion, dynamically typed programs are statically typed, just with not-very-useful types.
You say: "when features look like they'd be useful to integrate into languages". How do you know a feature would be "useful" without compelling evidence that this is really a problem for programmers? Now you're back to the original problem of figuring out what the most significant problems are for programmers. Wouldn't it be better to figure this out BEFORE PL people plunge into a particular topic?
Remember all the research done on typestates? The motivation section of those papers was usually a few paragraphs of total BS. AFAIK, there was no real data nor experiment that demonstrated this was a real problem for professional programmers.
FYI: I like static type systems and used Haskell et. al. But no one has demonstrated that it is better than even Visual Basic!
Not really. The research, again, isn't in programmer usability. It's in formal logic and analysis of the semantics of computer programs. Benefits for programmers are just a side-benefit. The goal is advancing humanity's understanding of computer science, not in helping programmers, although sometimes the two goals are somewhat linked and deeper understanding occasionally yields industry benefits. In my example, and in many instances of expressive type systems, research has yielded tangible benefits for industry.
I recently talked to a friend of mine who expressed the frustration of not knowing what a function returns because it's a language without static typing. It was something I hadn't thought of before personally. I thought about it a bit and realized that as a Ruby programmer, I depended on two things to figure out what a function returns, and neither is the source or documentation (most of the time at least):
1) a REPL. Just give it a whirl and see what you get.
2) intuitive method names and good abstractions. If I
call @user.posts in a Rails app, I can assume I will get an Array-like object that I can treat as an array without any problems. The actual type does not matter 90% of the time.
I see where you're going but thats not really what we're doing with that ... @user.posts actually returns an array of posts and @user.posts.first returns an actual Post object.
You could also change @user.posts to be @user.contributions, for example, and that would hide the fact that contributions was just an array of Post objects.
I think we are talking about different things. Duck-typing doesn't help in depicting what the return type of a function is when looking at the function signature only. You need to look at the function implementation (the actual duck) to tell what the return type is.
My point I guess was that with proper abstractions and duck typing you really don't have to care about the type. I only notice something isn't a duck when I try to use it like a duck and it fails - which is surprisingly infrequent.
that's one aspect of dynamic typing that i strongly dislike. you can't figure out what a function does by looking at its signature. if documentation is absent (which is almost always the case), a coding task turns into a reverse engineering task.
A good repl that automatically prints out the function's heredocs is usually enough for me. I like bpython particularly because it shows the argument list and the heredoc as soon as you type the opening parenthesis for a function call. In other REPLs you can usually call a docs(fn) or a source(fn) to handle the rest.
> a REPL. Just give it a whirl and see what you get.
A REPL is in no-way tied to having a dynamically typed or interpreted programming language. E.g. Haskell's GHCi has an excellent REPL environment where you can, in addition to evaluating code, ask for the type of an expression and do other static analyses.
I agree that a REPL makes a huge (mostly positive) difference on how you write code.
> a REPL. Just give it a whirl and see what you get.
In a large system you might have to jump through a lot of hoops to get an accurate result from the REPL. What if the thing you're trying to puzzle out is 20 call levels deep? As a new Ruby dev I struggle with this.
In python if I was trying to figure out something complex 20 levels deep, I would stick in an "import pdb; pdb.set_trace()" to get a REPL at that spot. There's probably an equivalent for Ruby?
(Off-topic, the werkzeug debugger even lets you debug browser apps by jumping into a REPL anywhere in the stack trace, in the browser, after an exception is thrown. I think it also has a command that bakes you a pie.)
If it is a pure function it makes the call depth irrelevant. While not always the case, it seems to be pretty common in Ruby development to keep things as pure as possible.
No, it's not irrelevant. If you're trying to figure out what gets passed up the call stack, you need to trace it through. Being able to plug in your own values into a single method call via the REPL isn't helpful if you're trying to analyze what really happens when it runs. Either you need to run it under a debugger, or manually trace it through. In a static language you just have to read the type signatures.
As mentioned by JackC, if you need to understand the context in which a method is called, use a REPL. But to understand what a pure method does should be context free, by definition.
It seems to be very basic static typing along the lines what Java or C has stashed on top an Smalltalkesque object-oriented language, so I am not surprised, the many years I used dynamically typed languages I maybe once or twice had a type error of the kind "confusing int with string" (e.g. when selecting a number from database and it being returned as string).
For the benefits of static typing I would rather look into something like Haskell which has a much stronger notion of type and builds many language features around this notion. It's probably harder to design a reasonable study comparing that with for example Smalltalk, though.
I'd be more interested in comparing Haskell to one of those even stronger typed programming languages. I'm forgetting the name of them, but it would be interesting to see how much is too much.
Are you thinking of Agda/Coq/Idris, which have dependent types? If you understand a bit of type theory, this is an interesting paper: http://www.cs.cmu.edu/~fp/papers/popl99.pdf
Development time with static languages is not an issue.
Try comparing the time one needs to change the codebase written in ML and the one written in Lisp and come back. It is experimentation and change that is slow in static typing.
I do not believe so. In languages like ML, static typing is not just used for type safety. Proper use of static typing extends to be able to use it in establishing invariants all over your code. Now, your entire codebase becomes dependent on the type. No matter how expressive a language is, this causes a lot of friction.
If anything, it is my experience that static typing makes it much easier to refactor large amounts of code.
I remember numerous instances where I make a large change in one Haskell module resulting in a few changes in the module's external interface, the compiler systematically warns me of all the 20 other modules that needed changing, and lo-and-behold, if it compiles, it works.
This study, while perhaps not very relevant to real world software engineering (development, maintenance, refactoring), seems to show that there when developers design code and applications, there's little difference for them between using dynamically or statically typed languages. And that's interesting, because it implies that the way developers think about code is somehow type independent.
I've wondered about languages that are "hybrid" in the sense that the type system can be turned on and off, because from an intuitive sense, whether the language is meant to be used as a dynamic language has an impact on its design, similarly with a static language.
You take Smalltalk for example, as it is the original archetype of a language that benefit from blurring the lines of the type system. A language like Haskell on the other hand was design with the intention that the programmer would use the type system to do a lot of heavy lifting.
Of course, no real-world language is "pure" in that sense. We've graft type systems onto Smalltalk's "way of doing things", and the GHC Haskell compiler now allows you to treat type errors as warnings.
The change in GHC is just to make development easier. Basically, just imagine that instead of stopping at a type error, the compiler replaces the incorrect function with code to throw an error at runtime. So here the poorly typed function always errors, no matter what you pass it; in a dynamically typed language, it would work as long as the arguments passed at runtime have the correct types.
So it isn't anything like dynamic typing, or even quite like making type errors warnings--it's just a way to compile partially incorrect code to make changing the code incrementally easier. It's also not designed to be used in production.
It just automates the existing practice of temporarily commenting out offending functions and replacing them with `undefined' to get the whole file to compile.
While interesting, I don't think we can consider the conclusion as the final word on it. There are many other variables to take in account.
The programmer's expertise is an important one and it is difficult to take in account. Static typing is to check code validity. If the programmers make no mistake, then of course specifying types is an overhead. When it comes to collaborative work, strong typing can save a lot of debugging time, meeting time and documentation writing and reading time.
Another aspect is the resulting code efficiency. Code is programmed once and executed many times. It is much easier for a compiler to generate efficient code when it is given more information to do so by the programmer. This benefit of the typing overhead is not taken in consideration.
As repeatedly said, use the appropriate tool for your application.
If anything, I think this study is actually in favor of static typing. Why? Well, most of the proponents of dynamic typing claim it makes development faster. Most proponents of static typing concede this point, but claim that while it may slow down initial development, it makes maintenance much easier.
Of course, this study is fairly limited and drawing conclusions from it is probably premature. I would not base any policy decisions on it alone.
But, if static typing does not slow you down, it seems worth using even if benefits to maintainability are slight. And if they're nonexistent, using a statically typed language over a dynamically typed one still doesn't hurt you.
Besides, I think static typing is awesome. And I'm a college student--you can trust me! That should be all the validation you need :).
"In the experiments the existence of the static type system has neither a positive nor a negative impact on an application's development time (under the conditions of the experiment)."
Maybe the abstract is lying, but that seems to agree with my original premise. That is, the type system had no effect on development time.
Now, the paper makes no comment either way--all it says is that the type system did not have an effect on development speed. My argument is that not having an effect is actually a point in static typing's favor: the main advantage of dynamic typing is supposed to be faster development, after all.
Read further than the abstract. It's just an abstract.
To quote:
We measured two different points in the development: first, the development time until a minimal scanner has been implemented, and second the quality of the resulting software measured by the number of successful test cases fulfilled by the parser. In none of these measured points the use of a static type system turned out to have a successful impact. In the first case, the use of a statically typed programming language had a significant negative impact, in the latter one, no significant difference could be measured.
Statistically speaking, 'a significant negative impact' doesn't mean the same as it does in regular English. It doesn't say much about how large the difference is, only that they have enough data to show that there _is_ a difference. When an abstract summarizes that the difference wasn't much to talk about, that does mean a lot more than you give credit.
Now, judging from their test setup, and also from the very low number of testers, I find it very hard to agree with what it seems to be you're taking away from this article. Static typing has more benefit in large code-bases, with multiple programmers, and for avoiding hard to find bugs related to dynamic typing. This doesn't seem to be well reflected by the setup they had.
Also, don't think you should repeat the same comments throughout the HN articles, you are not replying to individuals, but to general readers.
In the first case, the use of a statically typed programming language had a significant negative impact
Considering the size of the sample, I'd guess the difference is rather large to considered significant. By looking at the numbers quickly, it seems to be around 25%. I'm a bit shocked, in fact, because in my own experience, the difference is much larger, but this experiment controls for language and my experience doesn't.
Yes, I was able to read that in both your comments, as well as in the paper.
I suspect you are not understanding my point, or what I said about the meaning of the use of the word "significance" when used in statistics.
edit:
Say you flip a loaded coin that is 50.1% likely to be heads. Now you want to test whether this is loaded, and flip the coin a certain number of times and count the outcomes. If the number of times you flip the coin is too low, you won't be able to say the coin is 'significantly loaded'. It might be either way, you don't have enough data. If you flip it enough times, you will be able to say something about it -- i.e. that either it significantly is, or significantly isn't loaded.
However, in vernacular English, you would still say that the difference isn't very significant. Who cares if it is 50% or 50.1%.
Yes, but I'd risk that a 25% deviation is significant when the sample is 49 students. The smaller the sample, the larger the deviation must be to be significant, but 25% is quite a difference.
Also, it's worth to notice they controlled for language - they used the same language in two flavors - to isolate the typing system difference. It's not a Lisp vs. C thing.
There is no reason to quarrel about the meaning of 'significance' in statistical testing vs. in natural language. Just look at figs. 4 and 5 in the paper, and see that mean times spent for the scanner task were:
And that the difference in statistically significant (p=0.04, Mann-Whitney U-test). Whether 5 vs. 8 hours is significant in the natural language sense, everyone can decide for themselves.
191 comments
[ 5.2 ms ] story [ 372 ms ] threadAt the end of the day it's about what YOU can hack it with.
Use that language.
For me, switching from static (java) to dynamic typing (common lisp) has saved me a lot of development time. Or did it? Maybe it was Emacs? Maybe it was functional programming? Maybe it was the REPL? Maybe experience?
Cf. node.js
The crucial difference is that in python, these faster python idioms are for using C as the control code
The paper's title is casting doubts on the positive impact of statically typed languages as they had a negative impact on development time. (though not quality)
In the experiments the existence of the static type system has neither a positive nor a negative impact on an application's development time (under the conditions of the experiment).
What's the point of dynamic languages if they don't even make the initial implementation faster? They're certainly harder to maintain afterwards.
We measured two different points in the development: first, the development time until a minimal scanner has been implemented, and second the quality of the resulting software measured by the number of successful test cases fulfilled by the parser. In none of these measured points the use of a static type system turned out to have a successful impact. In the first case, the use of a statically typed programming language had a significant negative impact, in the latter one, no significant difference could be measured.
Dynamic code can be very fast, but in lisp, after algorithms, that means actually going and finding all of the calls to elt and replacing them by calls to nth. With a static language the compiler has a much better chance of specializing the code for you.
Sometimes you may need to type-dispatch when you want to do smart functions. A typical example is a function that takes a sequence/iterable/generator of strings or a string. Since a string is iterable in python, you have no choice but to check for BaseString.
Hopefully for the next experiment they have them tackle a longer project, and have the programmers tackle short problems in multiple languages to 'rank' them.
I don't think it's a big deal.
What I found after using Java, Gosu (basically Java but less stupid), Python and JavaScript fairly significantly is that all of them have their own problems. In that particular set, I'd probably value a static type system a little more than a dynamic one, but it gets eclipsed by other language characteristics (e.g. support for functional programming).
In fact, thanks to being scarred by Java, for the longest time I really liked dynamically typed languages more. However, I later realized that it was the lambdas and the higher-order functions and the clever abstractions and the terseness that I liked more than the dynamic typing.
Once I learned Haskell (and later OCaml), I've been converted to liking static typing quite a bit. The beauty in these two languages is not just that the type system catches more bugs than Java's while being less of a burden, but also that they type system can actually make the code more expressive.
My favorite example of this is the read function from Haskell. It is the opposite of toString--it goes from a String to some value. However, the really awesome thing is that you do not need to explicitly specify what type to parse to--the type inference can figure it out for you! Imagine being able to use "5".fromString() the same way you use 5.toString(). In a dynamically typed language, you would have to somehow specify what to parse to explicitly. So your fromString code would something like Integer.fromString("5"), which is less nice and throws away the toString/fromString symmetry.
I have about the same experience with dynamic (Perl, Scheme, JavaScript, Python) and static (Java, Gosu, Haskell, OCaml) languages and I like good static typing most.
Long story short: use Haskell!
On the rare occasion you really need the kind of anything-goes dynamism of Ruby/Python/JS you can just roll your own dispatch tables.
I'd like to see them hand a pre-written codebase of say, 10,000 lines to a bunch of students, and ask them to make large changes/new features to the codebase.
Absolutely.
And also: static typing benefits is not about "development time" but about maintenance, adding stuff, refactoring etc.
If anything, dynamic typing would be expected to lead to faster development time, which is also why it's used in most prototyping.
Lastly: small sample. Did enough of them use an editor/IDE that could take advantage of a static type system?
static typing would more benefit:
* large groups of programmers
* large codebases
* maintenance (which is usually taking more resources then the initial development of a large code base)
aditionally i've come to believe that not all static typing is created equal. most allow `null` to be returned instead of adhering to the type, then some have exceptions and not many languages type side effects (like IO). i'm basically saying that C/C++/Java/C#'s kind of typing is not Haskell's kind of typing, and that the potential gains from Haskell's type system are much bigger on the long run (while it also comes with a steeper learning curve).
For instance, Scala has fairly good integration on top of Eclipse and IntelliJ IDEA, however every time I try doing some work in Scala, I end up cursing and screaming, because these Scala IDE plugins are slow, incomplete and unstable and get in my way. And I can't blame their authors, because Scala is a difficult language to deal with.
Another example would be C++, which has been around for a very long time and is one of the most popular programming languages ever. And yet even Visual Studio has problems with its refactoring/IntelliSense support.
And then there are the Smalltalk environments, still around, still kicking ass.
Here's the thing ... there are static languages, and then there are languages designed for usage within an IDE ;-) Java, C# and Smalltalk are like that, while Scala, Haskell and C++ aren't.
Dynamic languages are very good for prototyping or small scale projects.
But they fail to address the context of programming large scale applications with teams distributed across multiple sites.
- Multi-site development at least in 3 countries; - Sometimes up to 100+ developers - CI systems - Source code of several MB of source code - Enterprise like infrastructures for Fortune 500 companies - Different skill sets from the "just out of the university" to the "top developer"
Maybe YouTube beats this, but Google only has Phd guys able to crack out crazy algorithms/data structures in minutes. Not typical in most software houses.
Also, as for large and complex projects done by distributed teams, I don't have to point much further than Django or Plone to prove dynamic typing works well in that context.
Heck, the examples you give aren't even anecdotes. They're just name-dropping. One would have to be pretty familiar with the codebases in question and the history of their development in order to be able to give a clear assessment of what, if any, impact dynamic typing might have had on them.
There was a time I was familiar with both codebases (I have some catching up to do) and that's why I mentioned them. Both projects carry heavy heritage and are experiencing huge pressures to evolve and both are doing very well (from what I hear on the dev lists).
That doesn't imply that they are being successfully maintained because they are written in dynamic languages, or despite being written in dynamic languages. And of course there's also the possibility that static vs. dynamic is a wash and doesn't really have an impact at all. Or that the potential impact of going with static or dynamic is heavily influenced by other factors - does the static language have type inference, does the dynamic language support duck typing, stuff like that.
Long story short, correlation does not, in and of itself, imply causation.
- Big consultancy company working with lots for Fortune 500 company groups;
- At least 3 development sites active at any time;
- Some projects can have 100+ active developers across sites;
- Several MB of written source code, plus many modules generated via specific DSLs or code generation tools
- The typical enterprise architectures
- Lots of crappy developers in some of the teams
The implications of static typing helps keeping the possible damages under control.
Everywhere people talk about 'large' teams maintaining a large codebase in this thread, substitute 'mediocre' teams stuck with a poor bloated codebase that is the vehicle for their ambitions. It's just not worth anybody's time to understand its unique needs in detail, especially since any improvement you make today risks being messed up tomorrow.
My lisp interpreter above allows me to tear out a lisp function and replace it with a C function, while leaving the unit tests untouched.
Most enterprise projects with modules developed in offshore, suffer from crappy developers that barely know one single programming language.
Forcing them to write lots of unit tests as required by dynamic languages development leads to nothing.
The only way to minimize broken systems is to use languages that force them to stay on course.
From my personal experience, dynamic typing usually works well with 1-2 developers and continues to be useful until about 4-5 developers (static typing too for that matter). After that, static typing usually wins pretty consistently.
My personal preference is static typing. I find it makes the code more readable and catches a lot of oopses early, at compile time. Otherwise you just end up writing a ton of test cases and hope to catch the same issues at runtime. So you have less protection (you have to catch it with a test), and you just end up spending more time writing tests anyways.
I do understand the benefits of both, I just prefer static typing because of the way I work. Both work and have their places. Like everything else, you need to use the right tool for the right job AND the right people.
I've never seen these consistent wins you talk about, despite my years at some of the largest corporations.
Someone mentioned maintenance, so I will also ask how old the codebases that you're working on are? For dynamic typed languages, as the code grows older (with more bug fixes and add-ons and one-offs) then it gets harder and harder to maintain as 100% error free (let alone debug).
I will also throw in the fact that doing software library version upgrades on dynamically typed systems is a pain in the ass. Some code branches have system level calls changed, and unless you have very good unit test coverages (which may or may not need to be re-written on a library version upgrade) then you may or may not catch the problem during the upgrade. With a statically typed language, a static analysis tool can tell you exactly where and how the library or interface upgrade affects your codebase.
"I will also throw in the fact that doing software library version upgrades on dynamically typed systems is a pain in the ass."
This is very true, but you gain a lot with dynamic languages in other areas. The last company I worked for had about 300 programmers in the office I was in. Roughly half did Java and the other half did Ruby. I was one of the few that moved between both ruby and java projects. I did not see these great benefits of static typing that people always insist exist for large teams at large companies.
2. They ought to check on maintenance effort and time with different type systems as well.
It seems the only question this experiment could possibly claim to answer is whether or not the additional finger typing required by static languages affects the coding speed of these particular 49 undergrads.
My feeling about this has always been that dynamically typed languages will save time on initial implementation and small projects, and statically typed languages will win on large projects and things that require more maintenance. They acknowledge this, and didn't find anything to contradict it.
Their own graphs seem to show that effect too. In some areas, dynamically and statically typed languages were equal, while in others dynamically typed languages had lower times. I don't actually see where the conclusion that a "static type system had no effect on development time" came from, it appears that it had an overall (if not very large) negative effect. Just like I would expect for a small (27 hours) project.
I didn't crunch the numbers though, maybe I'm just eyeballing it wrong. (again, I can't seem to copy-paste out of it, and I don't feel like reentering all of them)
I once read an article about a technique Intel had developed for improving cooling of processors by changing the shape of the fan. I related this to some of my co-workers. One of them proceeded to tell me that this can't possibly work, backing up his argument with "reasoning" based on off-the-cuff remarks about the way air and physics "must" work. The fact that Intel had actually done this seemed to have no effect on his eagerness to continue the "debate" about this scientific fact.
I have a hard time believing that I get no benefit from using Haskell over Smalltalk, but if a body of science were to appear that cast doubt on that belief, the appropriate thing to do is change the belief, not stand around debating from imagined first principles why all the science is wrong and can't be so. Shut up, design an experiment and go prove it!
Perhaps there's little of this kind of actual science in our computer science because it would mean asking hard questions and accepting difficult truths. "The prisoner falls in love with his chains."
Software development is so complicated that there is always an endless supply of objections to fire at any study at odds with one's beliefs. And that is exactly how all these discussions go. All we're doing is repeating shibboleths.
The most interesting studies would be ones that changed somebody's beliefs. That doesn't happen very often in our field. Does it ever?
Also, I suspect there are a couple of reasons such studies are uncommon in computer science. For one, CS isn't really a science; programmers and computer scientists are not trained in the scientific method or experimentation (beyond their general education); almost no CS papers I've read have contained empirical studies. If anything, they are closer to math papers than science papers!
Additionally, this sort of study is basically sociology. (Or something similar.) These sorts of fields are considered a little shady by hard scientists, and CS people tend to empathize more with the latter. I think this explains the immediate attack on methodology.
All that said, having more studies done about these questions would be great. I'm just not sure who's the best to do them. Maybe HCI researchers? I can't help thinking that the really intense PL people I know wouldn't be very interested in doing this.
You aren't reading the right papers then. At least in Software Engineering, you can't get into the main conferences (ICSE and FSE) without a pretty significant empirical study.
More to the point, the distinction of what is and isn't computer science has become even more blurry in the research community because research in itself has become more inter-disciplinary. There seems to be little to gain from attempting to "bucket" research into distinct taxonomies.
The only problem is, people have been doing these experiments for 30 years, and do you know what the net effect it's had on the world of programmers: none at all. Saying "no no, this time really listen to this study" seems to be having no effect.
There are a lot of causes for this, not least of all the things you mention (nobody cares about science, people like or dislike based on non-scientific evidence).
But it's also because people know that all these studies are flawed. As much as I hate to be the "nitpicker" who takes apart studies, EVERY SINGLE STUDY on programming doesn't even come close to real-life scenarios. In fact, one of the few studies that people actually believe is the "some programmers are 10x better" study, and that was actually fairly well conducted - many students were given identical tasks, and a fairly large amount of time to do them.
But take a look at the Dynamic vs. Static argument. For years, the Dynamic-fans have been saying "Quicker to program, so it's better", while the Static-fans have been saying "Quicker to program, but harder to maintain, hard to use with large teams". So now we have a study that doesn't even come close to addressing most of the issues that have been argued for years! Of course this isn't going to convince anyone.
Can you by chance point me to this paper? I'd like to add it to my paper collection, since most of the studies I've seen concerning programmer variability use members of the workforce. I'm not aware of the one involving students, but such replication studies are easy to miss.
Thanks!
That is the specific study I am searching for to add to my list of papers. Did you give me this link because you were referring to Humphrey (A Discipline for Software Engineering), or something else? I can track down Humphrey, but it will take me a few days, since it's a physical book.
Perhaps the previous author was thinking of the Prechelt "An Empirical Comparison of .." paper? http://page.mi.fu-berlin.de/prechelt/Biblio/jccpprtTR.pdf . Section 5.7 has "work times" for Java and C/C++ programmers using well-observed times. However, that is not for a "fairly large amount of time."
Laurent Bossavit does a masterful job of researching the origins of this myth (and others) in his new book The Leprechauns of Software Engineering, available on LeanPub [1].
[1] http://leanpub.com/leprechauns
However, I don't want to go down the rabbit hole and argue social vs natural sciences.
I find it dumbfounding that in this day and age, people can still create rather arbitrary social experiments with only 49 people and then think they can draw grandiose conclusions from their "data".
There is an ongoing experiment wrt the usability of programming languages, libraries, frameworks, concepts, ... It's called the market.
The number of programmers is too small and skills not representative (49 undergrad programmers). The problem (writing a simple scanner / parser) isn't one that will really benefit from a decent type system. All you need need are ints, strings, and arrays and you're good to go.
I agree in the sense that we see so many cases in the other direction - reasoning full of holes being trusted over empirics. The interesting thing is in any given situation, how much trust to give to the different tools that we have to evaluate a claim.
In fact, everyone here should flip this around and ask: can you design a practical experiment to compare productivity of dynamic vs. static languages? Will others find it convincing? Probably not. PL advocates are no different than religious missionaries. They have no objective proof of any of their claims. And both will murder the natives if they don't convert.
edit: Fowler's view here http://martinfowler.com/bliki/CannotMeasureProductivity.html
For example, controlling variables has to be done by constructing artificial systems from the ground up. You can't just pull two commercial products off the shelf and then pretend you're examining the impact of only one of the hundered different things that differs between the two.
Similarly, if we wanted to compare the impact of dynamic vs. static typing, we'd have to make sure that that is the only variable. Which means you basically have to construct a new programming language from the ground up, so that you can easily create new dialects of it that differ in only one very specific characteristic.
I suppose I should have nodded to that. Got me there.
In this case, we have demonstrably covered every case. That's what an expressive type system gets you: you can completely preclude certain classes of bugs like null pointer dereferences by having a sufficiently expressive type system. It's the same in more relevant PL research: people are attempting to formalize systems so that whole classes of bugs can be removed at compile time. It's not about user case studies or anything like that: those can come later, when features look like they'd be useful to integrate into languages.
It looks like the featured article gives a case study about a pretty bad type system. A sufficiently expressive type system doesn't get in the way--it aids the programmer, not hinders her. Heck, in Haskell and ML, you don't even have write down types--the compiler will infer them for you. (It is Haskell practice to type-annotate toplevel functions anyway).
[0] When I say type system, I mean a static type system. For the purposes of this discussion, dynamically typed programs are statically typed, just with not-very-useful types.
Remember all the research done on typestates? The motivation section of those papers was usually a few paragraphs of total BS. AFAIK, there was no real data nor experiment that demonstrated this was a real problem for professional programmers.
FYI: I like static type systems and used Haskell et. al. But no one has demonstrated that it is better than even Visual Basic!
1) a REPL. Just give it a whirl and see what you get.
2) intuitive method names and good abstractions. If I call @user.posts in a Rails app, I can assume I will get an Array-like object that I can treat as an array without any problems. The actual type does not matter 90% of the time.
You could also change @user.posts to be @user.contributions, for example, and that would hide the fact that contributions was just an array of Post objects.
A REPL is in no-way tied to having a dynamically typed or interpreted programming language. E.g. Haskell's GHCi has an excellent REPL environment where you can, in addition to evaluating code, ask for the type of an expression and do other static analyses.
I agree that a REPL makes a huge (mostly positive) difference on how you write code.
In a large system you might have to jump through a lot of hoops to get an accurate result from the REPL. What if the thing you're trying to puzzle out is 20 call levels deep? As a new Ruby dev I struggle with this.
(Off-topic, the werkzeug debugger even lets you debug browser apps by jumping into a REPL anywhere in the stack trace, in the browser, after an exception is thrown. I think it also has a command that bakes you a pie.)
For the benefits of static typing I would rather look into something like Haskell which has a much stronger notion of type and builds many language features around this notion. It's probably harder to design a reasonable study comparing that with for example Smalltalk, though.
Try comparing the time one needs to change the codebase written in ML and the one written in Lisp and come back. It is experimentation and change that is slow in static typing.
It's all about the expressiveness of the language. If you can express more with less code, than it's also easier to change, to experiment.
If you change something in a dynamically typed language you still have to catch all places, where the change has consequences.
Only If you want to experiment with something locally without needing to update the rest of the code base, than dynamic typing has an advantage.
I think it's GHC 7.4, which allows something similar for Haskell, to have the typing checked at runtime, to allow this kind of experimentation.
I think that's the best of both worlds, because for the production code you can still activate the compile time type checking.
It will be in GHC 7.6.1 and the extension is '-XDelayErrors'.
http://hackage.haskell.org/trac/ghc/ticket/5624 http://hackage.haskell.org/trac/ghc/wiki/DeferErrorsToRuntim...
I remember numerous instances where I make a large change in one Haskell module resulting in a few changes in the module's external interface, the compiler systematically warns me of all the 20 other modules that needed changing, and lo-and-behold, if it compiles, it works.
Even from a Bayesian perspective, having a small sample that is too small to reach strong conclusions does not shift your posterior towards 0.
You take Smalltalk for example, as it is the original archetype of a language that benefit from blurring the lines of the type system. A language like Haskell on the other hand was design with the intention that the programmer would use the type system to do a lot of heavy lifting.
Of course, no real-world language is "pure" in that sense. We've graft type systems onto Smalltalk's "way of doing things", and the GHC Haskell compiler now allows you to treat type errors as warnings.
Explain? Or post a link?
EDIT: found it: http://hackage.haskell.org/trac/ghc/ticket/5624
So it isn't anything like dynamic typing, or even quite like making type errors warnings--it's just a way to compile partially incorrect code to make changing the code incrementally easier. It's also not designed to be used in production.
It just automates the existing practice of temporarily commenting out offending functions and replacing them with `undefined' to get the whole file to compile.
The programmer's expertise is an important one and it is difficult to take in account. Static typing is to check code validity. If the programmers make no mistake, then of course specifying types is an overhead. When it comes to collaborative work, strong typing can save a lot of debugging time, meeting time and documentation writing and reading time.
Another aspect is the resulting code efficiency. Code is programmed once and executed many times. It is much easier for a compiler to generate efficient code when it is given more information to do so by the programmer. This benefit of the typing overhead is not taken in consideration.
As repeatedly said, use the appropriate tool for your application.
Of course, this study is fairly limited and drawing conclusions from it is probably premature. I would not base any policy decisions on it alone.
But, if static typing does not slow you down, it seems worth using even if benefits to maintainability are slight. And if they're nonexistent, using a statically typed language over a dynamically typed one still doesn't hurt you.
Besides, I think static typing is awesome. And I'm a college student--you can trust me! That should be all the validation you need :).
The paper clearly states that static typing slowed development time down.
Maybe the abstract is lying, but that seems to agree with my original premise. That is, the type system had no effect on development time.
Now, the paper makes no comment either way--all it says is that the type system did not have an effect on development speed. My argument is that not having an effect is actually a point in static typing's favor: the main advantage of dynamic typing is supposed to be faster development, after all.
To quote:
We measured two different points in the development: first, the development time until a minimal scanner has been implemented, and second the quality of the resulting software measured by the number of successful test cases fulfilled by the parser. In none of these measured points the use of a static type system turned out to have a successful impact. In the first case, the use of a statically typed programming language had a significant negative impact, in the latter one, no significant difference could be measured.
Now, judging from their test setup, and also from the very low number of testers, I find it very hard to agree with what it seems to be you're taking away from this article. Static typing has more benefit in large code-bases, with multiple programmers, and for avoiding hard to find bugs related to dynamic typing. This doesn't seem to be well reflected by the setup they had.
Also, don't think you should repeat the same comments throughout the HN articles, you are not replying to individuals, but to general readers.
In the first case, the use of a statically typed programming language had a significant negative impact
Considering the size of the sample, I'd guess the difference is rather large to considered significant. By looking at the numbers quickly, it seems to be around 25%. I'm a bit shocked, in fact, because in my own experience, the difference is much larger, but this experiment controls for language and my experience doesn't.
I suspect you are not understanding my point, or what I said about the meaning of the use of the word "significance" when used in statistics.
edit: Say you flip a loaded coin that is 50.1% likely to be heads. Now you want to test whether this is loaded, and flip the coin a certain number of times and count the outcomes. If the number of times you flip the coin is too low, you won't be able to say the coin is 'significantly loaded'. It might be either way, you don't have enough data. If you flip it enough times, you will be able to say something about it -- i.e. that either it significantly is, or significantly isn't loaded.
However, in vernacular English, you would still say that the difference isn't very significant. Who cares if it is 50% or 50.1%.
Also, it's worth to notice they controlled for language - they used the same language in two flavors - to isolate the typing system difference. It's not a Lisp vs. C thing.
5.17 hours (dynamical typing) 7.71 hours (static typing)
And that the difference in statistically significant (p=0.04, Mann-Whitney U-test). Whether 5 vs. 8 hours is significant in the natural language sense, everyone can decide for themselves.