I can try it, if you'd like. I haven't looked at the exercise at all, so it would be fair. I'd rather do the exercise in Scala though... So if anyone wants to benchmark that I can do it instead :)
To me, it's not interesting at all to ask who can write a 100-1,000 line program faster in which language. The real interesting question is how efficient developers are once you're talking about a 100,000 - 1,000,000 line program that has been in development for years. Solving small problems is relatively easy.
So he's saying that a Java version of Emacs would be 10 million lines of code long? That's ridiculously unlikely, unless given to monkeys to code -- and even then incompetent monkeys.
I would be surprised if the said 100k-1M LOC programs would not be decomposable eventually into 100-1000 line subunits. The "faster" aspect might be overrated, but in a concise language the 1M-LOC might be reduced significantly or might be more manageable.
Perhaps but I think it depends on who's writing it.
I love Python and haven't touched Java in years but my colleagues who use Java seem to write programs that are maybe around 3 times longer (this is a ballpark figure).
I get the feeling that programmers that produce gobs of Java code are probably going to do the same thing in Python. A good Fortran programmer can write Fortran in any language :).
Overly wordy Java coders would still be overly wordy Pythonistas, except the Python culture enforces the "Python is not Java" mentality so they would be corrected on almost every turn, plus the code tutorials and examples from which they learn would be shorter. As a result they would become less verboten very quickly.
That and most new-python converts initially don't think in a very functional way. After the "aha" moment of lambda love, the Java ways can be unlearned.
In the past, I would've said read The Little Schemer book for better understanding recursion and lambda (basically, a lambda function is an anonymous function - meaning you don't have to name it). JavaScript is actually a great language to better understand functional programming - it really is Scheme in C-like syntax. Here's a great little chapter on functional programming in javascript (http://eloquentjavascript.net/chapter6.html). Just bear with me and read it - it's extremely worth the trouble. You'll understand the syntax fine even if you don't know javascript.
After you read that, hopefully you'll start passing around functions as arguments inside another functions. It's tremendously fun and changes your way of thinking about code.
The simplest explanation is think of your program functions like mathematical functions that spit out values, with that in mind you can start sending in functions as arguments. A very brief e.g from the link:
def addition(n):
return lambda x: x + n
f = addition(3)
f(4) # is 7
here, f is actually binded to a function - the addition function from above (because it returned a function called lambda).
You seem to be conflating the idea of closures and functional programming. Functional programming is more about avoiding state and side-effects, which is antithetical to most Python and JavaScript code. A lot of languages, including C, support the kind of programming you're describing, but that's not "functional" per se.
Frankly I've found python to be quite antagonistic to functional programming with very weak lambdas and poor building blocks for building interfaces in monadic pipeline style. I guess coming from java it would seem refreshing but coming back from haskell or even just javascript it feels like a straitjacket.
I hear this argument a lot, but I don't know these concepts well enough from haskell et al to fully understand it. Could you be more specific about this? Perhaps, provide a direct example?
Your comment provoked me to try Haskell and see what the fuss is all about. I came across this fun little Try Haskell thing (yeah, copycat of TryRuby) http://tryhaskell.org/ It's very interactive and fun.
I think basically what the parent comment is saying is that it's much easier to pass functions as arguments to other functions in Haskell than it is in python. They call them high order functions.
I think one key thing is that lambdas are second-class citizens in Python in terms of the kinds of code you can express. Python lambdas can only contain one single expression and nothing else. For more complicated functions, in particular any function that requires statements, you are always required to use nested named functions, e.g.
myList = [[1, 2], [3, 4], [5, 6]]
for elem in myList:
del elem[0]
cannot be done using a lambda, as
map(lambda elem: del elem[0], myList)
results in an error. Until Python 3, that also meant that
map(lambda elem: print elem, myList)
was also a syntax error because of the print statement. For a more elaborate example, consider the following JavaScript function which returns a function that increments a counter every time its called and returns the value before it was incremented:
function makeCounter() {
var value = 0;
return function() {
return value ++;
}
}
You would use this as follows:
> c = makeCounter();
> c();
0
> c();
1
... and so forth
There is no way to implement this in Python using lambda; the closest equivalent (assuming Python 3 for the nonlocal keyword) is
def makeCounter():
value = 0
def count():
nonlocal value
tmp, value = value, value + 1
return tmp
return count
because there's no way to have that assignment take place in a lambda.
Of course, Haskell has neither state nor statements, so the above examples don't literally show why Haskell's lambdas are necessarily superior, but the point isn't that Python disallows assignment in lambdas; it's that Python's lambdas can only express a subset of possible functions, while JavaScript and Haskell (and Scheme &al.) don't have that limitation.
my_list = [[1, 2], [3, 4], [5, 6]]
my_other_list = [ elem[1:] for elem in my_list ]
You could also reassign the my_list name to your processed list, but changing the state of things (you are not really changing anything here, just calling something different by an already known name) is very un-functional.
As for your second example, I would consider using "yield".
I agree wholeheartedly with all of that; in general, actually, I think lambdas in Python should be used sparingly, if at all, and I think Python makes up for any limitations with its other features (e.g. generators, function decorators, list comprehensions, &c.)
The issues with Python's lambdas really come up when you start trying to do things the Haskell-esque way—which is to say, the super-functional category-theory-on-the-brain way—which can be the right solution to certain problems (e.g. monadic parsers) but tends to be difficult to express in a way that's pleasant, non-hacky, and Pythonic. But I am of the personal opinion that Python's design choices make sense and I don't want to rail against it for not being Haskell or Ruby; I just wanted to show how anonymous functions in particular differ from language to language.
Excellent explanation. I think most people's issue with python's lambda is that it isn't what they expect. There are generally some gotchas and the syntax diverges from that of a regular function.
I side step the lack of closures by using named locals with the keyword arguments when defining inner functions. However, for numeric types, this doesn't really work:
def makeCounter():
value = 0
def count(value=value):
tmp, value = value, value + 1
return tmp
return count
inc = makeCounter()
print inc()
print inc()
The above prints '0\n0', but if we use an object that is used "by reference" in a such a function, one can get this behavior:
def set_adder():
s = set()
def add(item, s=s):
s.add(item)
return s
return add
s = set_adder()
print s(1)
print s(2)
t = set_adder()
print t(3)
print t(4)
Python actually does have proper lexical scope and real closures (as of version 2.2, I think), so you can do
def set_adder():
s = set()
def add(item):
s.add(item)
return s
return add
and it'll work the same way. The problem really comes in the fact that before Python 3, all assignment took place in the local scope, so anything involving assignment (such as the counter example) doesn't work. You can sidestep it in Python 2.6 using the same kind of trick:
def make_counter():
value = [0]
def count():
temp, = value
value[0] += 1
return temp
return count
which is sort of hacky but works. (Other languages sidestep this by having some other way of specifying declaration-versus-assignment, such as JavaScript's var keyword for variable declaration.)
I quite agree with you (especially after writing a lot of javascript) - it's funny you mentioned javascript as I said the same thing in a child comment I made below :-)
Functional programming is not just about passing function pointers. In this case, C is functional. It's about expressing your code as series of function applications i.e., no mutable state, no side effects, no differences between statements and expressions (everything has a value). Very few languages are purely functional, so it's rather a sliding scale.
Nonetheless, functional programming is not the encourage common style in Python: there are no data structures available, many methods have side effects, it's hard to isolate mutable values from mutable. Not saying it's right or wrong, it's just different.
I should also note that Common Lisp isn't all that functional compared to other languages: I'd rate it being just slightly more functional than Ruby. The reason it appeals to some Lispers is due to simple syntax, data structure literals and dynamic typing. These features are common now, are useful for experimental programming (of the sort that frequently happens in AI and ML work) but they originated with Lisp. Lack of syntax, presence of macros and resulting ease of metalinguistic abstraction are not present in Python, but not every Lisp(er) used them: some preferred functional style (e.g., never using loop macro and using higher order functions and closures as means of abstraction), some preferred macros and DSLs, some stayed away from both.
Java bashing is healthy: while I personally don't mind Java, many developers were forced into writing Java (having previously worked in Smalltalk, Lisp, C++, Perl, and others) as the language was heavily marketed to managers. Java hatred and bashing is thus a well understood reaction (although the culprit for verbosity are awfully horrid libraries such as Spring and J2EE: alternatives like Guice, for example, are much less verbose). However, JVM is now home to Scala, a much stronger candidate for an OO/functional hybrid language than Python (that's not to say Python isn't an elegant and powerful language with many advantages, especially when it comes to tools/automation development).
I feel like a lot of the languages meant to be written with an IDE end up being longer in terms of total LOC, but a pretty big chunk of that is all auto-generated noise.
> Perhaps but I think it depends on who's writing it.
Inexperienced programmers write sub-optimal (or plain weird) code. Once they learn their tools, they will write better code. That's not surprising at all.
I've worked on such huge Java programs being maintained and updated for over 5 years. I actually don't think it is Java as a language which helps in maintenance. Type checking by the compiler does help, but in reality you have Spring, Hibernate, JSF, JSP,.. where compile time checking doesn't apply anyway.
In my opinion there are 3 things that help with maintenance:
* automated tests (to see what breaks if you change something)
* refactoring tools (helps in keeping code clean)
* domain driven design (helps in mapping a business concept to a piece of code)
These are all things you could do in Python as well.
I believe one of the first refactoring was for Smalltalk, so it's not impossible. These days you also have IDEs with basic refactoring support for Python, Ruby, Javascript, Actionscript,.. from JetBrains.
Even at that scale, python code readability trumps over almost every other language out there. Compulsory indentation and there's only one way to do this mentality keeps everyone on the same page.
So, yeah, that's probably why Google uses python extensively.
Google doesn't use python extensively. It's used as scripting glue in a lot of places but there are no actual products written in python. It's all Java and C++.
The vast majority is Python (I'm a former 'Tuber). For a while, there was a real-time streaming/playlist/chat thing written in Erlang, but that's been gone for over a year now.
Are you at liberty to say why the Erlang thing got replaced? (Erlang & Python are my two favourite languages to use, but there are not that many problems I would interchange between them.)
The Erlang thing didn't get replaced, it got removed.
YouTube itself is an interesting project from the inside, though perhaps not that much different from any multi-featured website. Everyone has their own favorite features, their own favorite parts of the site, which they think will become "the next big thing", despite having a few orders of magnitude to go in order to reach the top 5 (which is 99.9% of the traffic).
The chat/individual playlist thing was a cute feature, but it had maybe a few hundred concurrent users at any one time. If you consider that the recent Lonely Island video pulled about 3 million views in a 24 hour period, and was less than 1/333rd of YouTube's video views (using the publicly released 1 billion views/day number), you come to the realization that any code that isn't being used heavily, is effectively an unused feature that offers you nothing, and even worse, can't earn the site any money in ad revenue.
Heck, I saw micro-communities as "the next big thing" on YouTube, and saw how poorly the original YouTube Groups fulfilled that. I envisioned small-medium sized forums where people would post, rate, and watch videos. I pushed and pushed and pushed until I got a little under a year to work on it, releasing Groups 2.0 in August/September of 2009. I watched how it was being used, got rid of some superfluous features, and pushed out an optional new version for people to use, which became the default on my birthday as my parting gift. Turns out that some code that I was relying on working from another part of the site had broken by another engineer, so they manually made it the default and only version a week or two after I left.
Sadly, YouTube Groups is no more, having been turned down on December 1, 2010, code deleted a couple weeks later. A friend and former coworker who had maintained Groups after I left had pointed me off to a WWE fan group with over 100k video posts and a few hundred thousand members (which would be a huge success anywhere else). Since no one was really willing to truly maintain and improve it (I can't blame them, there were some nasty hacks in there), and it wasn't driving as much traffic as even one Lonely Island video, they tossed it. It probably didn't help that I'd ruffled some feathers with the ways I'd pushed for Groups; I'm surprised it lasted even that long. Ah well, many lessons learned, many good memories.
Well, it's a bit of a new thing. For years there had been a push to start getting rid of lesser-used/unused features. Only a handful had been tossed over the years (including the Erlang, but that was at least as much to get rid of the Erlang as anything else). As I was leaving, the momentum to start reducing features was getting greater and greater.
The guy who ultimately deleted my code had taken on maintenance of at least two large pieces of code that 3-4 engineers (including myself) had written over the course of a year and a half, none of whom are still with Google/YouTube. Turning Groups down and deleting the code simplified his job immensely, of that I am sure.
The trick is that Python is a bit like special sauce. If your business is booming because of it's use, you are wary of expressing it as much, because it allows you to get so much more done so much more quickly than your competitors. If they find out, then they may have a chance to catch up.
It always blew me away seeing startups using JSP, .NET, etc., when starting out. Get it done first, then make it fast (if you need to).
I'm not really sure how Python's enforced indentation has anything to do with this. Java does not enforce an indentation style at compile time, but has a very well-established set of coding guidelines originally laid out by Sun that most coders follow. More importantly, Google has extensive coding guidelines for all the programming languages they use- their code will look consistent no matter what language they're working with.
I was just pointing out to the parent comment that it's not just the indentation but also the fact that a lot of things in python, there's one great way to do this is highly prevalent (and rewarded). This allows for easier readability in most cases.
As about the indentation, the inherent nature of mandatory indentation significantly improves code reading quality which will allow someone new coming in to work with "100,000 to 1 million" lines of code an easier time blending in.
I work at Google and we don't have that much Python in our codebase. There are some outlying cases that use it (like the YouTube front end), and it _is_ one of our official languages, but the vast majority of Google code is C++, Java, and JavaScript, with Python a trailing 4th place. Python is just not that suitable for large-scale software development.
Readability means different things at different scales. Reading Python "in the small" is easy; it's looks like pseudocode. Once you start writing larger programs, between the 5k and 10k mark, the looseness of Python starts to become a burden. It can be very difficult to look at a piece of Python code in a large project and know exactly what it does. Dynamic typing means that a whole class of serious programming errors go unnoticed until runtime - and often only for a specific, uncommon input.
("Compulsory indentation" ?! Whomever works on a large project with others and doesn't format their code to the standard of the project should be taken out and shot. I'm only half-joking. Indentation has almost nothing to do with readability, though. It's just common sense.)
(I'm surprised to hear you say "there's only one way to do this" with regard to Python, as in my experience there are always several ways of doing anything, and it's not uncommon to see different/competing approaches used in a single project.)
I'm biased, but Go is a great choice for large-scale software development. It has the brevity of a scripting language, but being statically typed and compiled Go can be more reliable and efficient. It's also smaller and more consistent than any mainstream language. Because of this, you can typically look at a piece of Go code from any project and understand what's going on. This is because it's difficult to abuse Go in the same way you see done in the other four languages I've mentioned, and that is truly valuable at scale.
>>It can be very difficult to look at a piece of Python code in a large project and know exactly what it does. Dynamic typing means that a whole class of serious programming errors go unnoticed until runtime - and often only for a specific, uncommon input.
The arguments I've read says that the "code scalability" is handled with more testing in the scripting languages. Is that wrong? I don't have statistics around, anyone got references?
(And yeah, indentation is mandatory. Implicit indentation often gets problems with copy/paste of code.)
Since it's been found that the types in mature dynamic language programs are mostly static after a certain point anyways, why not add optional typing and empirically determine the types at runtime using the same tracing technology tracing JIT VMs use?
This would give one the best of both worlds: fast prototyping for initial development, and type safety, added information when refactoring, and additional optimization when the code is mature.
This is the kind of article that should more frequently come to the attention of enterprise management. There really is an inverse relationship between the expressiveness of a language and the number of bugs created. (That is, I am taking at face value the studies which claim a more or less linear relationship between lines of code and bugs, regardless of language.)
I have heard this before many times, and I tried to find a study that would defend that assertion one weekend and failed miserably. If you have a citation for one of those studies I would be in your debt.
Hacker News is not 4chan, and as such, do not reply "Cool Story Bro", unless you are replying to your brother that you thought the story he gave you was popular.
I upvoted you and downvoted him because I really don't want HN to degrade into Reddit/Digg/4Chan/etc. I come here specifically because it's _not_ one of those, and has a good signal-to-noise ratio (e.g. relevant conversations not overrun by silly memes)
I wonder if another, equally valid, way of looking at the lines of code vs time is that if two programmers using different languages both complete the task in roughly the same amount of time, but one uses only half the total lines of code... that implies _not_ that that programmer is better, but that each of their lines of code took roughly twice as long to produce.
One of the things I used to do when working in large teams was wander around and look to see if anyone was having trouble debugging. Sometimes you'd find someone who'd been stuck on a bug for several days. So what I'd do was I'd sit down with them, and start putting in all the 'fluff' like indenting their code, splitting lines with multiple variable declarations each onto their own line etc. Essentially putting in all the 'long cuts' that they'd taken 'short cuts' around in order to 'speed up' the programming.
Often either they or I would see the problem immediately.
These days of course there are tools that do the auto-formatting, but I still try to optimise for readability, simplicity and elegance in my code.
Back to the lines of code as a metric issue (tm) in that scenario where it takes 2x as long to write a line in language X, and that line does twice as much, it is probably at least twice as complex as the 'more verbose' language... and I think this implies that it will be harder to debug and maintain later on. So arbitrarily complex languages are optimal for some internet 'toss off' (one off, toss away) competition, but not necessarily optimal for long term efforts.
Some of these languages that allow you to redefine the language itself on the fly allow you to do 'cool stuff' (tm) but tend to attract people who think they are much cleverer than they actually are. I think it is a regional variation, but all the Ruby fans in my city scare me. The thought of what would happen if I'd spent days or weeks on a bug only to find out it was caused by someone else 'tweaking' one of the core classes and not bothering to tell anyone else... twitch
I question your assumption that if something is done in 1/2 the lines of code then each line of code is twice as complex. You're equivocating on "complex". Each line may be "more complex" in the sense "does more", but not necessarily more complex as in "harder to understand" (and therefore more prone to bugs).
Code in a more productive (i.e. higher level) language is concise because it hides extra complexity you don't need to think about at the time, and is therefore simpler, not more complex (in the second sense).
Code in a more productive (i.e. higher level) language is concise because it hides extra complexity you don't need to think about at the time, and is therefore simpler, not more complex (in the second sense).
Another example of where this is true is automatic memory management. A line of Java code can do all the memory management that would take many lines in C. This is a mostly good thing. In this case, less lines = better.
I've had multiple bugs based on whitespace being significant. A C-descended language which had braces around the block would not have had the bug at all.
Another problem with it is that I can't just comment out, say, an if, and have the code under it run automatically. I have to go through and reformat the business, leading to potential bugs. This also plays into refactoring as well.
Basically, I find the whitespace rule something that should have been left in the 60s with punch cards.
I am not a fan of Python. I consider Common Lisp and Perl to be fundamentally less ambiguous languages.
I am a scheme neophyte. In my personal experience it has been a very thought compressible language.
By that, what I mean is this: if you give more thought to a program one tends to be able to say more and more but with less and less, and without making the code incomprehensible or complex.
Interestingly, when I show my code to a scheme veteran, he would often re-write it to make it even shorter and clearer. So I have a long way to go.
So line for line , yes it takes me more time to churn out scheme, but if I give the same amount of time to c++/java it doesnt compress as much. But, and here is the cool part, if it does, I can often trace it back to scheme way of thinking. This has happened a lot when programming with C++/STL.
"scenario where it takes 2x as long to write a line in language X, and that line does twice as much, it is probably at least twice as complex as the 'more verbose' language... "
I think this is debatable. After all - a function call compresses multiple lines into one as well.. And you want programmers to factor the repeated code into function calls.
The "dynamic redefinition of the language" is the same thing except at a different level.
The problem is not with the lines of code per se - it is whether the semantic compression is worth the learning curve, and the "surprise" factor.
Think C++ templates, for example, with a "+" operator that would do something "non-intuitive". The "non-intuitiveness" may vary - and it is a fine art to not cross the line.
FWIW, I've done the exercice in Lua, ~ 1 hour and 107 lines (and for some reason a different order of outputs, which I did not want to debug since I went for a nice dinner with a couple of beers earlier today) - https://gist.github.com/752460
I still try to optimise for readability, simplicity and elegance in my code.
That's a good goal. One advantage of Python is that indentation etc. are required. You can't take a 'short cut' and write messy code, cause your code won't run.
I am not arguing this but I think programming is in a sense "expressing oneself" and the "one way" and the "rigidity" of Python might turn some people off. Sometimes...you need to take that "short cut".
One advantage of Python is that indentation etc. are required.
This is such baloney.
I've written hundreds of thousands of lines of code in Algol-esque languages, and just about every last line (by which I mean > 99.9%, with possibly exceptions once in a blue moon where I accidentally hit tab before I compile or something like that, which just gets cleaned up the next time I reformat anyways...) of that was properly indented. It's just too easy not to do, and I'd fire, almost immediately, anyone that was checking in code that wasn't indented properly.
Really. Where are all these horrible developers that don't sanely indent their code? To hear the Python folks tell it, they're fucking up codebases all over the place with their carelessness, but I've never personally experienced any pain whatsoever over this. Certainly nothing that a simple CMD+i can fix...
Hell, different newline types interacting badly with version control have caused me so much more trouble than indentation, and Python's not immune to that. Nor is it immune to people's editors globally forcing tabs to spaces or vice versa.
Violates this requirement: "If and only if at a particular point no word at all from the dictionary can be inserted, a single digit from the phone number can be copied to the encoding instead."
The number you mention allows to get into the branch with the 'Koch o"d' - so no digit can be selected.
(I was reading the requirement as "If at a particular point no sequence of words from the dictionary can be inserted to complete the number, a single digit can be inserted" -- which still makes a lot more sense to me than the actual requirement!)
Sadly, no programming language could have helped me with this bug. :)
It is surprising that in 2010 we are still talking about efficiency in terms of lines of code whether the absolute number to express a problem or in the speed that these lines can be created. I realise that this is a proxy for the ability of a language to succinctly express a problem however it ignores all the major factors that determine how effective a piece of code is over its lifetime.
The ability of language X to solve a problem depends on:
1. The expressiveness of the syntax to capture algorithmic complexity.
2. The availability of tools to support the creation of the code.
3. The ability of the typical programmer, conversant in the language, to write the code.
4. The costs involved in maintaining the code over its lifetime.
Rendering all these dimensions into a single metric seems simplistic at best, undermining the argument as to which set of tools and technologies are deserving of more attention and ultimately plays into the hands of the people who wish to render programmers and programming into pure commodities.
For the most EXTREME algorithmic compression of information to number of lines in Python (with few to 0 intermediate values stored), I've found it hard to beat using nested function definitions with abstracted shared logic called by running the vars() builtin as the parameter list. Basically ship the current scope elsewhere to finish processing to ensure no logic is repeated.
I tend to not use separate URL mappers in controllers but have nested functions that define both logic and layout simultaneously. (Basically a tree structure of function definitions where the ending subpage elements are leaf nodes, and nodes render wrapper base templates on post traversal unless it was an ajax request). I can post some code if this doesn't make sense. Obviously not defining new variables nor using classes is extremely unpythonic and could pose difficulties for group work... minimum number of lines while fun isn't everything!
It took me about 2 hours (compared to a range of 2 to 8.5 hours for the other Lisp programmers in the study, 3 to 25 for C/C++ and 4 to 63 for Java) and I ended up with 45 non-comment non-blank lines (compared with a range of 51 to 182 for Lisp, and 107 to 614 for the other languages).
Seems like the programmer, not the language, is the biggest determinant. If you have the Java programmer who wrote it in 4 hours, and the Lisp programmer who wrote it in 8.5 hours, you're better off writing it in Java!
If you want to write concise programs quickly, using the same language Peter Norvig uses might help, but being as good as Peter Norvig helps more.
96 comments
[ 2.7 ms ] story [ 180 ms ] threadThis is just wishful thinking.
I love Python and haven't touched Java in years but my colleagues who use Java seem to write programs that are maybe around 3 times longer (this is a ballpark figure).
I get the feeling that programmers that produce gobs of Java code are probably going to do the same thing in Python. A good Fortran programmer can write Fortran in any language :).
After you read that, hopefully you'll start passing around functions as arguments inside another functions. It's tremendously fun and changes your way of thinking about code.
And this little answer in StackOverFlow summarizes what lambdas can do and how you can use it in python: http://stackoverflow.com/questions/890128/python-lambda-why/...
The simplest explanation is think of your program functions like mathematical functions that spit out values, with that in mind you can start sending in functions as arguments. A very brief e.g from the link:
def addition(n): return lambda x: x + n f = addition(3) f(4) # is 7
here, f is actually binded to a function - the addition function from above (because it returned a function called lambda).
http://en.wikipedia.org/wiki/Functional_programming
I think basically what the parent comment is saying is that it's much easier to pass functions as arguments to other functions in Haskell than it is in python. They call them high order functions.
Of course, Haskell has neither state nor statements, so the above examples don't literally show why Haskell's lambdas are necessarily superior, but the point isn't that Python disallows assignment in lambdas; it's that Python's lambdas can only express a subset of possible functions, while JavaScript and Haskell (and Scheme &al.) don't have that limitation.
As for your second example, I would consider using "yield".
The issues with Python's lambdas really come up when you start trying to do things the Haskell-esque way—which is to say, the super-functional category-theory-on-the-brain way—which can be the right solution to certain problems (e.g. monadic parsers) but tends to be difficult to express in a way that's pleasant, non-hacky, and Pythonic. But I am of the personal opinion that Python's design choices make sense and I don't want to rail against it for not being Haskell or Ruby; I just wanted to show how anonymous functions in particular differ from language to language.
I side step the lack of closures by using named locals with the keyword arguments when defining inner functions. However, for numeric types, this doesn't really work:
The above prints '0\n0', but if we use an object that is used "by reference" in a such a function, one can get this behavior: This will print:Nonetheless, functional programming is not the encourage common style in Python: there are no data structures available, many methods have side effects, it's hard to isolate mutable values from mutable. Not saying it's right or wrong, it's just different.
I should also note that Common Lisp isn't all that functional compared to other languages: I'd rate it being just slightly more functional than Ruby. The reason it appeals to some Lispers is due to simple syntax, data structure literals and dynamic typing. These features are common now, are useful for experimental programming (of the sort that frequently happens in AI and ML work) but they originated with Lisp. Lack of syntax, presence of macros and resulting ease of metalinguistic abstraction are not present in Python, but not every Lisp(er) used them: some preferred functional style (e.g., never using loop macro and using higher order functions and closures as means of abstraction), some preferred macros and DSLs, some stayed away from both.
Java bashing is healthy: while I personally don't mind Java, many developers were forced into writing Java (having previously worked in Smalltalk, Lisp, C++, Perl, and others) as the language was heavily marketed to managers. Java hatred and bashing is thus a well understood reaction (although the culprit for verbosity are awfully horrid libraries such as Spring and J2EE: alternatives like Guice, for example, are much less verbose). However, JVM is now home to Scala, a much stronger candidate for an OO/functional hybrid language than Python (that's not to say Python isn't an elegant and powerful language with many advantages, especially when it comes to tools/automation development).
I'm definitely going to use this definition of 'verboten' from now on. I like your style. Fuck the dictionary!
Also, to save you the work:
http://dictionary.reference.com/browse/verbotenInexperienced programmers write sub-optimal (or plain weird) code. Once they learn their tools, they will write better code. That's not surprising at all.
In my opinion there are 3 things that help with maintenance: * automated tests (to see what breaks if you change something) * refactoring tools (helps in keeping code clean) * domain driven design (helps in mapping a business concept to a piece of code)
These are all things you could do in Python as well.
So, yeah, that's probably why Google uses python extensively.
YouTube itself is an interesting project from the inside, though perhaps not that much different from any multi-featured website. Everyone has their own favorite features, their own favorite parts of the site, which they think will become "the next big thing", despite having a few orders of magnitude to go in order to reach the top 5 (which is 99.9% of the traffic).
The chat/individual playlist thing was a cute feature, but it had maybe a few hundred concurrent users at any one time. If you consider that the recent Lonely Island video pulled about 3 million views in a 24 hour period, and was less than 1/333rd of YouTube's video views (using the publicly released 1 billion views/day number), you come to the realization that any code that isn't being used heavily, is effectively an unused feature that offers you nothing, and even worse, can't earn the site any money in ad revenue.
Heck, I saw micro-communities as "the next big thing" on YouTube, and saw how poorly the original YouTube Groups fulfilled that. I envisioned small-medium sized forums where people would post, rate, and watch videos. I pushed and pushed and pushed until I got a little under a year to work on it, releasing Groups 2.0 in August/September of 2009. I watched how it was being used, got rid of some superfluous features, and pushed out an optional new version for people to use, which became the default on my birthday as my parting gift. Turns out that some code that I was relying on working from another part of the site had broken by another engineer, so they manually made it the default and only version a week or two after I left.
Sadly, YouTube Groups is no more, having been turned down on December 1, 2010, code deleted a couple weeks later. A friend and former coworker who had maintained Groups after I left had pointed me off to a WWE fan group with over 100k video posts and a few hundred thousand members (which would be a huge success anywhere else). Since no one was really willing to truly maintain and improve it (I can't blame them, there were some nasty hacks in there), and it wasn't driving as much traffic as even one Lonely Island video, they tossed it. It probably didn't help that I'd ruffled some feathers with the ways I'd pushed for Groups; I'm surprised it lasted even that long. Ah well, many lessons learned, many good memories.
I must say that it sounds great Youtube's culture allows features to be "turned down... code deleted a couple weeks later". You don't always get that.
The guy who ultimately deleted my code had taken on maintenance of at least two large pieces of code that 3-4 engineers (including myself) had written over the course of a year and a half, none of whom are still with Google/YouTube. Turning Groups down and deleting the code simplified his job immensely, of that I am sure.
The trick is that Python is a bit like special sauce. If your business is booming because of it's use, you are wary of expressing it as much, because it allows you to get so much more done so much more quickly than your competitors. If they find out, then they may have a chance to catch up.
It always blew me away seeing startups using JSP, .NET, etc., when starting out. Get it done first, then make it fast (if you need to).
As about the indentation, the inherent nature of mandatory indentation significantly improves code reading quality which will allow someone new coming in to work with "100,000 to 1 million" lines of code an easier time blending in.
Readability means different things at different scales. Reading Python "in the small" is easy; it's looks like pseudocode. Once you start writing larger programs, between the 5k and 10k mark, the looseness of Python starts to become a burden. It can be very difficult to look at a piece of Python code in a large project and know exactly what it does. Dynamic typing means that a whole class of serious programming errors go unnoticed until runtime - and often only for a specific, uncommon input.
("Compulsory indentation" ?! Whomever works on a large project with others and doesn't format their code to the standard of the project should be taken out and shot. I'm only half-joking. Indentation has almost nothing to do with readability, though. It's just common sense.)
(I'm surprised to hear you say "there's only one way to do this" with regard to Python, as in my experience there are always several ways of doing anything, and it's not uncommon to see different/competing approaches used in a single project.)
I'm biased, but Go is a great choice for large-scale software development. It has the brevity of a scripting language, but being statically typed and compiled Go can be more reliable and efficient. It's also smaller and more consistent than any mainstream language. Because of this, you can typically look at a piece of Go code from any project and understand what's going on. This is because it's difficult to abuse Go in the same way you see done in the other four languages I've mentioned, and that is truly valuable at scale.
Um...it has EVERYTHING to do with readability. That is WHY it is just common sense.
The arguments I've read says that the "code scalability" is handled with more testing in the scripting languages. Is that wrong? I don't have statistics around, anyone got references?
(And yeah, indentation is mandatory. Implicit indentation often gets problems with copy/paste of code.)
Think in terms of the budget of whatever you're building - the python guys delivered in a quarter of the time it will take the Java guys.
It's all about solving lots of small problems quickly, imho.
This would give one the best of both worlds: fast prototyping for initial development, and type safety, added information when refactoring, and additional optimization when the code is mature.
One of the things I used to do when working in large teams was wander around and look to see if anyone was having trouble debugging. Sometimes you'd find someone who'd been stuck on a bug for several days. So what I'd do was I'd sit down with them, and start putting in all the 'fluff' like indenting their code, splitting lines with multiple variable declarations each onto their own line etc. Essentially putting in all the 'long cuts' that they'd taken 'short cuts' around in order to 'speed up' the programming.
Often either they or I would see the problem immediately.
These days of course there are tools that do the auto-formatting, but I still try to optimise for readability, simplicity and elegance in my code.
Back to the lines of code as a metric issue (tm) in that scenario where it takes 2x as long to write a line in language X, and that line does twice as much, it is probably at least twice as complex as the 'more verbose' language... and I think this implies that it will be harder to debug and maintain later on. So arbitrarily complex languages are optimal for some internet 'toss off' (one off, toss away) competition, but not necessarily optimal for long term efforts.
Some of these languages that allow you to redefine the language itself on the fly allow you to do 'cool stuff' (tm) but tend to attract people who think they are much cleverer than they actually are. I think it is a regional variation, but all the Ruby fans in my city scare me. The thought of what would happen if I'd spent days or weeks on a bug only to find out it was caused by someone else 'tweaking' one of the core classes and not bothering to tell anyone else... twitch
Code in a more productive (i.e. higher level) language is concise because it hides extra complexity you don't need to think about at the time, and is therefore simpler, not more complex (in the second sense).
In Python: for obj in seq: ...
In C++: std::base_class<template_type>::iterator obj; for (obj=seq.begin(); obj != seq.end(); seq++) {...}
The reduction in lines for Python doesn't make it more complicated, it makes it cleaner and easier to express ideas.
Another example of where this is true is automatic memory management. A line of Java code can do all the memory management that would take many lines in C. This is a mostly good thing. In this case, less lines = better.
One of the many nice things about Python is that whitespace - specifically, indenting - is significant.
Another problem with it is that I can't just comment out, say, an if, and have the code under it run automatically. I have to go through and reformat the business, leading to potential bugs. This also plays into refactoring as well.
Basically, I find the whitespace rule something that should have been left in the 60s with punch cards.
I am not a fan of Python. I consider Common Lisp and Perl to be fundamentally less ambiguous languages.
Interesting. In several years programming with Python, I've had exactly zero bugs based on whitespace being significant.
> I can't just comment out, say, an if
What I do is comment out the if statement and add another if statement under it, like so:
I always grep my code for FIXMEs before pushing to production, and have my text editor configured to highlight FIXMEs and TODOs.By that, what I mean is this: if you give more thought to a program one tends to be able to say more and more but with less and less, and without making the code incomprehensible or complex.
Interestingly, when I show my code to a scheme veteran, he would often re-write it to make it even shorter and clearer. So I have a long way to go.
So line for line , yes it takes me more time to churn out scheme, but if I give the same amount of time to c++/java it doesnt compress as much. But, and here is the cool part, if it does, I can often trace it back to scheme way of thinking. This has happened a lot when programming with C++/STL.
I think this is debatable. After all - a function call compresses multiple lines into one as well.. And you want programmers to factor the repeated code into function calls.
The "dynamic redefinition of the language" is the same thing except at a different level. The problem is not with the lines of code per se - it is whether the semantic compression is worth the learning curve, and the "surprise" factor.
Think C++ templates, for example, with a "+" operator that would do something "non-intuitive". The "non-intuitiveness" may vary - and it is a fine art to not cross the line.
FWIW, I've done the exercice in Lua, ~ 1 hour and 107 lines (and for some reason a different order of outputs, which I did not want to debug since I went for a nice dinner with a couple of beers earlier today) - https://gist.github.com/752460
That's a good goal. One advantage of Python is that indentation etc. are required. You can't take a 'short cut' and write messy code, cause your code won't run.
This is such baloney.
I've written hundreds of thousands of lines of code in Algol-esque languages, and just about every last line (by which I mean > 99.9%, with possibly exceptions once in a blue moon where I accidentally hit tab before I compile or something like that, which just gets cleaned up the next time I reformat anyways...) of that was properly indented. It's just too easy not to do, and I'd fire, almost immediately, anyone that was checking in code that wasn't indented properly.
Really. Where are all these horrible developers that don't sanely indent their code? To hear the Python folks tell it, they're fucking up codebases all over the place with their carelessness, but I've never personally experienced any pain whatsoever over this. Certainly nothing that a simple CMD+i can fix...
Hell, different newline types interacting badly with version control have caused me so much more trouble than indentation, and Python's not immune to that. Nor is it immune to people's editors globally forcing tabs to spaces or vice versa.
/78698/37395746288: Koch 8 Sud Hab Tirol
What did I miss in the rules?
The number you mention allows to get into the branch with the 'Koch o"d' - so no digit can be selected.
(I was reading the requirement as "If at a particular point no sequence of words from the dictionary can be inserted to complete the number, a single digit can be inserted" -- which still makes a lot more sense to me than the actual requirement!)
Sadly, no programming language could have helped me with this bug. :)
https://gist.github.com/752506
https://gist.github.com/752455
The ability of language X to solve a problem depends on:
1. The expressiveness of the syntax to capture algorithmic complexity. 2. The availability of tools to support the creation of the code. 3. The ability of the typical programmer, conversant in the language, to write the code. 4. The costs involved in maintaining the code over its lifetime.
Rendering all these dimensions into a single metric seems simplistic at best, undermining the argument as to which set of tools and technologies are deserving of more attention and ultimately plays into the hands of the people who wish to render programmers and programming into pure commodities.
I tend to not use separate URL mappers in controllers but have nested functions that define both logic and layout simultaneously. (Basically a tree structure of function definitions where the ending subpage elements are leaf nodes, and nodes render wrapper base templates on post traversal unless it was an ajax request). I can post some code if this doesn't make sense. Obviously not defining new variables nor using classes is extremely unpythonic and could pose difficulties for group work... minimum number of lines while fun isn't everything!
It took me about 2 hours (compared to a range of 2 to 8.5 hours for the other Lisp programmers in the study, 3 to 25 for C/C++ and 4 to 63 for Java) and I ended up with 45 non-comment non-blank lines (compared with a range of 51 to 182 for Lisp, and 107 to 614 for the other languages).
Seems like the programmer, not the language, is the biggest determinant. If you have the Java programmer who wrote it in 4 hours, and the Lisp programmer who wrote it in 8.5 hours, you're better off writing it in Java!
If you want to write concise programs quickly, using the same language Peter Norvig uses might help, but being as good as Peter Norvig helps more.
It takes a different type of programmer to even want to venture to learn lisp in the first place.