Ask HN: Programming language power and 10 years to mastery
I’m around year eight of hacking away professionally and bumped against the dramatic power differential in programming languages when I primarily became a database developer. I started "thinking in sets" and found that I was much more effective. SQL got more work done in fewer lines of code in a simple way than hooking up my J2EE java junk via JDBC in more traditional programming. But it took the direct use of SQL for a couple of years before the experience pushed my coding to the next level. Much of this benefit was due to the fact that the apps I worked on were really data driven and "fit" the relational data model well, coupled with the fact that I was in a role that allowed me to do architecture design and low level implementation. And tear the design and implementation apart to do it again after finding all the nuances in the app and data.
The switch from imperative programming languages to a declarative programming language to take advantage of SQL has made me curious about Lisp and functional programming. I have an idea for a web app and I’ll use it as a vehicle to test out Lisp. Worst case, I pick up a language that purports to have almost mystical powers (all hail the mighty macro). Best case, a nice startup could grow out of the project. But the real point is to get real hands-on experience with a more powerful language than the standard C, Java, Perl, and SQL stuff I’ve already done. Plus, I’d hate to miss out on some key part of a hacker’s personal growth over that 10 year march . . .
That’s a lot of background to get to two simple questions.
Do any of the HN readers have any tips or advice on speeding up the transition to "thinking in Lisp"? I’ll probably be hacking away using Practical Common Lisp and On Lisp as my guides (SICP is probably somewhere in the mix). Have any of you followed a similar path as you head towards a decade of hacking and did the Lisp (or other fp language) dance grow your skills in a big way (ie, now I’m a faster developer in general or better algorithm designer or communicator of code ideas)? It would be great to hear specifics.
Thanks.
43 comments
[ 3.3 ms ] story [ 96.5 ms ] threadHaskell changed the way I program because laziness can allow you to express your code in much clearer and simpler ways - this encouraged me to use more memoization.
Functional programming is definitely where we want to be in theory, so if practice permits, use it!
Also consider learning a dynamic 'scripting' language like Ruby or Python. They have some functional aspects, and will teach you programming style and idioms that you probably haven't picked up from Java and C.
The big advantage to picking up one of these Lisp-like languages (rather than Lisp itself) is that with their corresponding web-frameworks (Rails and Django), you will be able to put up some pretty cool apps pretty quickly. This helps your motivation as well as your bottom line personal ROI (they're both pretty marketable skillsets).
I don't think you lose all that much, as far as the experience of picking up a functional paradigm goes. (This last bit is not fully backed up by experience with Lisp on my part.)
Lua is very Scheme-y, though, much the way Javascript is.
While I don't use Ruby much (Python and Lua fit that niche for me), it seems more Lisp-y than Python.
But then he goes on to explain how and why Python got more functional programming features.
Also, I based my assumption that Python is "lisp-like" on a statement I remembered from one of PG's essays:
"Each one is progressively more like Lisp. Python copies even features that many Lisp hackers consider to be mistakes. You could translate simple Lisp programs into Python line for line."
If there is more context to all this that I am missing, then my mistake. Anyway, I wasn't looking to make any detailed comparisons between languages for the OP. I was just saying...hey, Python or something else could be fun to learn...
Python's lambdas are quite a bit less powerful than any Lisp's, and it doesn't have any tail call optimization. It's like, you can kinda sorta do functional programming in C, you just need to recast function pointers and do manual memory allocation.
Python is carefully designed to encourage programming in a certain relatively straightforward style, and while you can force some FP techniques into it, it doesn't sit well with the rest of the language, and it's kind of unfair to anybody else who will have to deal with the code down the road.
While I don't have much experience working with web frameworks (not my thing at the moment), Python is a very good all-purpose language.
Well, I picked up PG's book and was starting on the language, when I came up with a project I wanted to get up and running quickly. So, I figured Django would help me get something going (it was either that or Ruby/Rails).
What I failed to ask, I guess is...is there an equivalent web framework for Lisp? I guess I might have gone in that direction if I knew of one.
http://javascript.crockford.com/little.html
it's the lispiest (or fp) framework i used (i used prototype before and it felt like java)
http://www.amazon.com/JavaScript-Good-Parts-Douglas-Crockfor...
It's interesting to point out that, while not optimized for it, Javascript (don't think ruby can do this, since you can't pass blocks to blocks) can perform tail recursion: http://warfangled.com/js_examples/fib.js
He mentioned he knows Perl, which is arguably the lispiest of the lot (not in syntax, but in sheer number of lisp-inspired features). But perhaps he "knows" Perl in the way that one who knows how to whip up a one to twenty line script to do something simple but repetitive, and then throw it away.
My personal history goes something like QBasic -> Visual Basic -> PHP -> Java -> C# -> Haskell.
Since Haskell is a purely functional language there's no cheating like you could in, say, OCaml. This forces you to learn how to work with immutability, which I have become a big fan of. When working with C# I try to avoid mutability as much as possible (naturally this is quite a bit uglier than in Haskell, but it does avoid some problems). Most of my C# projects now acquire a Haskell.cs file fairly quickly, containing things like a Tuple class, zip, >>=, etc.
In the time I've spent learning it, Haskell has quickly become my favourite language. It's not perfect (my current main wishes are existential types and an extensible record system), but it is considerably less painful than the other languages I know.
As for speeding up the transition, I can't offer anything other than just diving in. At some point it will just click and you will wonder what all the commotion is about (another monad tutorial? why?). Granted, I still don't have the slightest idea about why I would want to use a hylomorphism or what to do with a comonad, but perhaps I will some day. Fortunately you don't need them in everyday Haskell programming :)
afaik hn doesn't use sql/rdbms either, i guess it's because threaded forum is better suited using a tree-like data structure instead of simple table
i stopped using sql (after reading pcl and seeing the fact of hn) and just having a lisp process running, writing to disk every 5 mins
i'm now experimenting with nginx's proxy_pass and upstream to many lisp unix sockets (weighted round robin)
virgin territory for me, it's quite fun so far ... foolish and hungry :D
Why is this figure being bandied about as though it were an actual scientific fact?
Why do you think so? What actual evidence is there for it?
This sort of behavior is why, even among scientists, a generation often needs to die out before new ideas can be seriously evaluated. (http://en.wikipedia.org/wiki/The_Structure_of_Scientific_Rev...)
Edit in reply to your edit:
Norvig's theories and conclusions are most certainly subject to being proven wrong by anyone who can manage to do so. You make it sound as if someone is attempting to force someone else to agree with his theories and conclusions, or to prevent someone from agreeing with opposing (or any other) set of conclusions and theories.
I see no evidence of that, and come think of it, I do not see evidence of that figure "being bandied about as though it were an actual scientific fact", if you can provide evidence to the contrary, please, by all means do so, or better yet, disprove Norvig's theories and conclusions yourself, no one is stopping you.
A second excellent book is "Higher Order Perl", which shows how Perl, also, shares a lot more in common with Lisp than most developers realize. I'm on my 13th year as a professional developer and 27th as a hacker, and I still find some of the concepts in HoP challenging.
I would also recommend learning XSLT. It's pure declarative, side-effect-free programming. If you're used to procedural code (including OOP), or even functional code, you'll still have to reattach your head sideways to learn to use XSLT effectively. It's well worth the challenge; there are problems that would be very difficult to handle in other languages which XSLT handles with ease; it's a powerful tool for your toolchest.
As for improving my skills, that's an unequivocal yes. I think the biggest eureka moment came when I understood the idea of first class functions. That opened many doors to better code.
Before, I used functions primarily like a writer uses paragraphs: to break up the narrative into chunks. After SICP, I saw functions more like (for lack of a better term) filters. They transform their input into output. The transformation is well defined, and the natural boundaries between functions become clear rather than arbitrary.
Actually that is the power of Lisp. In Lisp, batch or stream data processing is not only straightforward, but the way to think and program.
For me, I need to be able to create something tangible. Having access to all the Java libraries gives me a lot of cool things to import, then play around with in functional ways.
http://code.google.com/p/clojure-contrib/wiki/DatalogOvervie...
Both are very worthy goals, but I don't know if using Lisp, Haskell or erlang to write a web app is really a great idea right now. (Sorry PG). I think that is was true that writing web apps in Lisp probably gave you a huge leg up against the competitors in the mid 90's. I don't really know if that's the case now.
Languages like Python and Ruby have tons of libraries, frameworks and documentation that really help you get productive making web apps. That's a huge plus in my book.
Then again, maybe you're interested in rolling your own libraries and framework ala arc. If that's your thing go for it. I'm sure you'll learn a lot along the way.
I also found learning OCaml to be a real eye-opener, both for learning FP techniques (it emphasizes a different set of them than Lisp) and how to work with, rather than against, static typing. (_The Little MLer_ helped, as well, and is at least as good for learning types as _The Little Schemer_ is for recursion, IMHO. Highly recommended.)
There's not a whole lot of material published in English on OCaml (the Joshua Smith book is terrible!), but there's a translation of a French O'Reilly book online (http://caml.inria.fr/pub/docs/oreilly-book/), and it's quite good for learning the main ideas. The first few chapters are on functional programming, imperative programming, and then their relative strengths & when to use each. (Their emphasis on using FP when appropriate, rather than treating it as the new best thing ever, suggests more insight to me than the blind enthusiasm for languages that tends to cycle through here and reddit.)
Also: OCaml is not without its flaws. In particular, its errors can be cryptic, and it's standard library has some infuriating omissions. The language is fantastic for some kinds of problems (ML's forte is anything involving complex data structures), though, and it's a great way to learn some very useful techniques.
With web app backends, for example, most of the problems somehow relate to difficulty maintaining state over widespread, always-on networked systems - things like authentication, sessions, concurrency, large-scale, flexible datastores. So the systems architected gradually get more and more tuned towards serving this particular goal with little programmer effort. And because those problems are a good match to functional programming - because it eliminates state - functional has become popular.
But if you were building an embedded system, statefulness is less of a problem: there's usually only one thread, and all memory can be mapped in an orderly way. Your performance constraints are reliant on a single fixed platform, so the methods of inspecting and debugging code change.
So to get back to the original question, think about reframing your familiar problems in a way that would simplify the solution with a functional technique. Preferably, look for simplifications that are really big changes and throw out traditional assumptions - the use of relational databases in webapps is one common target for this. Even if you get it wrong you'll get a better understanding of where the functional solution can and can't work.