Ok lets say i want to program a webcrawler in lisp & c# and cant figure out how to do it. I simple go and google it, right?
how to crawl web page in lisp - returns unrelated websites. i still have no clue how to do it.
how to crawl web page in c# - second link returns me a solution to my question.
So why would i waste my time if i can get things done faster in more popular programming languages? English or Esperanto?
This poem was actually written satirically by Frost about going for walks with a friend who always complained afterward that they should have taken some other route and acted as if it would have been significantly better... when in reality it would have made no real difference:
Frost's biographer Lawrance Thompson is cited as saying that the poem's narrator is "one who habitually wastes energy in regretting any choice made: belatedly but wistfully he sighs over the attractive alternative rejected." According to the Thompson biography, Robert Frost: The Years of Triumph (1971), in his introduction in readings to the public, Frost would say that the speaker was based on his friend Edward Thomas. In Frost’s words, Thomas was "a person who, whichever road he went, would be sorry he didn’t go the other."
If you need to create a webcrawler out of pre-made pieces, you need to know what to look for. In this case, it would be a high-level networking library and an HTML parser. I haven't looked, but I'm sure you could find these libraries in Lisp.
If you don't know what a webcrawler entails, then you might not be the right audience for Lisp as it exists at the moment.
If I don't know how a webcrawler works, I would look at an programming language independent source like Wikipedia. After that I know what components I have to search for.
If you've finished building a webcrawler, please create a website where you explain webcrawling for (common-)lisp. So the next one will not have the same problem :-)
If you want to make your own DSL for crawling web sites, and you know Lisp, you'll probably be able to get it up and running pretty quickly. That's the kind of thing Lispers do for fun. Otherwise, just look at the language and what people say about it and decide for yourself if you're interested.
> The YC internal software is mostly written in Ruby on Rails. We also use: JavaScript, React, AWS, and PostgreSQL. Some experience developing web apps would be helpful, but we don't care if you've used Rails before.
I think it would be valuable for other commenters to share why they did or did not choose Lisp for their most recent project(s).
I chose Go for a recent project, and TypeScript for another. Both languages make getting started very easy: they have straightforward installation, good tooling, and a lot of support for when things go wrong.
I did not seriously consider LISP, and I think the main reason is that it's a very confusing landscape. Google "how to install LISP" and you get a page like https://en.wikibooks.org/wiki/Common_Lisp/First_steps/Instal... with half a dozen options. It's hard to choose and most of these do not look promising. For example, Steel Bank Common Lisp is one I have heard of, but their binaries are out of date and they use SourceForge (shiver). Arc is another, but their installation instructions do not inspire confidence with disclaimers like "don't use the latest version." My concern is I will spend two weeks diving in before hitting some hard limitation, and I'll be stuck.
(Clojure is surely the best known modern LISP, but I do not want to depend on the JVM.)
Speaking as an outsider, I think LISP would gain a lot more traction if its community coalesced around a single implementation. (Sorry if that's a clichéd argument). This would allow for investment in areas like documentation, platform support, tooling, etc. The functional language community went through this and the result was Haskell, which is excellent in all of those areas.
Usually apt-get doesn't feature the most recent up to date release, also you assume everyone uses a Debian derived distribution of Linux, bad assumption. I'm on openSUSE and I'm sure many are on Windows or Mac.
If you need the absolute most recent release of a language to write code in it, that in my opinion is a bad sign -- maybe it's still a bit new and evolving rapidly, if there's a runtime component it might be awkward to distribute to machines you want to run on, and setting the dev environment up becomes a little painful which can discourage others from contributing. Not a dealbreaker, especially if you're just experimenting or doing a small private project, but for anything serious I would be worried if I couldn't develop and run it with whatever version of a language is shipped with the major Linux distros.
I don't know if this is still the case, but for SBCL on Debian installing with apt-get would get you an incredibly outdated version, and all that common-lisp-controller nonsense. The standard advice was NOT to install with apt-get, but to install manually.
Lisp is a family of languages. All the dialects are good in their own way - "standardization", if such a thing even makes sense, would be counter-productive.
This paper describes the situation in the mid-1980s as a "lazy Tower of Babel" with competing lazy-functional languages like Miranda, MLs, Orwell, Clean, Ponder, each good in its own way, but none of them with critical mass. Thus they decided to create Haskell as a new, common language. No doubt each language designer felt that Haskell compromised on their particular vision, but rallied around it anyways, resulting in a far more potent force for lazy functional programming than ever could have been achieved working separately.
Why does this not describe LISP today? Honest question.
Lisp thrives in Babel, while the other kinds of languages may be harmed by it. Lisp is a meta-language and it facilitates building a Babel tower of eDSLs for each and every little project. It is the main source of its unmatched power.
But this doesn't require the proliferation of implementations. Each little project doesn't require its own garbage collector, for example.
The paper I linked discusses some of the "practical" areas of Haskell: profiling, debugging, packaging, FFI, compiler errors and optimizations, etc. Surely there's a lot of energy being expended replicating this stuff across the various Lisp implementations?
There happens to be wide community consensus around a single Haskell compiler, GHC, at the moment, but that's a contingency and there are some other compilers, interpreters, and stuff like GHCJS. Profiling and debugging support is a feature of GHC (and other implementations), not of the language standard.
Common Lisp has unified packaging and FFI and stuff through various community projects like ASDF and Quicklisp.
If you're talking about Lisp in the general vague sense of "Lisps" then there can't be general interoperability because that umbrella term covers lots of very different languages with no formal common ground.
Even GC cannot be a common ground for everything. Take a look at Harlan, a Lisp-like language with a region-based memory management. Unification harms diversity and flexibility.
Scheme is another language with its own common standards.
The way people are so quick to talk about "Lisp" as a general meta-language is kind of confusing.
We could invent a definition by which Ruby, Python, and Perl are "really the same language" that just failed to standardize, but that would be obviously silly.
You could say Ruby, Python, Perl, C, Java, etc are really ALGOL-like languages whose data structures are generally mutable, and whose syntax is made for expressing how the state of the program is mutated in-place.
Whereas LISP-like languages have data structures that are generally immutable, and whose functional orientation results in expressing the computation of new data from old (mostly) without mutating in-place.
The thing about imperative programming languages is that the paradigm matches up to the mental model of "a computer" that most programmers would recognize and find comfortable. Without having any concrete evidence, I have a hunch that this paradigm difference is what deters new developers the most. And stacked on top of that you have the non-ALGOL syntax and the lack of libraries.
Your opinion is about an idea of Lisp that's simply not true. Both Common Lisp and Scheme are pervasively mutable and imperative constructs occur just as often as in "Algol." Clojure diverges from this with its nice immutable data structures, but that's a rather novel and distinguishing feature, not a core trait of "Lisp."
I'm curious about where the idea originated that Lisp is an especially non-mutating language. Some of it is just that it's a garbage collected language with first class functions, so it's easy to classify it as "functional" which has associations to "pure functional." But Lisp is very far from a pure functional language. It's more like a general multi-paradigm language—like Perl and Ruby.
Common Lisp _IS_ the consolidation of various Lisp dialects that you've asked about.
Back in the 70s and 80s there are numerous Lisp dialects (way, _way_ more then there are today, since each dialects has their own sub-dialect and so forth), and so Common Lisp was created back in the 80s (the work started in 1981 and finished in 1994) to unified various Lisp dialects into one common language.
And Common Lisp accomplishes that objective, well mostly anyway. Soon(-ish) most people, companies, and institutions ported their Lisp code base to Common Lisp, and by the late 80's and early 90's, when you say Lisp most people will (correctly) assume that you mean Common Lisp, not MacLisp or InterLisp, or some earlier, long depreciated dialects.
Now, about Scheme. Remember when I said that Common Lisp _mostly_ accomplishes its objective of unifying various Lisp dialects? Yeah, basically the Scheme community didn't like how Common Lisp turns out, so they decided to stick with Scheme. And it is understandable why they feel this way, because despite outward appearance, the two languages are very different, with Scheme focuses on small (and mathematically elegant) core language, while Common Lisp, with its focus on being practical, is a huge language with a standard library that includes everything and a kitchen sink. Scheme also has a totally different evaluation model compared to Common Lisp and its own macro system with emphasis on hygienic. Not to mention that Scheme has its own standard, the R*RS (now in its 6th revision, with 7th in the drafting process).
Now, the thing about Common Lisp is that it is very hard, if not downright impossible, to change its standard. This means that it is not easy to make a substantial changes to the language and has the wider community adopt it (and we haven't even talk about changes that will break backward compatibility here). This is one of the reason that Rich Hickey created Clojure as an entirely new Lisp dialect (before this, he had implemented a Java FFI for Common Lisp). Clojure has a new syntax that is far more consistent than Common Lisp (which comes at the cost of breaking backward compatibility), as well as many new and experimental features, such as immutability, asynchronous operations, and the focus on functional programming style. All of these would be very hard to introduce into Common Lisp without significant efforts and time.
> Scheme focuses on small (and mathematically elegant) core language, while Common Lisp, with its focus on being practical
I don't think it's really fair to say modern Schemes aren't focused on being practical. Nor that they're small.
Take a look at Chicken or Racket, and you'll see that they have quite substantial libraries and many features above and beyond the specs that make them much more practical than the core Scheme spec alone.
Quite beside that, there's also the new R7RS[1] spec's "large" language is just that: large. It's also focused on being practical. So I wouldn't say that the traditional Scheme vs CL divisions in terms of size and practicality still hold.
I would say that Scheme is more elegant than CL, and more modern, and less crufty. Unfortunately, there are still far more programmers using CL than Scheme. Hopefully with Emacs' move to Guile that will change.
That said, I still love CL, and would far rather use it than any other language, except Scheme.
The easiest Lisp I would recommend is probably Racket, it is what got me into Lisp personally. After that I would give Clojure a shot, it has some amazing tools like lein.
I think first time setup is a huge key for programming languages. For example Go has the benefit of an amazing out of the box standard library, unlike D and Ruby where you would install via a package manager the right tools to get a web project going, you could do it all from vanilla Go code.
Of course in both Ruby and D you could do it from vanilla code, but it's not as seamless and tried and tested I suppose?
If you are reading SICP or want to, Racket is also a good choice here. I bounced between SICP and The Little Schemer with Racket. I didn't use the SICP language until later on but I found that Rackets documentation was so good that it was pretty easy to fill in any blanks.
All in all, that experience was useful to me to both learn Scheme, learn core concepts, such as recursion, higher order functions, closures and so on whilst also paving a familiarity that has aided in quickly picking up CL, Clojure and also, in my opinion, Haskell.
This is spot on for me. I actually have used lisp in one large project but I haven't used it since. It wasn't my choice then and the guy who's choice it was was a total lisp guru. I don't have his chops. I did enjoy parts of it but not enough to switch
The biggest issue for me vs other options is lack of community / packages
Pick Python, JavaScipt, even Perl and there's millions of solutions to pick from so you can get shit done. Maybe that stuff exists for lisp but I'm unaware of it.
If there was a popular lisp with something like npm, easy integration with non lisp libraries I might consider it.
There's also the issue of perf. The lisp I used was optimized for generating machine code for games. My understanding is most lisps don't perform at that level. Of course as a replacement for Python, perl, JavaScript that's not an issue. But it would be for C/C++ for games
> Clojure is surely the best known modern LISP, but I do not want to depend on the JVM.
Why don't you want to depend on the JVM?
You are deliberate ignoring the most well known modern lisp, which has straightforward installation, great tooling, and lots of support available.
Not to mention the JVM is one of the most powerful and highly optimised vm platforms and has hundreds of thousands of man years invested in high quality libraries.
> Speaking as an outsider, I think LISP would gain a lot more traction if its community coalesced around a single implementation
And surely they would focus around a JVM implementation. The only other platforms that even come close are c/c++, and the .net clr.
Lisp isn't just a concept, it's also a family of implementations. The two significant branches are Scheme and Common Lisp. The only thing they have in common is using parentheses - they're about as different from each other as JavaScript and C.
Scheme is more functional programming oriented, Common Lisp is less so. Scheme is very minimalist, it's most recent standard contains only about 20 syntax definitions! Unfortunately this transformed Scheme into a concept from an implementation. Compare this to CL, which has just about everything and the kitchen sink, and is more of a "concrete" language.
To choose a Lisp is to choose an implementation. I'd recommend playing with all of them. Common Lisp, a Scheme (CHICKEN or Guile), or the "recent" ones like Clojure and Racket. Racket is very much a Scheme - it used to be called PLT Scheme - and has a bunch of PLT superstars building really cool stuff with it.
Clojure, the JVM-based Lisp, has more in common with Common Lisp than any other, but embodies a strong philosophy about identity and value. It has a lot of common with Haskell in that regard, supporting immutable data structures and STM right out of the box. It also has really cool ideas like transients [1], transducers [2] and protocols [3], features which have since crept into other languages.
Clojure is probably the most popular right now, and its ecosystem is growing every day, and can be used for building serious stuff. This doesn't mean you can't build cool stuff with other Lisps! CHICKEN Scheme has a great package ecosystem, so does Racket, Guile lags behind, and for Common Lisp there's a recent "State of the Ecosystem" article[4] that summarizes its ecosystem.
"Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot."
I have a litmus question about this assertion that programming in language X is the secret to writing great software (quote from another pg essay[1]):
> Our hypothesis was that if we wrote our software in Lisp, we'd be able to get features done faster than our competitors, and also to do things in our software that they couldn't... What were the results of this experiment? Somewhat surprisingly, it worked... We eventually had many competitors, on the order of twenty to thirty of them, but none of their software could compete with ours... It must have seemed to our competitors that we had some kind of secret weapon... We were just able to develop software faster than anyone thought possible.
Here's a better experiment:
- Take two teams: one with great programmers, and one with mediocre programmers.
- Force them to swap toolsets.
Question, which team will outperform? If language is the secret, the mediocre team should outperform the talented team.
But that's preposterous. Great developers write good software fast. Bad developers do otherwise. All developers work best in their favorite tools. Because they are adept using their favorite tools.
Great team with mediocre tools would work many times slower (and will be more likely to quit due to boredom). You have to compare equivalent teams instead.
Great team with mediocre tools would come up with their own tools to make up for the limitations of the basic ones. After wasting some time to do that they would then proceed as great team is supposed to :)
If you have a set of "great" developpers and you force them to use a crappy toolset you are not an interresting company for them. They will simply resign.
It so quite possible the long term productivity will be better with the mediocre programmers team regardless of their tooling.
If you have a team of great programmers you also need to give them the possibility to do some interresting stuff.
And that's basically that, when the great programmer will come and ask to change tooling or improve you'll refuse. They'll understand you prevent them to do anything efficiantly and so they'll resign as they waste their time here.
But even with great tools at their disposal, a mediocre team of programmer will perform averagely.
Because you don't just need to have the tool lying around, you have to know how to use it, what is good, what is bad, when to do this or that. if there not somebody behind them to coatch them all the time, they will not leverage it.
When for example a person learn design patterns, they want to use them all the time.
The great developper know the patterns, understood the key concepts behind and know when to use and not use them.
As for the language it is more crutial than you think. It is basically the shared basic knowledge and discussion medium in your company. And your APis are also part of it. If you have high level concepts coming from both your API and programming language and your developper know them they'll think directly in theses high level concepts and will be able to ignore the irelevant details.
What make Java more efficiant mostly is that you can forget to think about freeing memory and it is almost impossible to get a core dump. Instead of having to learn and speak the concepts of allocating and freeing memory your developpers will be able to concentrate on other things.
And what make C more efficiant than assembly is that you can actually write easily functions, that you can define structures and that the compiler take care to compute the proper shift in memory to access the differents field and so on.
But java, especially before version 8 is lacking key concepts like basic functionnal programming. So when a developper want to filter a collection, a pretty common operation, he will just iterate, check the condition of the filtering and return a filtered collection. It will be very specific and the level of expressivity of this code will be quite low.
A great developper would at leas isolate that code into a function with meaningfull name or define a generic filter function and pass predicate as anonymous functions. But the lack of proper lambda expression will prevent to fully leverage it.
But once everybody is accustomed to functionnal programming they'll just use filter and a predicate. The intent is much more clear, almost self documenting.
The more expressive you can be, the more common knowledge you can have, and the more you can concentrate on issue key to your program and the less you can spend time on unecessary details, the more productivity you get.
And yes, this start with a good programming language with a good echosystem and a team that efficiant with it.
That's why Java or C/C++ are so often used: they are the easiest languages when you need to hire because more people know them. This doesn't mean they are the best language for your problem at hand.
So, what about Lisp makes it great? I learned some Lisp in a CS course a while ago, and while it certainly seemed interesting, I didn't feel anything special (perhaps I ought to spend more time with it?). Could a pro give me a short list of the biggest advantages, compared to C or Python?
The major advantages are extreme modularity and conciseness - you can write much shorter expressions than competing languages, like C, Python, Java etc. Macros are another big feature, which allow you to generate code easily and lastly the ease with which you can write self-modifying code (Homoiconic code) is often praised.
I don't think you should choose the tool before you know the problem. Computer programming is a pretty vast domain.
Lisp was probably a good tool for pg's startup. It was an ecommerce builder, most lisps are great at metaprogramming and creating DSL's. But his startup also used C and some other languages for different problems.
What I think is great about Lisp is its idealistic simplicity.
(function parameter1 parameter2)
Everything is a list, pretty much everything that doesn't look like a list is just syntax sugar.
My opinion is: unless you have a hard requirement, just use the language you already know and like.
If you need to squeeze every tiny bit of performance, you might just need to use C++ or C.
If your client wants a complete website in a ridiculous short amount of time, going for python,ruby or even php would be a sane choice.
If your problem involves DSL's, metaprogramming or impressing people with how alien your code looks, check out some lisps, maybe Haskell.
Like Smalltalk, and to some extent Erlang, Lisp is an integrated programming platform that loads and reloads code while the system runs. Learning Lisp is not just learning a language grammar—it's learning a habit of development that is incremental and live in a way that's miles away from C coding and quite far from most typical styles of Python coding.
You open your editor and connect to Lisp. Then you spend your time interactively defining functions, changing them, defining variables and inspecting them, and so on.
This has consequences that you need to experience or see to understand how it plays out in practice. You tend to start coding in a different way. Paul Graham calls it bottom up programming.
There are also synergistic effects with features like macros which enable radical kinds of convenient code generation / DSL-building, and conditional restarts which are like exceptions but fundamentally more powerful in a way that helps greatly when doing interactive coding, the multiple dispatch object system, the online documentation system, and so on and so on.
Lisp is shock full of good ideas that are still absent from most other languages. But mere bullet lists aren't convincing, like in so many other situations.
I wrote a web service in Lisp and just for convenience I wanted to have it incrementally recompile and reload its own source code on every request if the commit hash of the working tree had changed. This was a three line hack.
I also generate a memory image every time I make a deploy, which means that the server starts in a finger snap including the whole runtime for all this dynamic stuff.
Another advantage for me is the HyperSpec, an excellent hyperlinked reference of the whole standard language. It's an industrial quality manual written by literate professionals and so it's exhaustive, correct, and mostly unambiguous.
I also simply enjoy how Emacs with Lisp is a world-class IDE. Right now I'm mostly offline with just slow pricey 3G but I made sure to install a bunch of Lisp packages and the HyperSpec and it's a joy to code using just Emacs and SBCL in the background. I can look up any documentation, semantic tab completion and argument list popups work perfectly since the editor is aware of all the code and the namespace rules and so on.
I just don't know any other language that's so pleasant and fun to develop with. (After learning it, of course.)
People call Lispers smug because they love the language more than any other, but I don't feel smug because I didn't come up with the thing, I just learned it and like it. I'm humbled by the engineering and quality of the tools and everything. It's not some weird academic obsession. It just makes me happy on a daily basis.
The obvious "killer feature" of LISP is macros. One concrete example where I saw the power of macros in a problem I was working with was when I saw how clean the parser generator library in Racket was.
Instead of being like yacc, which uses an external DSL to define the parser, with its own syntactic conventions, the Racket parset generator is built with macros. The reason yacc has to be a metaprogram instead of just a library is because of how the semantic actions are "chunks of code" which you can't really represent in C but in LISP its easy to represent "chunks of unevaluated code" in your program, which lets you define the parser generator as an embedded dsl implemented via macros.
Anyway, the reason I found this nice is that in Racket you can define your own macros to abstract over the repetitive parts of the grammar description. For example, in yacc you need to write multiple rules to describe a comma-separated list
/* Use left recursion for efficient stack use
and reverse at the end so the list is in
left-to-right order */
arglist:
arglistrevdeclsx { $$ = reverse($1); }
;
arglistrev:
/*empty*/ { $$ = NULL; }
| arglistrev ',' arg { $$ = cons($2, $1); }
;
and this pattern repeats for every other list you might have in the grammar. In yacc there is no alternative but to copy and paste this boilerplate but in Racket you could define a helper macro that would let you write grammars that look a bit like this:
;; create a non-terminal called arglist
;; that is a list of "arg"s, separated by commas
,@(makelist arglist ',' arg)
;; and this code can be reused:
,@(makelist decllist ',' identifier)
To answer some of the people in the thread that have asked why you might want to choose a lisp, here's an answer.
I have chosen a lisp (Clojure) because:
- Clojure allows me to write code that is robust, fast, and simple
- because it's simple, most Clojure code is easy to reason about, now, tomorrow and six months from now
- focus on the data (JSON, html parsing, whatever) and the transformation of it, which is arguably what we do most of the time in modern backend and frontend (React-based) applications
- pervasive and strongly encouraged immutability, managed concurrency, discouraged callback hell, etc which makes it hard to write bad code in the first place
- I can change functional+dynamic+immutable code faster than any other code, which allows faster experimentation and iteration on features
- it works very well for both backend services (Clojure/JVM), and browser applications (ClojureScript/Google Closure compiler), with the language staying 97% the same
Sadly this article (IMO) is not one of PG's better ones, and I am quite surprised by the lack of self-awareness around it - it boils down to 'if you're clever, you'll use and appreciate this language', when actually there are practical reasons as to why lisp is not so popular.
It actually feels really closed-minded - like you've walked into a conceited professor's office and told him you have doubts about one of his ideas and he's looked down his glasses at you and talked about how most people prefer McDonalds to amuse-bouche at Pierre Gagnaire in Paris.
How about considering whether there are reasons other than 'it looks foreign so lesser programmers don't bother with it'? If you google around you find there are plenty of people who suggest practical reasons (too many fragmented implementations, hard to debug, etc.) as to why it's not caught on, and in fact a lisp like clojure seems like an attempt to learn lessons about what might work better or not. But no, we have these patronising proclamations and the article is over.
Don't get me wrong, I'm not anti-lisp per se, but I've come to think as I've seen more of the programming world (and worked with good developers as well as bad) that language has less of an impact on a project than I previously thought.
What I hate is the smug attitude a lot of lispers have, as if it's some incredible magical secret that only smart people can understand and if only we'd all see the light this world of shit code would turn upside down. It's hard to prove anything in the complicated world of programming, and I just don't buy this idea that it's magically better.
We should admit the subjective nature of these things, admit that it's really hard to tell what kind of a difference a language makes and instead of smug patronising talk of how wonderful language X is, point out what is unique about the language and stay humble.
I much prefer PG's more practical detail-orientated explorations of lisp and what makes it powerful in the first place to this one.
Good question. For Paul Graham it certainly worked.
By combining a great team of competant people, a small startup environement were you can concentrate on getting thing done and by using powerfull tools they made their software, managed to have it evolving faster than their competitor and then sold it to Yahoo.
Ironically since the tool has been brought back by Yahoo its growth stopped and nobody know of it anymore. Is it because of the different people working on it? Is it because they had now a different management structured that cared more of politics? Or was it because they decided to recode everything in C++ ?
To me that's likely a combination of all the things.
That's the key point. A great programming language and echosystem is a great start. But that's not enough. You also need pragmatic management that make decision for the good of the company and not for politics and you need to have employees that care of their job.
At the end, you should know if you are in a model of small efficiant teams or huge average teams. Most company start with the former and end up in the later as they grow. Ultimately that why the next start up has a possibility to beat them. You would never be able to beat microsoft if everybody inside had a startuper mind-set. They would own the world.
That why people fear google so much. I'am pretty sure they have lot of politics and are already more in the later phase than the former but they at least managed to keep bright people working for them.
I definitely agree that the most popular languages, tools, frameworks, paradigms, etc. are by definition not the best. These are popular because they're used by average programmers. They get the job done, though not in the best way, and are popular because of that (see for example Javascript).
I found that Clojure (my first Lisp) was the easiest language to pick up that I've ever learned. Partly, that's because I've been doing this a long time, but I also think it speaks volumes to the simplicity of Lisps and Clojure in particular. I have no doubt that I could teach it to programmers and non-programmers alike rather quickly and I really wish it was at least studied, if not used, more so that people would have more experience with functional concepts. Even just a week or two of studying it is quite beneficial and a good contrast to typical OO languages.
66 comments
[ 2.7 ms ] story [ 123 ms ] threadSo why would i waste my time if i can get things done faster in more popular programming languages? English or Esperanto?
And sorry I could not travel both
And be one traveler, long I stood
And looked down one as far as I could
To where it bent in the undergrowth;
... ...
I shall be telling this with a sigh
Somewhere ages and ages hence:
Two roads diverged in a wood, and I—
I took the one less traveled by,
And that has made all the difference.
Frost's biographer Lawrance Thompson is cited as saying that the poem's narrator is "one who habitually wastes energy in regretting any choice made: belatedly but wistfully he sighs over the attractive alternative rejected." According to the Thompson biography, Robert Frost: The Years of Triumph (1971), in his introduction in readings to the public, Frost would say that the speaker was based on his friend Edward Thomas. In Frost’s words, Thomas was "a person who, whichever road he went, would be sorry he didn’t go the other."
If you don't know what a webcrawler entails, then you might not be the right audience for Lisp as it exists at the moment.
For common lisp, the first thing I would search for is a HTTP Client and a HTML Parser. https://duckduckgo.com/?q=common+lisp+http+client https://duckduckgo.com/?q=common+lisp+html+parsing
CLiki.net is always a good resource.
If you've finished building a webcrawler, please create a website where you explain webcrawling for (common-)lisp. So the next one will not have the same problem :-)
EDIT: found the link
https://jobs.lever.co/ycombinator/8703c1d9-af67-4826-90e4-74...
> The YC internal software is mostly written in Ruby on Rails. We also use: JavaScript, React, AWS, and PostgreSQL. Some experience developing web apps would be helpful, but we don't care if you've used Rails before.
It's easy to make a Lisp, but hard to make them compatible. So you get one language with a billion non-communicating dialects.
I chose Go for a recent project, and TypeScript for another. Both languages make getting started very easy: they have straightforward installation, good tooling, and a lot of support for when things go wrong.
I did not seriously consider LISP, and I think the main reason is that it's a very confusing landscape. Google "how to install LISP" and you get a page like https://en.wikibooks.org/wiki/Common_Lisp/First_steps/Instal... with half a dozen options. It's hard to choose and most of these do not look promising. For example, Steel Bank Common Lisp is one I have heard of, but their binaries are out of date and they use SourceForge (shiver). Arc is another, but their installation instructions do not inspire confidence with disclaimers like "don't use the latest version." My concern is I will spend two weeks diving in before hitting some hard limitation, and I'll be stuck.
(Clojure is surely the best known modern LISP, but I do not want to depend on the JVM.)
Speaking as an outsider, I think LISP would gain a lot more traction if its community coalesced around a single implementation. (Sorry if that's a clichéd argument). This would allow for investment in areas like documentation, platform support, tooling, etc. The functional language community went through this and the result was Haskell, which is excellent in all of those areas.
Clojure: http://clojure.org/getting_started
Racket: http://docs.racket-lang.org/getting-started/
Common Lisp: https://www.quicklisp.org/beta/
Lisp is a family of languages. All the dialects are good in their own way - "standardization", if such a thing even makes sense, would be counter-productive.
This paper describes the situation in the mid-1980s as a "lazy Tower of Babel" with competing lazy-functional languages like Miranda, MLs, Orwell, Clean, Ponder, each good in its own way, but none of them with critical mass. Thus they decided to create Haskell as a new, common language. No doubt each language designer felt that Haskell compromised on their particular vision, but rallied around it anyways, resulting in a far more potent force for lazy functional programming than ever could have been achieved working separately.
Why does this not describe LISP today? Honest question.
The paper I linked discusses some of the "practical" areas of Haskell: profiling, debugging, packaging, FFI, compiler errors and optimizations, etc. Surely there's a lot of energy being expended replicating this stuff across the various Lisp implementations?
Common Lisp has unified packaging and FFI and stuff through various community projects like ASDF and Quicklisp.
If you're talking about Lisp in the general vague sense of "Lisps" then there can't be general interoperability because that umbrella term covers lots of very different languages with no formal common ground.
For Common Lisp, the open source ones are SBCL and Clozure CL, and the proprietary ones are Allegro CL and LispWorks.
For Scheme, there are Racket (PLT Scheme), Chicken Scheme, and Guile.
I'm quite certain that the majority of people who use Common Lisp and/or Scheme use one of the aforementioned implementations.
Scheme is another language with its own common standards.
The way people are so quick to talk about "Lisp" as a general meta-language is kind of confusing.
We could invent a definition by which Ruby, Python, and Perl are "really the same language" that just failed to standardize, but that would be obviously silly.
Whereas LISP-like languages have data structures that are generally immutable, and whose functional orientation results in expressing the computation of new data from old (mostly) without mutating in-place.
The thing about imperative programming languages is that the paradigm matches up to the mental model of "a computer" that most programmers would recognize and find comfortable. Without having any concrete evidence, I have a hunch that this paradigm difference is what deters new developers the most. And stacked on top of that you have the non-ALGOL syntax and the lack of libraries.
In my opinion, anyway.
I'm curious about where the idea originated that Lisp is an especially non-mutating language. Some of it is just that it's a garbage collected language with first class functions, so it's easy to classify it as "functional" which has associations to "pure functional." But Lisp is very far from a pure functional language. It's more like a general multi-paradigm language—like Perl and Ruby.
Back in the 70s and 80s there are numerous Lisp dialects (way, _way_ more then there are today, since each dialects has their own sub-dialect and so forth), and so Common Lisp was created back in the 80s (the work started in 1981 and finished in 1994) to unified various Lisp dialects into one common language.
And Common Lisp accomplishes that objective, well mostly anyway. Soon(-ish) most people, companies, and institutions ported their Lisp code base to Common Lisp, and by the late 80's and early 90's, when you say Lisp most people will (correctly) assume that you mean Common Lisp, not MacLisp or InterLisp, or some earlier, long depreciated dialects.
You can read more about the nitty gritty details in the Evolution of Lisp essay by Richard Gabriel here: http://www.dreamsongs.com/Essays.html
Now, about Scheme. Remember when I said that Common Lisp _mostly_ accomplishes its objective of unifying various Lisp dialects? Yeah, basically the Scheme community didn't like how Common Lisp turns out, so they decided to stick with Scheme. And it is understandable why they feel this way, because despite outward appearance, the two languages are very different, with Scheme focuses on small (and mathematically elegant) core language, while Common Lisp, with its focus on being practical, is a huge language with a standard library that includes everything and a kitchen sink. Scheme also has a totally different evaluation model compared to Common Lisp and its own macro system with emphasis on hygienic. Not to mention that Scheme has its own standard, the R*RS (now in its 6th revision, with 7th in the drafting process).
Now, the thing about Common Lisp is that it is very hard, if not downright impossible, to change its standard. This means that it is not easy to make a substantial changes to the language and has the wider community adopt it (and we haven't even talk about changes that will break backward compatibility here). This is one of the reason that Rich Hickey created Clojure as an entirely new Lisp dialect (before this, he had implemented a Java FFI for Common Lisp). Clojure has a new syntax that is far more consistent than Common Lisp (which comes at the cost of breaking backward compatibility), as well as many new and experimental features, such as immutability, asynchronous operations, and the focus on functional programming style. All of these would be very hard to introduce into Common Lisp without significant efforts and time.
I don't think it's really fair to say modern Schemes aren't focused on being practical. Nor that they're small.
Take a look at Chicken or Racket, and you'll see that they have quite substantial libraries and many features above and beyond the specs that make them much more practical than the core Scheme spec alone.
Quite beside that, there's also the new R7RS[1] spec's "large" language is just that: large. It's also focused on being practical. So I wouldn't say that the traditional Scheme vs CL divisions in terms of size and practicality still hold.
I would say that Scheme is more elegant than CL, and more modern, and less crufty. Unfortunately, there are still far more programmers using CL than Scheme. Hopefully with Emacs' move to Guile that will change.
That said, I still love CL, and would far rather use it than any other language, except Scheme.
[1] - http://trac.sacrideo.us/wg/raw-attachment/wiki/WikiStart/r7r...
I think first time setup is a huge key for programming languages. For example Go has the benefit of an amazing out of the box standard library, unlike D and Ruby where you would install via a package manager the right tools to get a web project going, you could do it all from vanilla Go code.
Of course in both Ruby and D you could do it from vanilla code, but it's not as seamless and tried and tested I suppose?
All in all, that experience was useful to me to both learn Scheme, learn core concepts, such as recursion, higher order functions, closures and so on whilst also paving a familiarity that has aided in quickly picking up CL, Clojure and also, in my opinion, Haskell.
I would recommend Racket whole heartedly.
The biggest issue for me vs other options is lack of community / packages
Pick Python, JavaScipt, even Perl and there's millions of solutions to pick from so you can get shit done. Maybe that stuff exists for lisp but I'm unaware of it.
If there was a popular lisp with something like npm, easy integration with non lisp libraries I might consider it.
There's also the issue of perf. The lisp I used was optimized for generating machine code for games. My understanding is most lisps don't perform at that level. Of course as a replacement for Python, perl, JavaScript that's not an issue. But it would be for C/C++ for games
Another issue is finding collaborators.
What are the reasons you don't want to depend on the JVM that don't apply doubly so to Common Lisp?
Why don't you want to depend on the JVM?
You are deliberate ignoring the most well known modern lisp, which has straightforward installation, great tooling, and lots of support available.
Not to mention the JVM is one of the most powerful and highly optimised vm platforms and has hundreds of thousands of man years invested in high quality libraries.
> Speaking as an outsider, I think LISP would gain a lot more traction if its community coalesced around a single implementation
And surely they would focus around a JVM implementation. The only other platforms that even come close are c/c++, and the .net clr.
Nice "syntactical" touch :)
Let me try a variation:
(shiver 'me 'timbers)
Scheme is more functional programming oriented, Common Lisp is less so. Scheme is very minimalist, it's most recent standard contains only about 20 syntax definitions! Unfortunately this transformed Scheme into a concept from an implementation. Compare this to CL, which has just about everything and the kitchen sink, and is more of a "concrete" language.
To choose a Lisp is to choose an implementation. I'd recommend playing with all of them. Common Lisp, a Scheme (CHICKEN or Guile), or the "recent" ones like Clojure and Racket. Racket is very much a Scheme - it used to be called PLT Scheme - and has a bunch of PLT superstars building really cool stuff with it.
Clojure, the JVM-based Lisp, has more in common with Common Lisp than any other, but embodies a strong philosophy about identity and value. It has a lot of common with Haskell in that regard, supporting immutable data structures and STM right out of the box. It also has really cool ideas like transients [1], transducers [2] and protocols [3], features which have since crept into other languages.
Clojure is probably the most popular right now, and its ecosystem is growing every day, and can be used for building serious stuff. This doesn't mean you can't build cool stuff with other Lisps! CHICKEN Scheme has a great package ecosystem, so does Racket, Guile lags behind, and for Common Lisp there's a recent "State of the Ecosystem" article[4] that summarizes its ecosystem.
"Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot."
-- Eric S. Raymond
[1] http://clojure.org/transients
[2] http://clojure.org/transducers
[3] http://clojure.org/protocols
[4] http://eudoxia.me/article/common-lisp-sotu-2015/
> Our hypothesis was that if we wrote our software in Lisp, we'd be able to get features done faster than our competitors, and also to do things in our software that they couldn't... What were the results of this experiment? Somewhat surprisingly, it worked... We eventually had many competitors, on the order of twenty to thirty of them, but none of their software could compete with ours... It must have seemed to our competitors that we had some kind of secret weapon... We were just able to develop software faster than anyone thought possible.
Here's a better experiment:
- Take two teams: one with great programmers, and one with mediocre programmers.
- Force them to swap toolsets.
Question, which team will outperform? If language is the secret, the mediocre team should outperform the talented team.
But that's preposterous. Great developers write good software fast. Bad developers do otherwise. All developers work best in their favorite tools. Because they are adept using their favorite tools.
[1] http://www.paulgraham.com/avg.html
It so quite possible the long term productivity will be better with the mediocre programmers team regardless of their tooling.
If you have a team of great programmers you also need to give them the possibility to do some interresting stuff.
And that's basically that, when the great programmer will come and ask to change tooling or improve you'll refuse. They'll understand you prevent them to do anything efficiantly and so they'll resign as they waste their time here.
But even with great tools at their disposal, a mediocre team of programmer will perform averagely.
Because you don't just need to have the tool lying around, you have to know how to use it, what is good, what is bad, when to do this or that. if there not somebody behind them to coatch them all the time, they will not leverage it.
When for example a person learn design patterns, they want to use them all the time.
The great developper know the patterns, understood the key concepts behind and know when to use and not use them.
As for the language it is more crutial than you think. It is basically the shared basic knowledge and discussion medium in your company. And your APis are also part of it. If you have high level concepts coming from both your API and programming language and your developper know them they'll think directly in theses high level concepts and will be able to ignore the irelevant details.
What make Java more efficiant mostly is that you can forget to think about freeing memory and it is almost impossible to get a core dump. Instead of having to learn and speak the concepts of allocating and freeing memory your developpers will be able to concentrate on other things.
And what make C more efficiant than assembly is that you can actually write easily functions, that you can define structures and that the compiler take care to compute the proper shift in memory to access the differents field and so on.
But java, especially before version 8 is lacking key concepts like basic functionnal programming. So when a developper want to filter a collection, a pretty common operation, he will just iterate, check the condition of the filtering and return a filtered collection. It will be very specific and the level of expressivity of this code will be quite low.
A great developper would at leas isolate that code into a function with meaningfull name or define a generic filter function and pass predicate as anonymous functions. But the lack of proper lambda expression will prevent to fully leverage it.
But once everybody is accustomed to functionnal programming they'll just use filter and a predicate. The intent is much more clear, almost self documenting.
The more expressive you can be, the more common knowledge you can have, and the more you can concentrate on issue key to your program and the less you can spend time on unecessary details, the more productivity you get.
And yes, this start with a good programming language with a good echosystem and a team that efficiant with it.
That's why Java or C/C++ are so often used: they are the easiest languages when you need to hire because more people know them. This doesn't mean they are the best language for your problem at hand.
Here are some more good answers to that question in a Stackexchange thread http://programmers.stackexchange.com/questions/9320/why-is-l....
edit: another one - http://stackoverflow.com/questions/2036244/whats-so-great-ab...
Exactly. For instance, consider Clojure, PicoLisp, Racket and Wolfram in http://rosettacode.org/wiki/24_game/Solve.
Lisp was probably a good tool for pg's startup. It was an ecommerce builder, most lisps are great at metaprogramming and creating DSL's. But his startup also used C and some other languages for different problems.
What I think is great about Lisp is its idealistic simplicity.
(function parameter1 parameter2)
Everything is a list, pretty much everything that doesn't look like a list is just syntax sugar.
My opinion is: unless you have a hard requirement, just use the language you already know and like.
If you need to squeeze every tiny bit of performance, you might just need to use C++ or C.
If your client wants a complete website in a ridiculous short amount of time, going for python,ruby or even php would be a sane choice.
If your problem involves DSL's, metaprogramming or impressing people with how alien your code looks, check out some lisps, maybe Haskell.
You open your editor and connect to Lisp. Then you spend your time interactively defining functions, changing them, defining variables and inspecting them, and so on.
This has consequences that you need to experience or see to understand how it plays out in practice. You tend to start coding in a different way. Paul Graham calls it bottom up programming.
There are also synergistic effects with features like macros which enable radical kinds of convenient code generation / DSL-building, and conditional restarts which are like exceptions but fundamentally more powerful in a way that helps greatly when doing interactive coding, the multiple dispatch object system, the online documentation system, and so on and so on.
Lisp is shock full of good ideas that are still absent from most other languages. But mere bullet lists aren't convincing, like in so many other situations.
I wrote a web service in Lisp and just for convenience I wanted to have it incrementally recompile and reload its own source code on every request if the commit hash of the working tree had changed. This was a three line hack.
I also generate a memory image every time I make a deploy, which means that the server starts in a finger snap including the whole runtime for all this dynamic stuff.
Another advantage for me is the HyperSpec, an excellent hyperlinked reference of the whole standard language. It's an industrial quality manual written by literate professionals and so it's exhaustive, correct, and mostly unambiguous.
I also simply enjoy how Emacs with Lisp is a world-class IDE. Right now I'm mostly offline with just slow pricey 3G but I made sure to install a bunch of Lisp packages and the HyperSpec and it's a joy to code using just Emacs and SBCL in the background. I can look up any documentation, semantic tab completion and argument list popups work perfectly since the editor is aware of all the code and the namespace rules and so on.
I just don't know any other language that's so pleasant and fun to develop with. (After learning it, of course.)
People call Lispers smug because they love the language more than any other, but I don't feel smug because I didn't come up with the thing, I just learned it and like it. I'm humbled by the engineering and quality of the tools and everything. It's not some weird academic obsession. It just makes me happy on a daily basis.
Instead of being like yacc, which uses an external DSL to define the parser, with its own syntactic conventions, the Racket parset generator is built with macros. The reason yacc has to be a metaprogram instead of just a library is because of how the semantic actions are "chunks of code" which you can't really represent in C but in LISP its easy to represent "chunks of unevaluated code" in your program, which lets you define the parser generator as an embedded dsl implemented via macros.
Anyway, the reason I found this nice is that in Racket you can define your own macros to abstract over the repetitive parts of the grammar description. For example, in yacc you need to write multiple rules to describe a comma-separated list
and this pattern repeats for every other list you might have in the grammar. In yacc there is no alternative but to copy and paste this boilerplate but in Racket you could define a helper macro that would let you write grammars that look a bit like this:I have chosen a lisp (Clojure) because:
- Clojure allows me to write code that is robust, fast, and simple
- because it's simple, most Clojure code is easy to reason about, now, tomorrow and six months from now
- focus on the data (JSON, html parsing, whatever) and the transformation of it, which is arguably what we do most of the time in modern backend and frontend (React-based) applications
- pervasive and strongly encouraged immutability, managed concurrency, discouraged callback hell, etc which makes it hard to write bad code in the first place
- I can change functional+dynamic+immutable code faster than any other code, which allows faster experimentation and iteration on features
- it works very well for both backend services (Clojure/JVM), and browser applications (ClojureScript/Google Closure compiler), with the language staying 97% the same
It actually feels really closed-minded - like you've walked into a conceited professor's office and told him you have doubts about one of his ideas and he's looked down his glasses at you and talked about how most people prefer McDonalds to amuse-bouche at Pierre Gagnaire in Paris.
How about considering whether there are reasons other than 'it looks foreign so lesser programmers don't bother with it'? If you google around you find there are plenty of people who suggest practical reasons (too many fragmented implementations, hard to debug, etc.) as to why it's not caught on, and in fact a lisp like clojure seems like an attempt to learn lessons about what might work better or not. But no, we have these patronising proclamations and the article is over.
Don't get me wrong, I'm not anti-lisp per se, but I've come to think as I've seen more of the programming world (and worked with good developers as well as bad) that language has less of an impact on a project than I previously thought.
What I hate is the smug attitude a lot of lispers have, as if it's some incredible magical secret that only smart people can understand and if only we'd all see the light this world of shit code would turn upside down. It's hard to prove anything in the complicated world of programming, and I just don't buy this idea that it's magically better.
We should admit the subjective nature of these things, admit that it's really hard to tell what kind of a difference a language makes and instead of smug patronising talk of how wonderful language X is, point out what is unique about the language and stay humble.
I much prefer PG's more practical detail-orientated explorations of lisp and what makes it powerful in the first place to this one.
http://practicaltypography.com/why-racket-why-lisp.html
Here's what I think the article was linking to: http://web.archive.org/web/20051202060638/http://shootout.al...
This is also potentially relevant (the snapshot is noninteractive): http://web.archive.org/web/20070216224601/http://shootout.al...
* If you are so smart why aren't you so rich
* If your product is so great, why aren't you the market leader already
By combining a great team of competant people, a small startup environement were you can concentrate on getting thing done and by using powerfull tools they made their software, managed to have it evolving faster than their competitor and then sold it to Yahoo.
Ironically since the tool has been brought back by Yahoo its growth stopped and nobody know of it anymore. Is it because of the different people working on it? Is it because they had now a different management structured that cared more of politics? Or was it because they decided to recode everything in C++ ?
To me that's likely a combination of all the things.
That's the key point. A great programming language and echosystem is a great start. But that's not enough. You also need pragmatic management that make decision for the good of the company and not for politics and you need to have employees that care of their job.
At the end, you should know if you are in a model of small efficiant teams or huge average teams. Most company start with the former and end up in the later as they grow. Ultimately that why the next start up has a possibility to beat them. You would never be able to beat microsoft if everybody inside had a startuper mind-set. They would own the world.
That why people fear google so much. I'am pretty sure they have lot of politics and are already more in the later phase than the former but they at least managed to keep bright people working for them.
I found that Clojure (my first Lisp) was the easiest language to pick up that I've ever learned. Partly, that's because I've been doing this a long time, but I also think it speaks volumes to the simplicity of Lisps and Clojure in particular. I have no doubt that I could teach it to programmers and non-programmers alike rather quickly and I really wish it was at least studied, if not used, more so that people would have more experience with functional concepts. Even just a week or two of studying it is quite beneficial and a good contrast to typical OO languages.