HN Explorer

Ask HN: Why Isn't Functional Programming Taking Over?

47 points by elamje ↗ HN
I have recently been enlightened by LISP, specifically Clojure, and I'm not sure if I can go back to OOP.

After experiencing the joy of programming once again, I wonder why FP isn't widely adopted?

I understand in startupland that Django and Rails are simple to learn and prototype, but why aren't big companies and others using FP as a main language more often?

What are your thoughts?

83 comments

[ 2.8 ms ] story [ 144 ms ] thread
diehunde
Big companies are indeed using FP for some of their stuff. For example almost every big company uses Spark for data processing, and the best API for Spark is Scala. If you are talking specifically about Web apps, it's probably because MVC is almost ubiquitous.
draw_down
For me, it’s a useful tool when processing data, and I can appreciate some ideas like avoiding mutation, but as an overarching paradigm it doesn’t seem particularly helpful. But I guess I’m just a blub programmer
kwhitefoot
Most of the code that I write, whether C#, VB.Net, or even VBA, I generally write as much as possible in functional, single assignment style, with all information needed for a function passed as arguments even if the routine could have just grabbed the value from a global. So generally only the top most layer does any mutation. I think it saves me a lot of debugging because I don't have to worry much about side effects and I always know what a variable actually means. It also means that when I want to reorganize the code or extract a function to use it in another project that there is less work to do.
[dead] draw_down
Sure, that sounds like the code I write too. What I meant by FP as an overarching principle was stuff like wrapping side effects in monads and generally pretending your program is just a math routine that happens to include a few side effects. I don’t find that a useful way to organize programs.
gitgud
Functional programming is hard, it's hard to have pure functions which don't mutate anything in a system, there's always state somewhere.

Businesses also like to isolate and manage their state in certain areas. OOP more accurately reflects this.

Although functional programming is becoming popular when scaled-out, take for example lambda functions or docker containers, which generally don't use state and nicely suit the functional programming paradigm.

strangattractor
Because programming languages are practically irrelevant in making products. They are only important to developers. In the last thirty years of programming I have yet to see a project fail because of the programming language or code quality. More often than not it is because someone made something that no one wanted. From a reliability stand point Windows has always been a nightmare. It is still one of the best and most successful products of all time. It could be written in Urdu and no one would care.

What about developer productivity you might say? Yes a consideration - one of many. The last 20 years of language design has been focused on making people uninterested in programming programmers. That is what frameworks primarily accomplish. They waste memory and processor resources - are difficult to use in ways outside of their design. But any person can pick up rails for dummies and type gem something and presto a website. Believe it or not I think this is a good thing.

joatmon-snoo
To piggyback on this, if I have a codebase that has 10+ years worth of code in it that's been refactored multiple times, uses carefully benchmarked libraries, and has accumulated bug fixes for tricky edge cases (oh, and of course "bugs" that we can't fix because people depend on the buggy behavior), it's going to be very hard for me to justify rewriting the entire thing in a new language.

Putting on my Googler hat as well, there's a lot of work that's gone into our core libraries.

- Stubby, our RPC libraries, powers literally everything (Cloud, GSuite, Search, etc). I can trust that if I use it, I'm not going to be finding bugs in it - it's going to be some other big team that's got all sorts of fancy shmancy test that can and do catch strange, arcane bugs. The thought of relying on a brand spanking new implementation scares the shit out of me.

- We have a lot of powerful tooling (c.f. clang-mr, errorprone) that our core language teams use to constantly improve our codebase. That's years of tooling that a new language isn't going to have.

Taking the hat off, at the end of the day it's a tradeoff - either I spend my time continuing to work in a so-called boring language, or I learn a new one to use. If I choose the latter, I'm going to want it to be worth my time.

And frankly, sure, I don't get stuff like currying in Java, but Java 8's streams certainly do make life more bearable.

CuriouslyC
If you haven't seen projects fail due to code quality you're probably either on the green field end or writing well specified software.

The cost of adding unexpected features or dealing with unexpected changes is directly proportional to the quality of the code-base, and in many cases a high quality code-base can make things profitable that otherwise wouldn't be (and vice versa). If software that has the ultimate goal of saving money actually costs money, I would say that it has failed, even if you ultimately produce fully functional software.

democracy
I am yet to see a project that failed due to "code quality" - almost 20 years greenfield/support in corp/startup world. There are multiple ways for stakeholders on too many levels to not let it happen.
apta
> I have yet to see a project fail because of the programming language or code quality.

There was some kickstarter project a while back trying to make a game in golang and it failed because golang is not suited for that sort of stuff.

chromanoid
It's a fallacy to compare programming languages to spoken languages. And even if you do so, is it really irrelevant in what language a book is written in?
sergiotapia
Hard to hire, hard to shake the mental model, not mainstream enough so it's hard for it to become mainstream.

Some clojure teams are moving to elixir because elixir is functional, performant, but imperative-y enough to seduce imperative programmers.

adamnemecek
They are. Scala, Rust, js.
elamje
I’ve never heard of rust being used in the functional world, interesting.
SamReidHughes
Rust is arguably a pure functional language if you take out UnsafeCell and system calls.
millstone
No way. Rust supports mutation, does not have referential transparency, etc.
SamReidHughes
Rust functions perform mutation by consuming the parameter and emitting a new one in its place, effectively. You can take any data type and build an alternate version of it that can be used without using mut (and with a much less convenient API, but it’s basically a rote transformation of the code).

Edit: I’d nominate as a good definition of pure functional programming the absence of undeclared data flow (i.e. side effects) by your code.

[deleted]
(comment deleted)
millstone
Give me a break. I can increment a variable in Rust.

Yes you can perform an SSA-versioned-variables or linear-types-changing-the-world transformation. That's just sophism; we can conclude that C is pure functional via the same line of argument.

In response to your edit: Rust does not encode side effects in its type system and does not require system calls to achieve side effects. If that's your point then we're in agreement.

SamReidHughes
You can’t do that with C. Nor can you do that with Rust, because it has UnsafeCell. In the absence of UnsafeCell and system calls (and, I guess, unsafe blocks), Rust encodes “all” its side effects (conservatively, and not trivially conservatively). All of its information flow, anyway.
millstone
It's trivial in Rust to share mutable memory via RefCell, Rc, others. These are core, not exotic.

We can imagine a language in which these don't exist, but it's not Rust and would not enjoy the ergonomics of Rust. That's why Rust is not pure functional, or even "almost" pure functional.

SamReidHughes
RefCell uses UnsafeCell inside. Everything that does mutation, under shared ownership, has UnsafeCell somewhere -- IIRC this is or was a requirement to guarantee the compiler won't do certain optimizations.

> but it's not Rust

That's why I stipulated Rust without UnsafeCell.

srparish
Kotlin, java has been adding functional features
waste_monk
I didn't specialise in comp sci (my degree is in networking/security with a minor in software development) so the most we touched on is statistics and linear algebra.

The "mathematician mindset" needed to program in Haskell is alien to me. I like the idea of immutability etc. but there's just too much friction to get into it.

At least with Erlang it still feels like programming, Haskell just feels "wrong".

pdamoc
Give Elm a chance. It is less complex than Haskell while retaining a lot of the declarative nature. Since it is mainly involved in generating HTML, the declarative nature is a good fit.
Gibbon1
Was it John Carmack? Made a comment that he got into functional programming and got deeper and deeper into it and then realized there was nothing preventing him from using the same concepts in C++ but without the friction.
smt88
I disagree with the premise that it isn't.

Every major language, like Java, C#, and JavaScript, is adding common FP features. Immutability by default is becoming more common.

Even React is FP-inspired.

It's happening, just not all at once.

BenMorganIO
As a person who did OOP for for years, then FP for two and just coming back to OOP once in a while, I ask why I ever left. They're BOTH good. Both FP and OOP have a special place in my heart. I've had a positive experience with both of them. They are both ahead of the game in their own respective rights and both have shortcomings. You may experience something similar in the next few years when you touch Ruby after working on Clojure. Or you may not.
millstone
Because functional programming makes a ton of stuff harder for no benefit. Biggest example is loops.
rickycook
welllll, i definitely wouldn’t say for no benefit... recursion is a much more flexible construct than loops, and when you combine that with pattern matching you can do some really nifty things in a way that’s incredibly easy to reason about and add edge case handling to
millstone
Every mainstream language supports recursion. If it also supports loops, like most do, then loops are used more often, because they're more flexible (example: early-exit).
tome
Not every mainstream language supports tail call optimisation. Without TCO, rewriting loops as recursive functions is a bad idea.
feanaro
Functional doesn't imply loops are impossible or even hard. Just to name one possible implementation, if your language supports monads, loops with early returns are trivial.
millstone
Well put, this makes my argument in a much better forM_.
nimih
To the contrary: the reduced (heh) power of functional looping constructs is a benefit in and of itself. Explicitly naming and separating patterns of looping such as map, filter, fold/reduce, etc is in some sense a direct application of the rule of least power, and provides two distinct and immediate benefits:

First, the purpose and action of the code in question is immediately apparent to the (typically clueless, in my own case) reader. A for (or, heaven forbid, while) loop can do anything at all, but a map is quite explicitly transforming values of a collection (and/or turning them into a collection of side effects), whereas a filter's only job is to, well, filter, and a fold/reduce only needs to be conjured up when the programmer actually needs the full computing power of a generalized aggregation--quite the rare sight when good FP languages and libraries helpfully provide restricted (more descriptive and weaker!) versions such as sum and concatenate.

Second, using weaker functions means that the programmer (again, in my case, just an utter rube who needs all the help he can get) is saved from a breathtaking assortment of errors. Even in the chaotic anarchy of a dynamically typed language, a map will ensure you don't make off-by-one errors in your iteration, a pipeline of transformations will let you verify you did your processing in the correct order, and fold will at least give you the friendly encouragement to think hard about your initialization condition, and--more to the point--you won't even have the ability to carry forward state while iterating through a list unless you specifically request it.

The general point here, is that functional programming is trying to stop you from thinking about "looping" as a single abstract process, and instead have you explicitly reify the control flow of your program as higher-order function calls: this allows you to be explicit about what is supposed to be happening (to the benefit of both the reader and the writer, listed here in order of importance). Even better, in the presence of a Real Type System, this allows the compiler to actually tell you if your execution logic has the same general shape as your mental model, which is quite frankly a magical feeling once you see it actually happen when solving a difficult problem. That all being said, these benefits aren't free (as can be confirmed by anyone who has read a paper/tutorial on generalized recursion schemes...), so if you're able to write clear, maintainable, descriptive code while avoiding uninitialized states and incorrect boundary conditions and accidentally skipped elements and missing base cases and all the other horrible violence that I, personally and repeatedly, have inflicted on the poor algorithms I've clumsily attempted to implement, shine on you crazy diamond you.

unscaled
In some sense, it is, it's just happening every slowly.

Functional programming features are getting a foothold in mainstream languages. It's a drizzle, but the direction is clear.

Let's look at closures. Back in the 90s and early 2000s, if you were a mainstream developer using C, C++, Java or C# you were out of luck. But during the following decade all said languages (except C) added them, and nowadays closures are taken for granted in almost any new language being designed.

Functional list/stream processing APIs are the next feature being popularized, with such APIs being available in most mainstream languages nowadays (Go is a notable exception).

Other FP concepts like immutability, sum types, pattern matching and even type classes are finding their way into mainstream languages. Their adoption is far from universal but direction is clear. FP is becoming more widely adopted, just not necessarily FP languages.

externalreality
> In some sense, it is, it's just happening every slowly.

Yup, real world engineering doesn't heed purity of concept. What you get is a hodgepodge of best practices and ideas that work. Today we see JS, Golang, Ruby, modern C, C++, all of these things have ideas and features directly drawn from functional programming and likewise, many functional programming languages use ideas from other paradigms.

We likely won't see Lisp or Smalltalk become dominant anytime soon but there is no denying that many of their features have made it into many of the main stream languages that have become mainstream for reasons far detached from their programming paradigms (um maybe not Java, for better or worse, Java sold OOD and OOD sold Java).

apta
> Golang

> all of these things have ideas and features directly drawn from functional programming

golang is the complete opposite of that, they disregarded useful research with unsubstantiated claims and it's just a basic procedural language with some concurrency, that's it.

externalreality
You kind of forget the fact that the language has first class functions. That alone qualifies it as a functional programming language in every sense of the MF word. It also has interface and (like you mention CSP). Its more than a strictly procedural language. I'm afraid I am going to have to standby what I said.
apta
By that argument, C# is a functional programming language, so is Python, and maybe even C.
externalreality
Yes, they are. Its not the dominate paradigm and you wouldn't want to program them that way, but that's my point.
Rotareti
> In some sense, it is, it's just happening every slowly.

The web frontend world has become very functional as well. The react stack is full of FP, for example. React was initially written in SML, then transcribed to JS and recently to OCaml/ReasonML.

anon12413
Because being a purist and trying to solve all problems with a single paradigm isn't realistic nor good. The world is multi-paradigm. Why not use a language that gives you the flexibility to use functional concepts when appropriate and others while appropriate?
garyclarke27
SQL is mostly functional - it’s had some adoption. A Function is just a: m-1 or 1-1 Relation - with a guarantee that same input always gets the same output. It’s quite easy to setup Postgres to behave this way (it was originally designed as immutable with time travel) ie never delete anything, then use bi-temporal logic to retract records previously asserted as fact.
tannhaeuser
I'd say SQL is closer to logic programming with its use of tuple variables, and generally being about specifying what you want to select rather than how. You could also say it's closer to algebra (relational "algebra", that is).
dajonker
Even if everyone in a company wanted to switch to a different language, they would not be able to just switch overnight. There could be millions of lines of code running their business already, and rewriting all of that in another language would take quite an investment. Developers would need to spend a lot of time learning the new language, how to build it, debug it, fix issues in production, etc. A lot of (very) stable libraries such as database drivers would need to already be available or written for the new language. Now that seems like there might be a lot of costs involved. Especially opportunity cost, i.e. the time spent on switching to another language could be spent on building new features or fix existing issues. It would have to be a very significant improvement in productivity (i.e. you would 10x less developers) to make this worth the initial investment.
mrkeen
Because Java has 5 values for boolean. Haskell only has 2. Java is 250% better.
flukus
Because they don't solve real world problems as well. Rails and django show off themselves with "here's how to build a a website" while functional languages like to show off how terse they can make a Fibonacci sequence and all sorts of other academic crap that doesn't help $people do $job. For a language to be successful you have to show a real world problem that can be handled better than imperative languages and with them adopting the best features from functional languages that's getting harder and harder.
Gibbon1
I've thought one of the reasons procedural languages are popular is because of a lot of problems/tasks/specs are framed as procedural problems. And 95% of the time a procedural solution is 'fine'
0db532a0
The Fibonacci one-liners are just one example of the large amount of wank that exists in the functional world. However, there are quite a few banks, hedge funds and quant trading firms using functional programming now to model their financial instruments and contracts. Off the top of my head, I know for a fact that Standard Chartered and Barclays use Haskell to this end. Jane Street uses OCaml as its main language.

Functional programming lends itself to writing solid, verifiable code, and I trust these firms know what they’re doing given the amount of money involved.

Obviously you use the right tool for the job. For low-latency/HFT, it’s still probably going to be C/C++/ASM.

flukus
> Off the top of my head, I know for a fact that Standard Chartered and Barclays use Haskell to this end. Jane Street uses OCaml as its main language.

KDB is also fairly common at these sorts of companies and that was top of my mind when thinking of Fibonacci one liners. I've been through their youtube tutorials and various other bits of documentation and their are plenty of fibonacci like examples and precisely zero information on how to create a project, share code, deploy, etc. Smalltalk and other image based environments are much the same, they don't seem to have a lot of good answers for sharing code, or at least they don't show how. Another thing to keep in mind is what precisely they're used for at these companies, for some it's the core of the whole company for others it's just a fairly irelevent side project, I'd be willing to bet if you added up lines of code java would still come out on top by a mile. Our industry has a "you can prove anything with case studies" problem, see that one game built with lisp 20 years ago.

> Functional programming lends itself to writing solid, verifiable code,

I'm not sure that's true, it's a largely self selecting group that use these languages and I'm sure if you put them in front of an imperative compiler they'd be writing a lot of pure functions with little state. At worst imperative languages make it easier for state to creep in, but I consider that more of a feature in reality.

> and I trust these firms know what they’re doing given the amount of money involved.

I wouldn't be so sure, the industry has everything from old conservative companies still running COBOL mainframes to more cutting edge stuff, I know of one big progress user and even a large betting company building on nodejs. I personally worked on a 64-bit upgrade for a large company last year, finally they can run code on servers from this decade. And they're all held together by csv and tab delimited files flying in every direction, automated systems watching for FTP file drops, others scanning for email attachments and being run through a complex web of regexes. Not to mention the excel, if every copy of excel disapeared the industry would cease to function overnight. I've spent enough time in finance that keeping my money under a mattress seems somewhat prudent. There might be some good companies but there is no correlation between size/wealth and quality.

0db532a0
I completely agree that the publicly available tooling in the functional world is shit. Haskell doesn't even have a proper debugger, let alone IDE support.

KDB as you say is common with financial companies. In fact I have only heard about it being used by financial companies. I have only used J, another array based programming system, and I found it brilliant for processing data. KDB has a niche in the financial world, and there is a reason for it. First Derivatives relies on KDB.

Obviously anyone who is doing anything interesting and profitable in this area is not going to be making it public, just like the world of betting.

There's another problem in the functional world, which is that the vast majority of publicly available talks you can find involve 90% wanking over the type system or how functional can beat C in some edge case (inevitably coded C-style) rather than focussing on the real world applications of what functional is suited to.

You're right that there are a lot of people in the functional world who have no idea about imperative, and could probably not tell the difference between a MOV and an LEA. I wager that these are not the same types of people who have actual high-paying jobs writing functional code.

The companies I mentioned have their own tooling, even their own compilers. They're applying functional programming to the niche it's suited to, namely providing a well-typed, functional interface to the more nitty-gritty systems, which are necessarily made up of Excel, COBOL and FTP file drops.

You're just not going to hear about this in the typical type-system wank talk/blog on Youtube/Medium.

YeGoblynQueenne
But, Rails and django are frameworks, not languages. And the languages they're written in, Ruby and Python, are classic examples of modern languages with both imperative and functional features.
flukus
Python/django I'm not so sure, but ruby/rails illustrates the point well. Ruby rose and fell with rails. It existed in relative obscurity for a decade before rails came along and showed that world what you could do with it. Despite it's merits (I like ruby) without this killer app most programmers would probably have never heard of it. Functional languages don't really have that killer app, the only things I can think of that's close is emacs.
IshKebab
I definitely agree. They're find for pure functional problems (like making a compiler, or converting between document formats) but most real world problems like web servers, games, desktop applications, etc. involve a ton of state and IO, and they just make that really awkward.
prewett
From my vantage point as a unenlightened traditional developer, functional programming seems to be the opposite of Java. Java tried to genocide all verbs and make everything an object. It didn't work, we just got a bunch of verbose adjectives instead (Runnable.run, Comparable.compare). Functional programming, on the other hand, seems to ignore data: data is transient, just pass it around, but verbs live forever!

The problem is that computers fundamentally have both verbs and objects (data). And the real world has state. Managing state is a real problem when it is immutable and there is a lot of it. I'm guessing a functional programming network stack or image editor with undo would have a lot of performance-sapping copying going on.

So if you tend to have a lot of state, like UI, you probably want object-oriented programming. If you are munging through data once to get a result, maybe functional programming is better.

Or maybe your LISP enlightenment has been that you can run your data? I'm not FP expert, but it seems like that isn't the FP feature people people usually extol. I'm not even sure that FP languages besides LISP support it.

slifin
Clojure under the hood isn't copying on every operation it's partitioning the same data structure to give immutability to the developer

These data structures are called persistent data structures

prewett
I was thinking more along the lines of a copy being necessary when you update a game state, or something like that. Is Clojure able to do something like that with partitioning? I think someone had tried doing a game in Clojure and run into problems with copies, and posted his findings to HN a number of years ago.

Thanks for pointing out the use of persistent data structures, that is an interesting approach.

mrkeen
> seems to ignore data

Not even close.

YeGoblynQueenne
>> to genocide all verbs

Haha. See what you did there.

prewett
Ha, nice! I'll try to do it on purpose next time :)
Double_a_92
Because it's really hard to programm something without having a state... And than you have to shoehorn that somehow with FP terms.
Benjmhart
So I think, just reading this thread, there there are a lot of misconceptions of FP's capabilities. Things like statelessness - Haskell, Purescript, et al have ways of representing state, and for optimizing recursive behaviour into constant space/time (they even have great abstractions and data structures that can compete with C)

The number one reason is that ultimately it feels too different, and Functional programmers tend to be terrible educators. So it all feels very esoteric. To learn a pure functional language is almost to start learning programming from scratch.

The secondary reason is the package ecosystem. Generally speaking these languages have a rough go at dependancy mgmt/build tooling (stack vs cabal and the lack of good, simple guides for their usage, and the same for bower(deprecated) psc-package and spago. The problem is that once the tools improve - the docs and guides are all outdated. Good reliable information takes more digging in these languages.

BUT it is achievable. These are perfectly serviceable languages that can provide robust, provable behaviors. And in the case of purescript, a strongly typed environment on the web.

The development processes are out there and there are folks writing full stack FP. (myself included). But as a whole, pure FP needs a lot more educational material in order to grow.

richjdsmith
>To learn a pure functional language is almost to start learning programming from scratch.

I think this is exactly it. When I decided to try Elixir, I figured it'd be just a bit more effort than trying out any other OO language. It hasn't been. It's been a matter of reconsidering just about every concept I had to solving a problem with code.

That said, it has been awesome and I love it!

democracy
In a large company it is almost impossible to switch to a new language. You can try and convince to use some related language on some internal small project (say Scala in a Java shop). But the truth is the new language is not going to be a dramatic improvement in any area. The questions will be asked and there are no easy answers - who supports, maintains and updates it? Can we get commercial support, for how much and from who? Can we hire people? Will we able to hire in 5 years? In 10 years? Is it secure enough? How are the releases rolled out? Patches/fixes? Build processes? Code scanning? Maturity of libraries? More security questions. Can we scan the code in Fortify? Can we use AppDynamics on these apps? Do we understand its runtime well? You could probably get a few enthusiasts and build a team and get something started but the reality is it is very hard and going to take some time and the people don't tend to stick around for too long. And as a manger there is a chance you will find yourself in the middle of a large product with noone from the original team and no people excited enough to pick it up. This is why even rewrites are often a bad idea, rewrites in another language could be a business suicide for some. The truth is if we ever have a new ground-breaking programming paradigm we will not be asking the OP questions. I worked on a Scala project, it changed my thinking a bit and taught me some nice techniques, but no, I wasn't convinced enough to switch to it.
redcat7
FP programmers are above the nonFP in terms of skills. Mangers are non-functional programmers who understood they are shitty programmers and ran to do management instead. They never knew anything about FPs cause they didn't get there. Also mangers read marketing mumbo jumbo on the web - this is the cause why some languages are used in wrong industries etc. If you dont know the benefits or that something even exists - you wont use it. Most people on the planet dont know what FP can give them.
diminoten
They only skill functional programmers are better at is functional programming...
redcat7
No. What is sky for non functional programmers is the floor for functional. If you dont understand the difference then you still have a lot to learn.
slifin
My thoughts on this are that Clojure is a outlier even in the functional world

But it is where all the innovation is (subjective opinion) and it's features are easier to port to other languages like JavaScript and PHP

Transducers for example solve so many problems with higher order functions it's insane

Persistent data structures solve immutability performance concerns

Datomic is a time travelling universal immutable database that solves so many problems verse SQL

Clojure has graph based end to end stacks thanks to Datomic Fulcro Pathom EQL

All of these things are amazing, not found in other FP languages and relatively new to programming as a whole

My rant on this is, is now I feel like I wasted a lot of time (years) learning Haskell F# ReasonML etc where FP appears to be conflated with a more feature filled type system but not much else when imo that's not where most of the value is

dragonwriter
> I wonder why FP isn't widely adopted?

FP is fairly widely used, as a programming approach. Using FP, especially impure FP of the type most Lisps are optimized around, doesn't require an FP language (pretty much any structured or OO language with first class functions supports it, and functional idioms have been driving features in OO languages for many years.

> why aren't big companies and others using FP as a main language more often?

FP is not a language.

aryehof
* Frameworks

An area in which object-orientation is particularly well suited, unlike functional programming. They are very much the reason for object-orientation's continuing success/ubiquity. Programming against "platforms" especially requires them.

* Writing extensible/overridable libraries and code bases that others can use (and understand).

Again an area in which object-orientation excels over functional programming. Consider why something like Node uses object-orientation to facilitate that?