Ask HN: How did Python become the lingua franca of ML/AI?

There were some other DSLs that were being developed at the time but the ones that stuck were the Python ones. [1]

1: https://terralang.org/

ML/AI/Scientists aren't systems people. They don't want to care about memory management/parallelization/etc. - they want to write perfect little mathematical poems which get executed on a perfect Turing machine.

Python is good at that. Thanks to the efforts of actual systems people, its libraries (numpy, scipy, etc.) run quick enough to be practical on a lot of workloads.

Originally, it was only available in Scala/Java but then they added Python support courtesy of Py4J. And since Python was massively simpler than Scala it exploded in popularity very quickly becoming the default language.

So then you had Data Scientists who were already writing a lot of data transformations in Spark looking around at the rest of the Python ecosystem finding libraries like pandas, IDEs like Jupyter and basically staying there since it was so much easier than alternatives.

Their interests aren't really in computer science and so they look for whatever language can get them to an outcome as quickly and easily as possible. Even if it's not the most optimal, elegant or maintainable.

JavaScript, Ruby, and Perl either don't have this abstraction at all, or they have much weaker versions of it, and many fewer scientific libraries.

NumPy started in the early 2000's and continues to this day. It takes decades to build up this infrastructure! This recent interview with NumPy creator Travis Oliphant is great:

https://www.youtube.com/watch?v=gFEE3w7F0ww

He talks about how there were competing abstractions like "Numeric" and another library, and his goal with NumPy was to unify them. And how there are still some open design issues / regrets.

There were multiple people in the nascent Python community who were tired of MATLAB, not just because it's proprietary, but because it's a weak and inefficient language for anything other than its scientific use cases. You won't have a good time trying to write a web app wrapper in MATLAB, for example.

The much more recent Julia language is also inspired positively and negatively by MATLAB, and is very suitable for machine learning, though it doesn't have the decades of libraries that Python has.

-----

The NumPy extension was in turn enabled by operator overloading in Python (which is actually a very C++ influenced mechanism). JavaScript doesn't have operator overloading; I'm pretty sure Perl doesn't, but not sure about Ruby. Lua and Tcl do not have it. (Lua does have a machine learning framework though -- http://torch.ch/ -- but I think PyTorch is more popular now.)

So if Guido didn't design Python with operator overloading, then NumPy would not have grown out of it.

Also relevant is Guy Steele's famous talk Growing a Language (late 90's or early 2000's I think). He advocates for operator overloading in Java so end users can evolve language with their domain expertise! Well Java never got it, and Python ended up having the capabilities to grow linear algebra.

Guido has even said he doesn't really use or even "get" NumPy! So it turns out that an extensible design does have the benefits that Steele suggested (although it's a very difficult language design problem.) There have been several enhancements to Python driven by the NumPy community, like slicing syntax and semantics and the @ matrix multiplication operator. And I think many parts of the C API like buffers.

-----

Another interesting thing from Oliphant's interview is that he really liked that Python has complex numbers. (I don't think any of JavaScript, Ruby, Perl, or Lua have them in the core, which is important.) That piqued his interest and kicked off a few decades of hacking on Python.

He was an electrical engineering Ph.D. student and professor, and complex numbers are ubiquitous in that domain. Example:

    $ python3 -c 'print(3j * 2 + 1)'
    (1+6j)

    $ Python-3.9.4$ wc -l Objects/complexobject.c 
    1125 Objects/complexobject.c

Most ML people there cannot build large robust systems and some struggled with the non-algorithmic bits of software. I am sure that some can out there in the world, but for the most part our ML people were very good at creating models and not very good at the development part, especially as the program grew (part of the motivation to hire devs like me in the first place).

Python gets rid of as much of the developmental complexity as possible. No types, no memory management, libraries for everything, No need to create a class to run "hello world." Pip makes it trivial to import things. Use PyCharm and you just need to click the run button, with no complicated JRE and JDK setup.

It is the fastest way to start writing models.

I attended Strata conference in 2014 and after visiting various technology exhibition booths there, I saw a common theme: tech companies were building data solutions using Python as R was no good for the purpose

In a meeting scheduled to share my take-aways from the conference, I predicted "Python will emerge to be the language of DataScience in few years"

Arguably its biggest competitor then was R, but R is not well accepted by programmers. Yet another alternative is Matlab, but OMG, using matlab for anything string related is killing me.

While there is some history to it, Python won in the end isn't a surprise to anyone. It is simple but not toyish for real world system. I am working in one of the big techs, and Python is running the production workload for most AI services just fine.

I took a LOT of issue with dynamic typing, but for ML/AL you are going to write a lot ad-hoc data wrangling code, sometimes even Python feels verbose.

TL;DR: It had already won.

You say compared to other scripting languages'. Let's list them.

Ruby: no numeric support Go: unnecessary typing, modest numeric support, shitty generics Bash: ha ha ha Scala, java, c, cpp: not a scripting language, complex Tcl, php: out of favor Rust: hadn't happened yet R: in memory bias, not as simple Other languages were obscure or owned by monoliths (kotlin, swift, c#)

Python also has multiple implementations, a minor thing, but not really. Pypy keeps cython on its toes.

C# really could be a contender. I am more productive in c# than any other language except python (although I think I will be more productive in rust)

Python is, almost unarguably, the easiest language to code in, right now, period. It has the greatest expressiveness and the simplest syntax. I use it for large scale open source art projects, and you can use it for ai.

Why are you asking?

Many people of STEM fields without any programming background, had their first taste of programming with python. And it caught on.

Also, the real stuff is probably written in C/C++/CUDA/ASM. Its only the interface that is python (because of its inertial popularity)

Python's dynamic nature also made a lot of what's in NumPy and the various ML libraries possible or more convenient to use. The performance is not as much of an issue if you start thinking in NumPy terms, doing operations on whole arrays where the loops are then in C. Really, Python itself is just acting as orchestration for a bunch of C code that's doing all the work. In the case of something like Tensorflow or PyTorch, it's actually a bunch of CUDA code that's doing all the work and orchestrated by Python.

1) Python -- specifically CPython -- made it easy to wrap existing, thoroughly tested high performance libraries in Python APIs. So, you got easy access to things like GSL and BLAS and LAPACK, but you get to call numpy.linalg.svd instead of GESDD.

2) Python was a general purpose language, unlike R or MATLAB, so you could extend existing systems to do more without running into a wall.

3) Python was a heck of a lot less effort to use than C++.

It is used as a shell. It's merely an interface to some gigantic, highly optimized libraries (numpy, scipy, and later, Tensorflow, Pytorch, etc.), and it does a very decent job at being an interface.

- The language is easy to grasp, at least the part that is used in data science and ML;

- The syntax is "familiar", as compared with R;

- There are many more general purpose libraries in Python than in R;

- There's no memory management problems;

- The standard library is packed with batteries;

- No compiling, which is important for being a shell;

- It's better than bash etc. at dealing with non-text data, especially numerical values;

- The community was already writing extensions in C;

Some other language could work well, too, had someone written a numpy for it at the time. But there really aren't that many people who are capable, interested, and invested enough to write such a marvelous library.

For the curious, our primary hammer was “ROOT” https://root.cern - note its well-evolved ability to connect Python and C++ code.

The tl;dr: Python had the advantage of a mature legacy in supercomputing doing many of the same types of computations done in AI/ML. Those libraries and bindings provided a massive leg up versus other scripting languages that did not have this kind of capability effectively built-in.

Under the hood you get the benefits of CPP: stuff is dense in cache, operations are efficient.

But you can write it without a bunch of types, templates and allocators, which confuse people who aren't used to it. Most numeric code doesn't have a load of types anyway, it's just a few operations on some very large matrices.

Add to that the benefit that you can just ask of python's universe of libraries, which is quite large compared to rivals like MATLAB or R. Want to serve your model as a website? Jam it into Flask. Need crypto lib to grab the data? No problem, just pip it and import.

A lot of NumPy and SciPy ensued, and the rest is history.

There's an old saying that goes like "Python is the second best language for anything".

Python isn't the best for any kind of task; but you can do almost anything in any field with python and some libraries. It's reasonably easy for a non-programmer to use it.

I think my first experience with GPU programming was using CUDA with C (I think it was kind of customized C in mid-2000s), so Python is not there since forever.

But if you need to do a bit of web scraping/input data manipulation, a bit of "offering a gui" (e.g. a small web server that shows the data), a bit a of matrix/vectorized operations, a bit of model training or even just inference... python has everything and everything is reasonably good. At least some of those operations would be cumbersome in other programming languages.

Try using R for general-purpose programming. Or Java for number crunching/matrix operations. They just suck.

Try finding the "greatest common divisor", functionality-wise, for the many tasks that you need in a ML system (just as many other systems), and you'll find Python.

The drawback is, IMHO, that it doesn't "scale" well. Python makes great proof of concepts and prototypes, but I'll always pick a different stack (possibly with multiple languages and technologies) if I want a long-running, maintainable production system.

Wrapper generators and compiler tools like Cython and before that SWIG made it very easy to glue existing functionality to Python, so together with Pythons' great usability and user-friendly language it created a killer combination for productive data science & ML.

That said other languages could've pulled this off as well, Ruby for example. Python had more early traction in the scientific and high-performance computing communities though whereas Ruby was more popular in web development (due to Rails), which ultimately gave Python the edge and attracted more and more toolmakers to its ecosystem, which in turn spurred further growth. Great "IDEs" like the iPython/Juypter notebook were also a key factor in Pythons' success, as they provided a super user-friendly UI for data scientists.

Moreover, scientists are typically so-so programmers so not having to worry about complexities like dereferencing pointers, specifying types etc, makes the language much easier to pick up.

In some cases you might see a 3000 line python script with no defined functions… just loops and conditionals and lots of copy-pasted code with small variations in each section.

It’s really a shame, since there are so many more elegant languages which are equally or more powerful. But python is not a terrible language… it’s just an everyman get-shit-done language. We could be worse off.