I used Python for the last 10 years or so for all sorts of cross-platform command line scripting stuff, but watching Python3's progress I have the impression that this simple use case is no longer their main focus (I sometimes wonder if there's any focus or vision at all tbh). After having tinkered with Deno for the last 2 weeks or so I must say that this is indeed the 'better Python' (for me at least, and not because of Typescript - the language is more or less just an implementation detail for writing command line utilities - but because of Deno's approach to package management).
Python has been my favorite for products and scripting. I've been advocating for Python's dominance when the world was after Java. But now I'm more inclined towards JavaScript. And agree with you Deno is taking the right approach (as opposed to nodejs).
Having JavaScript in the front end and backed both can reduce resource requirements as well.
I learned Perl first and found it incredibly useful just because of regexps but Perl programs got messy as they got larger. Python regexps are just that bit less convenient and that trivial amount matters but in python I feel that once you get to writing functions and perhaps even classes it accelerates far ahead of Perl and you can write understandable, maintainable large programs in Python.
You're not forced to dip into the OO side of things though and I've heard people suggest to me that Python is a language where no rules apply and no structure exists. They feel that they can dive in without the care they would have to take in Java or C++. I think they are very wrong but you can certainly write terrible code if you want and perhaps this "all-things-to-all-people" aspect of it is part of the success.
Well said. The core to Python’s success, for me, has always been its system for containing code inside modules and for importing and exporting symbols between those modules.
Understandable code is maintainable code. Abstraction makes comprehension possible — imagine if every call to print were an inline block of assembly instead! — and Python’s modules provide a very quick and easy way to break up large code into small modules.
Eh, you massively overestimate the importance of performance.
For the vast majority of use cases, performance just isn't a priority. Doubly so for Python, that shines for simple automation, command line applications, and perhaps some serveless computing.
Being easy to write, having a good ecosystem of libraries, and being widely known is typically good enough. I wouldn't use Python to write a robust backend server side application, mostly because the language doesn't lend itself well for it.
Eh, you make incorrect assumptions about me. I'm stating a fact why Python is used - the data science ecosystem in Python thrives because of well-written libraries _written in C_ under the hood AND an easy-to-use language that writes like pseudocode.
If it was too slow, we'd be doing all of this in Java, the C# or maybe doing it in C/Fortran. But because of some early design decisions (Guido being on the matrix-sig helped), the history behind Numeric/Numarray and finally NumPy and SciPy being based on those efforts allowed it to thrive.
> it's the only way a tragically slow language like Python can keep up.
Those were your words, not mine. I need not make any assumptions.
I just replied listing use cases where Python shine due to its strengths, performance being mostly irrelevant. I didn't even mention data science.
And although it's beyond the point, if I was to use Python, why should I care in which language a library was written? If the language allows libraries written in other languages, this is actually a nice feature.
Depends what you are trying to teach. If you're trying to teach computer science and programming fundamentals, then sure. If you're trying to teach people how to get 'real work' done quickly and efficiently then Scheme will only get in the way and slow people down.
For example when I've taught programming, one of the tasks I taught fairly beginner programmers was to grab some satellite images between certain dates, try to detect if there is a forest fire, measure the spread of the fire and plot the spread on a map.
With python (and its excellent libraries) this is quite quick and easy, and most people are up and running and hacking around with their programs in pretty short order. They find it really cool and inspiring and makes them quickly realise that programming could be something useful in their day to day job. Trying to start with chapter 1 of SICP and Scheme and working up from there to solving the above problem would probably lead to most of these people giving up on programming very quickly. That being said the few people that made it all the way through that would no doubt be much much better programmers because of it.
This is an under-discussed point, at least from my biased opinion. It seems like the Python documentation ecosystem is the best, and has been building fantastic tooling for over 10 years now.
A third point is that python is the default scripting and plugin language in a lot of popular commercial and professional applications. A lot of people I know who learned python did so to automate or extend applications they used for their 'real' work, like ArcGIS, FME, Rhino/Grasshopper, Revit/Dynamo etc.
The classic "public static void main" example is also getting old. It's been so long since I last wrote it, that I actually had to copy/paste it from the article. I just generate a new project from the "Spring initializer" plugin and jump straight into the real work.
If we want to measure developer productivity we should try to compare actual real world usage that goes beyond "hello world" using Notepad. The author did provide more examples in the article, but I think we should just retire the basic minimal hello world example for these types of discussions.
If their argument is that Python is simpler for quick scripts and programs, I agree. The same applies to ML / AI where Python has lots of great tools. Once you start looking at other areas (Django and FastAPI), there are many alternatives based on other languages where you can be just as productive.
Most of the time consuming work is spent doing business logic where the differences between languages doesn't matter as much. They all have advantages and disadvantages. Personally I prefer static typing for larger projects and teams.
If their argument is that Python is better in general, they need to provide better arguments.
> - it’s believed to be beginner friendly compared to other languages. I’m not really sure why - maybe the whitespace?
Good bait. I'll take it.
- dynamic, weak (really "duck") typing, meaning users don't have to worry about conversion between things. Want to print() a dictionary of whatever? Sure!
- no semicolons to terminate a statement. End of line, that's it
- rich standard library, so you can actually get going on things without having to go on a quest to find the right library for your thing. JSON? It's right there. argument parsing? argparse. Http? http.client works...
- also yeah the "pseudocode" thing. Python is light on extraneous syntax.
Now eliti^H^H^H advanced programmers will frown on many of these same things that make it beginner-friendly...
Dynamic typing means that complex designs have to be careful with their APIs, lest you can get tangled up in deep type errors if you're not careful (I particularly hate the "mix-in" pattern). No semicolons mean, uh, long statements need to escaped? The standard library is "where lib go to die" because they can't evolve as much. "Significant whitespace!? What is this, COBOL!?!11"
I started in Python, as a Biologist (hooray for Jupyter-lab, coding so visually, in small steps, with output just there is so great when starting to learn Python). I ventured into other languages every now and then. For example I tried to make an Android app in Kotlin that gets info from some API. I expect something like this but with more brackets everywhere:
import request
data = request.get(https://some.api/get_some_json)
some_value_I_want = data['dig']['into']['nested_structure']
I didn't get it to work at all, I was lines and lines of code into the program when I didn't even get to see any returned values.
Also, say I want to make a nice plot of some tabular data (.tsv), I do:
import pandas as pd
import seaborn as sns
data = pd.read_csv('some_file.tsv', sep='\t')
pd.melt(data, value_vars=['s', 'max_vms'], id_vars=['sample', 'process', 'N'])
g = sns.FacetGrid(data=data, col='process', row='variable', hue='sample', sharey=False, margin_titles=True)
g.map(sns.barplot, 'sample', 'value', order=data['sample'].unique())
Boom, what a plot (or large grid of plots actually), so much information. Can anyone show me how to do this in some other language (but R)? Idk, maybe I'm just not so smart, I just learned to program at 35 after always being a biologist, but any venture into any language has me thinking: Why does this have to be so complicated?
(Btw, I'm putting 4 spaces in front of the code, why is it not rendered as code?)
When I just started I used Python to read, sort and transform images and output them to a PowerPoint file. One can use CSS like synthax to format the slides. Boom PPT with 120 slides with images and data from some Excel file that convinced a lot of people with a lot of data that fluorescent images next to H&E stains + metadata can be nice. Is it the best way to present such data? Meh. But man was it cool and easy and time saving.
Ok, then I'm not programming. Call it anything you want, I call it "Being productive." Or, "Saving on time spent clicking around in Excel." or "Automating the boring things." And I find it to be quite pleasant.
Maybe it also doesn't help that out there, in the C++, Rust, Javascript, Go world I'm going to run into people with usernames like "ihatepython".
Sometimes I wish for "block" function in HN for such trolls (I really hope it's no-life troll working for 5 rubles per comment and not a real person with genuine hateful opinions like that).
It is however what a lot of people who 'program' at work do every day. Python actually lets working professionals solve real problems they actually have at work quicker and easier than any other programming language.
Your examples don't depend on the language but on the libraries. You can be as simple and concise in most languages, provided the libraries let you, instead of needing to add boiler plate.
But what if your web request fails? Do you deal with it in time or use some_value_I_want which might contain junk?
This is just a property of having libraries. If you've got the same libraries in Java then the code looks identical except that you stick to word "var" in front of lines 4 and 6 and you don't have named arguments.
Python does have a great data processing ecosystem. But that isn't really a property of the language.
Creating a new venv, installing a few needed libraries that you know the names of with a simple command and writing a quick low-ceremony script that uses those libraries is frictionless in Python. Doing that repeatedly in nearly every other major language is not as easy.
Some languages have a good integrated tool chain (Rust, Go) but are not as approachable and forgiving.
Some are approachable but lack the friction-free story for locating and installing 3rd party dependencies.
The most similar language to Python is Ruby, not Rust or Go, and Ruby is better at the things you've listed. Which would bring us back to the previous point that it's not actually about ease of use.
Someone else identified Ruby's fatal flaw as not having good C tooling, and I think that's probably accurate.
Ruby has odd, once unique syntax that looks strange to anyone raised on C-like syntax, including js and java. Python is similar, simply removing redundant braces. So no ruby was not better at things most folks care about, i.e. being easy to learn.
Things that are familiar are easier to learn, it’s a fact. As someone raised on math and English notation, then industry exposure. Ruby being a odd duck did it no favors.
Python definitely found footing based on its pseudocode readability. It’s gotten a bit worse recently with too many colons and features but already made it.
1. Near zero boilerplate. Python's boilerplate is usually no more than setting up a class and then calling a method. Often you can skip the class and just call it directly. This is probably the biggest strength; to give you an example, my standard test for a languages approachability is "how much work do I need to put in to get a very simple JSON file from a web URL" (nothing fancy like POST, just an HTTP GET). With python, a call to urllib.request.urlretrieve and then a call json.loads are all you need. In Java, you need to manually implement a bunch of boilerplate code that looks horrible, need to think about the size of your response in memory and often need to pass in configuration options that should be a case of "works by default" but aren't because the standard is ages old so it needs to be manually toggled on. Part of that is that the Java stdlib consists largely of reference implementations rather than actual implementations, which means that anyone who wants to implement something in Java will usually end up falling back to the stdlib interfaces, which in turn suffer from being stdlib interfaces, so a lot of "should really be on by default" expectations aren't on by default since the stdlib is where code goes to die, so you instead start piling on features.
2. Interop with C. I hold the opinion that C is a great language that doesn't work very well once you get to any form of scaling. Python allowing developers to take the slowest parts of their code and writing it in C to speed it up is one of the easiest speed gains to make and it avoids the biggest bumps that come with using C as your primary language in terms of project structuring.
3. Library support is as you say, good. If you need it, there's probably a package for it. Pypi is dependency-wise kind of a disaster but if you know how to set up requirements.txt, it works really well. Most libraries ship with "sane" defaults too.
All of these combine to a language that's easy to prototype, easy to expand
It's important to keep in mind that pythons biggest successes aren't in the speed-focused, low-memory environments where every speedgain is necessary. Its success lies in conventional desktops and servers, which have much more processing power and often have more leniency in being a bit slower.
With python you can write something in 3 hours what would take a day in another language, at the cost that instead of being lightning fast and done in 10 seconds, you need to wait 30 seconds. That's an issue for some environments but in 99% of the cases that's not a problem.
That isn't to say the language is perfect (no language is), but speed of development at the cost of slightly slower execution time is the main reason why python got popular.
>"how much work do I need to put in to get a very simple JSON file from a web URL" (nothing fancy like POST, just an HTTP GET). With python, a call to urllib.request.urlretrieve and then a call json.loads are all you need.
In C# you just have to do this:
var things = await httpClient.GetFromJsonAsync<List<Thing>>("url");
Mostly because JSON is one of the most common formats used when sending over data. I think this practice started with pythons requests (which is my real answer as to what you should use in python but I wanted to focus on the stdlib), which has a json function on the Response object for convenience.
Most languages nowadays tend to implement some variation of this specific convenience because it's just one of the most frequently needed things; setting up a separate parser and then calling it might be the "cleaner" option, but it's also more boilerplate and the industry has largely moved to try and avoid that.
It doesn't. It's a standalone (static) helper method that uses HttpClient to perform the request and feeds the response body into a json parser. It's an extension method [0] which means that there's syntactic sugar so that you can write client.GetFromJsonAsync() and the compiler transforms it into the actual static method call, HttpClientJsonExtensions.GetFromJsonAsync(client).
I agree comparing hello world code in Python and Java is a bit pointless. Hello worlds might be much shorter, but at scale, this becomes less relevant. Also, comparing to a language that hates change is unfair, if you compare it to C#, which is improving over time, you'll see the hello world also takes one line (but is 13 characters longer, so Python still wins!)
> if you compare it to C#, which is improving over time, you'll see the hello world also takes one line
Well yes it it does now, since C# 10 or, but in Python is have been a one liner since the beginning 30 years ago, and over the decades it has build a following. And even with recent efforts to cut down on boilerplate, C# still has more cryptic syntax than Python.
I'd argue C# is more maintainable in the long run due to the static typing, but there is no way it is as accessible to beginners.
> it’s believed to be beginner friendly compared to other languages. I’m not really sure why - maybe the whitespace?
Experienced programmers tend to forget how the experience was as a beginner programmer. But Python grew out of actual usability research into teaching programming to beginners. Logo is another language with the same background, but it never escaped that niche. The success of Python is because it attractive to beginners but remains a powerful tool as the programmer becomes more experienced.
Whitespace is definitely a factor - or rather the lack of redundant braces. Beginner programmer seem to really struggle with the lack of correspondence between the visual structure and the logical structure in most languages. Even experienced programmers get tripped up by erroneous indents. Python just solves this once and for all.
The lack of type annotations is also a factor. For beginners this is just an additional layer of complexity.
Scheme is great for teaching computer science students, but if you are just a regular Joe researcher wanting to get the job done, you want to write "2 + 2" like everybody else in the world, not "(+ 2 2)"
Most languages are designed to attract experienced programmers, e.g. by having C-like syntax which they may already be familiar with. But it has a cost for beginners.
Eh, they're just a lot of ways to say "path dependence". Scripting languages are basically the same exact technology with respect to each other. In the alternate universe where numpy and scipy are, let's say, numruby and sciruby, wouldn't we be here asking why Ruby keeps growing?
That's not a sales pitch for python, it's a sales pitch for the concept of a scripting language; it's like saying "you should really buy a Ford, it comes with four wheels".
I admit I have a blind spot for Python, because I use PHP in my day job, so when I need to do some scripting, I mostly use PHP. But admittedly Python is a lot friendlier than some alternatives (Perl, Shell scripts etc.), and more universal than others (PHP being mostly used for web dev), so that's why people choosing a scripting language for their tool/library tend to choose Python.
I use Python all the time both for my personal stuff and for some side-projects at work, so this isn't a dunk on Python, but honestly it feels like a circular thing: it's popular because it's popular.
I wouldn't say it's friendlier than the alternatives, Perl and Shell scripts sure, but not when compared to Javascript, Ruby or Lua.
Now, if you're talking about libraries, support, etc. then sure, Python wins hands down, but that doesn't make it a better language in itself. I'd say Ruby and Lua are a little bit better as languages.
But then again, I don't care much for the language in itself, so Python is enough for most of my use cases.
I read "it's popular because it's friendlier". PHP, Javascript or R are popular, but are not friendlier. I find their error messages way worse for the beginner, when you need it more. Third party code is too "clever" for the beginner to read and learn, because it seems to be two languages: the one you are learning in the tutorials, and the other idiom that is used in the serious libraries. As a beginner you are hit with this feeling that you are far, far away from writting an useful thing.
In my job I've seen some beginners starting with R, and quickly hating it because they don't feel they can do much on their own, but copy-pasting and then modifying from the examples and the tutorials. And it the changes go too far, everything collapses with cryptic errors. When you show them Python as an alternative, pointing that they shouldn't use it over R for statistics and graphics, they like that they can build ideas from the scratch. That beginner is hooked for life.
I think Lua was always seen as a bit obscure, and not enough people invested in the language to write useful utilities. It has a solid C foreign function interface, and the compiler is quite fast, which leaves me puzzled about why it never gained traction. I think it's an embedded scripting language in the majority of use cases (e.g. NeoVim, LuaLaTeX, scripting in some game engines).
The story of Ruby is altogether different: they made the fatal mistake of not defining a C foreign function interface in the standard, otherwise I imagine we'd be seeing numerical computation and ML libraries with a Ruby interface today. Still, Ruby lives on in Metasploit, and in Sorbet and Crystal.
> The story of Ruby is altogether different: they made the fatal mistake of not defining a C foreign function interface in the standard, otherwise I imagine we'd be seeing numerical computation and ML libraries with a Ruby interface today.
> I think Lua was always seen as a bit obscure, and not enough people invested in the language to write useful utilities. It has a solid C foreign function interface, and the compiler is quite fast, which leaves me puzzled about why it never gained traction. I think it's an embedded scripting language in the majority of use cases (e.g. NeoVim, LuaLaTeX, scripting in some game engines).
- Lua's standard library is so weak that it makes most other batteries-not-included languages look like they have large, robust, and helpful standard libraries.
- It's got a bit of the quirkiness and gotcha-ability of JavaScript but without its being a language that's impossible to avoid due to capture of a mega-popular platform, which is what propelled JavaScript to ubiquity despite its being kinda shit and unpleasant to work with.
- Tooling's not as good as many other languages.
(FWIW sometimes I write Lua regardless, because it's the right tool for the job)
Luck of libraries and initial userbase are certainly involved in success, but not all scripting languages are equal. I mean we could add bash to that list then.
In fact I'd argue that python enjoying the success it has, despite probably the worst handling of a version bump in any language (2->3), is a testament to its popularity.
A lot of Java's verbosity isn't so much from the language, but from when the code was written. It happened to come to popularity at a time when big over-engineered Gang of Four-style design was hot. So you get a lot of code written in this over-engineered fashion where half the classes have names that end in DelegateFactoryFacadeMessengerImpl.
Some of it is the language too, but modern Java can definitely be reasonably terse. Record classes has done a lot for the language, so has lambdas and streams.
Java is still good for several use cases, which were popular at the time, or let's say much more code was needed. Now in the Enterprise world you have much better no- or low-code tools. But in large teams all of these public/private/protected key words are quite nice. Totally unnecessary for a small codebase. But Java with its great runtime, verbose patterns is great for web server applications in large teams. I would still use it as the foundation for a tech company, but not for the Fortune500 company that needs some customization. Where is no alternative where you can find people, and has the stable properties. Maybe Rust if you have the clout as a company. Otherwise it's too expensive.
Protected data was never as useful to me as package-only data. The former is only relevant with inheritance hierarchies, whereas the latter is more open while also restricting access only to other classes in your com.example.concern package. Extremely useful for keeping big teams on the right track.
For its warts, Java was supported by a company with a lot of smart engineers who were working to provide a “batteries-included”, cross-platform language ecosystem and Sun was really invested in its success. I’d love to see some of these successor languages get the same kind of corporate sponsorship, but I don’t think it’s profitable.
> I would argue that there is nothing simple about the last if-statement and will probably be very confusing for new developers.
The only thing that isn’t perfectly straightforward is “what is __name__”, but once you know how __name__ is defined…
OTOH, that example is much more complicated than needed.
As a simple executable script, that would do the same thing if antigravity. Fly() actually did anything, all that is needed is:
import antigravity
antigravity.fly()
The rest is unnecessary boilerplate to provide a module that can operate either as a library or a script, which is superfluous here. Moreover, since the actual functionality it is demonstrating is all in an import hook, all you actually need (the rest just produces an error message) is:
* Isn’t testable because it runs the sys.exit() call unconditionally on import, so there is no way to import the module and call main() from test code.
* Fails with an error, because main has a required argument that it isn’t supplied in the unconditional main() call inside the sys.exit() call.
* Isn’t following a typical python idiom by returning a value from a function and have it be unconditionally 0; if the only options are a fixed return and a exception somewhere, the convention is to return None (which doesn’t require an explicit return.)
* Since a normal exit is the default response of ending the main program, sys.exit() is unnecessary here; “Exit with 0 if main() returns successfully, and with an abnormal exit on exceptions in main” is achieved by just calling main() and having nothing after it in the main program.
Optimistic that we'll see maybe a 2x performance over the next 5 years, yea. Optimistic that python will catch up to any of the 'fast' scripting languages, no.
Python keeps growing in number of users because it’s easy to get started, has libraries to load basically any data, and to perform any task. It’s frequently the second best language but it’s the second best language for anything.
By the time a python programmer has «graduated» to learning a second language, exponential growth has created a bunch of new python programmers, most of which don’t consider themselves programmers.
There are more non-programmers in this world, and they don’t care - or know about - concurrency, memory efficiency, L2 cache misses due to pointer chasing. These people all use python. This seems to be a perspective missing from most hackernews discussions, where people work on high performance Big corp big data web scale systems.
There's also the argument that at a certain scale the time of a developer is simply more expensive than time on a server.
If I write something in C++ that does a task in 1 second and it takes me 2 days to write, and I write the same thing in Python that takes 2 seconds but I can write it in 1 day, the 1 day of extra dev time might just pay for throwing a more high performance server against it and calling it a day. And then I don't even take the fact that a lot of applications are mostly waiting for database queries into consideration, nor maintainability of the code and the fact that high performance servers get cheaper over time.
If you work at some big corp where this would mean thousands of high performance servers that's simply not worth it, but in small/medium sized companies it usually is.
Realistically something that takes 1 second in C++ will take 10 seconds (if you write efficient python and lean heavily on fast libraries) to 10 minutes in python. But the rest of your point stands
Damn! Is the rule of thumb really a 10x performance hit between Python/C++? I don’t doubt you’re correct, I’m just thinking of all the unnecessary cycles I put my poor CPU through.
Outside cases where Python is used as a thin wrapper around some C library (simple networking code, numpy, etc) 10x is frankly quite conservative. Depending on the problem space and how aggressively you optimize, it's easily multiple orders of magnitude.
FFI into lean C isn't some perf panacea either, beyond the overhead you're also depriving yourself of interprocedural optimization and other Good Things from the native space.
It really depends on what you're doing, but I don't think it is generally accurate.
What slows Python down is generally the "everything is an object" attitude of the interpreter. I.e. you call a function, the interpreter has to first create an object of the thing you're calling.
In C++, due to zero-cost abstractions, this usually just boils down to a CALL instruction preceded by a bunch of PUSH instructions in assembly, based on the number of parameters (and call convention). This is of course a lot faster than running through the abstractions of creating some Python object.
> What slows Python down is generally the "everything is an object" attitude of the interpreter
Nah, it’s the interpreter itself. Due to it not having JIT compilation there is a very high ceiling it can not even in theory surpass (as opposed to things like pypy, or graal python).
I don't think this is true: Other Python runtimes and compilers (e.g. Nuitka) won't magically speed up your code to the level of C++.
Python is primarily slowed down because of the fact that each attribute and method access results in multiple CALL instructions since it's dictionaries and magic methods all the way down.
Which can be inlined/speculated away easily. It won’t be as fast as well-optimized C++ (mostly due to memory layout), but there is no reason why it couldn’t get arbitrarily close to that.
How so? Python is dynamically typed after all and even type annotations are merely bolted on – they don't tell you anything about the "actual" type of an object, they merely restrict your view on that object (i.e. what operations you can do on the variable without causing a type error). For instance, if you add additional properties to an object of type A via monkey-patching, you can still pass it around as object of type A.
A function/part of code is performed say a thousand times, the runtime collects statistics that object ‘a’ was always an integer, so it might be worthwhile to compile this code block to native code with a guard on whether ‘a’ really is an integer (that’s very cheap). The speedup comes from not doing interpretation, but taking the common case and making it natively fast and in the slow branch the complex case of “+ operator has been redefined” for example can be handled simply by the interpreter. Python is not more dynamic than Javascript (hell, python is strongly typed even), which hovers around the impressive 2x native performance mark.
Also, if you are interested, “shapes” are the primitives of both Javascript and python jit compilers instead of regular types.
> it's a VM reading and parsing your code as a string at runtime.
Commonly it creates the .pyc files, so it doesn't really re-parse your code as a string every time. But it does check the file's dates to make sure that the .pyc file is up to date.
On debian (and I guess most distributions) the .pyc files get created when you install the package, because generally they go in /usr and that's only writeable by root.
It does include the full parser in the runtime, but I'd expect most code to not be re-parsed entirely at every start.
The import thing is really slow anyway. People writing command lines have to defer imports to avoid huge startup times to load libraries that are perhaps needed just by some functions that might not even be used in that particular run.
Of course it depends on what you are doing, but 10x is a pretty good case. I recently re-wrote a C++ tool in python and even though all the data parsing and computing was done by python libraries that wrap high performance C libraries, the program was still 6 or 7 times slower than C++. Had I written the python version in pure python (no numpy, no third party C libraries) it would no doubt have been 1000x slower.
Last time I checked (which was a few years ago), the performance gain of porting a non-trivial calculation-heavy piece of code from Python to OCaml was actually 25x. I believe that performance of Python has improved quite a lot since then (as has OCaml's), but I doubt it's sufficient to erase this difference.
And OCaml (which offers a productivity comparable to Python) is sensibly slower than Rust or C++.
It depends on what you're doing. If you load some data, process it with some Numpy routines (where speed-critical parts are implemented in C) and save a result, you can probably be almost as fast as C++... however if you write your algorithm fully in Python, you might have much worse results than being 10x slower. See for example: https://shvbsle.in/computers-are-fast-but-you-dont-know-it-p... (here they have ~4x speedup from good Python to unoptimized C++, and ~1000x from heavy Python to optimized one...)
And how much code is generally written that actually is compute heavy? All the code I've ever written in my job is putting and retrieving data in databases and doing some basic calculations or decisions based on it.
Code is "compute heavy" (could equally be memory heavy or IOPs heavy) if it's deployed into many servers or "the cloud" and many instances of it are running serving a lot of requests to a lot of users.
Then the finance people start to notice how much you are paying for those servers and suddenly serving the same number of users with less hardware becomes very significant for the company's bottom line.
The other big one is reducing notable latency for users of your software.
That is true, but there are relatively few real world applications that consist of only those operations. In the example I mentioned below, there where actually some parts of my python rewrite that ended up faster than the original C++ code, but once everything was strung together into a complete application those parts where swamped by the slow parts.
Most of the time these are arithmetic tight loops that require optimisations, and it's easy to extract those into separate compiled cython modules without losing overal cohesion within the same Python ecosystem.
I spend most of my time waiting on IO, something like C++ isn't going to improve my performance much. If C++ takes 1ms to transform data and my Python code takes 10ms, it's not much of a win for me when I'm waiting 100ms for IO.
With Python I can write and test on a Mac or Windows and easily deploy on Linux. I can iterate quickly and if I really need "performance" I can throw bigger or more VPSes at the problem with little extra cognitive load.
I do not have anywhere near the same flexibility and low cognitive load with C++. The better performance is nice but for almost everything I do day to day completely unnecessary and not worth the effort. My case isn't all cases, C++ (or whatever compiled language you pick) will be a win for some people but not for me.
Or there are programmers who write both. Something that I want to write once, have run on several different platforms, handle multi-threading nicely, and never have to think about again? Rust. Writing something to read in some data to unblock an ML engineer or make plots for management? Definitely not Rust, probably python. Then you can also churn out things at 10x the speed, but by writing the tricky parts in something other than python, you don't get dragged back down by old projects rearing their ugly heads, so you outpace the python-only colleagues in the long-term.
Programming is secondary to my primary duties and only a means for me to get other things done. I'm in constant tension between using Python and Rust.
With Python I can get things up and going very quickly with little boilerplate, but I find that I'm often stumbling on edge cases that I have to debug after the fact and that these instances necessarily happen exactly when I'm focused on another task. I also find that packaging for other users is a major headache.
With Rust, the development time is much higher for me, but I appreciate being able to use the type-system to enforce business logic and therefore find that I rarely have to return to debug some issue once I have it going.
It's a tough trade-off for me, because I appreciate the velocity of Python, but Rust likely saves me more time overall.
> coworkers who churn out shiny new things at 10x the speed
Sounds like a classic web-dev perspective, my customers hate when we ship broken tools because it ruins their work, new feature velocity be dammned. We love our borrow checker because initially you run at 0.5x velocity but post-25kSLOC you get to run at 2x velocity, which continues to mystify managers worldwide.
With Python, testing, good hygiene and a bit of luck you can write core that is maybe 99% reliable. It is very, very hard to get to (100-eps)% for eps < 0.1% or so. Rust seems better suited to that.
Anything else, especially if there isn't a huge premium on speed, meh - Python is almost always sufficient, and not in the way.
I use the same combo: lots of Python to analyse problems, test algos, process data, etc. Then, once I settle on a solution but still need more performance (outside GPU's), I go to rust.
I'm simulating an audio speaker in real time. So I do the data crunching, model fitting, etc. in python and this gives me a godd theoretical model of the speaker. But to be able to make a simulation in realtime, I need lots of speed so rust makes sense there (moreover, the code I have to plug that in is rust too, so one more reason :-)). (now tbh, my realtime needs are not super hard, so I can avoid a DSP and a real time OS :-) )
I don't need rust specifically. It's just that its memory and thread management really help me to continue what I do in python: focusing on my core business instead of technical stuff.
At some point, every engineer has heard this same argument but in favor of all kinds of dubious things such as emailing zip files of source code, not having tests, not having a build system, not doing IaC, not using the type system, etc.
I'm sure Rust was the wrong tool for the job in your case but I find this type of get shit done argument unpersuasive in general. It overestimates the value of short-term delivery and underestimates how quickly an investment in doing things "the right way" pays off.
The business owner (whoever writes the checks) prefers get shit done over "the right way". Time to completion is a key factor of the payoff function of the devs work.
The entire point of doing things the right way is that you end up delivering more value in the long term, and "long term" can be as soon as weeks or even days in some cases.
Business owners definitely prefer less bugs, less customer complaints, less support burden, less outages, less headaches. Corner cutting doesn't make economic sense for most businesses and good engineering leadership doesn't have much trouble communicating this up the chain. The only environment where I've seen corner cutting make business sense is turd polishing agencies whose business model involves dumping their mistakes on their clients and running away so the next guy can take the blame.
Try the travel/event booking business (where I'm in) - and no, people don't dump their mistakes on the next guy here - to the contrary, the "hacky" Python solutions are supported for years and teams stay for decades (allthough a decade ago we had not discovered how great Python was)
What business owners actually don't like at all is how long is takes traditional software development to actually solve problems - which then don't really fit the business after wasting a few years of ressources... and the dumping and running away is worse in Java and other compiled software. With Python you can at least read the source in production if the team ran away...
> the dumping and running away is worse in Java and other compiled software. With Python you can at least read the source in production if the team ran away...
Java (and dotnet, the two big "VM" languages) is somewhat of a strange example for that; JVM bytecode is surprisingly stable and reverse engineering is reasonably easy unless the code was purposely obfuscated - a bad sign on any language anyways.
The thing is that the short term is much easier to predict what you're going to need and where the value is, and in the long term you might not even work on this codebase anymore. Lot of incentives to get things done in the short term.
If you're dealing in areas with short time limits then Python is great,
because you can't sell a ticket for a ship that has sailed.
And I've seen "the right way" which, again, depending on the business may
result in a well designed product that is not what's actually needed (because
people are really bad at defining what they want)
What's brilliant with Python compared to other hacky solutions that it
does support test, type hints, version control and other things. It just
doesn't force you to work that way. But if you want to write stable, maintainable
code, you can do it.
That means you can write your code without types and add them later.
Or add tests later once your prototype was been accepted. Or whenever something
goes wrong in production, fix it and then write a test against that.
Oh and I totally agree you should certainly try to "do things the right way",
if the business allows it.
It is hard to believe that Python is objectively that much more productive than other languages. I know Python moderately well (with much more real world experience in C#). I like Python very much but I don't think it is significantly more productive than C#.
This. C#, Java or even newcomers such as Kotlin/Go are even in the same ballpark due to the REPL/Jupyter alone. Let alone when you consider the ecosystem
If you are in a lab (natural science lab) or anywhere close to data, I bet you it is much more productive, even more so when you have to factor in that the code might be exposed to non-technical individuals.
> underestimates how quickly an investment in doing things "the right way" pays off.
What time horizon should a startup optimize delivery for? Minutes, hours, days, weeks? Say you're a startup dev in a maximalist "get shit done now" mindset so you're skipping types, tests, any forethought or planning so you can get the feature of the week done as fast as possible. This makes you faster for one week but slower the week after, and the week after, and the week after that.
Say a seed stage startup aims for 12 months runway to achieve some key outcomes. That's still a marathon. It still doesn't make sense to sprint the first 200 meters.
My most successful career epiphany was realizing that everyone -- my customers, my boss, etc -- was happier if I shipped code when I thought it was 80% ready. That long tail from 80-100% generates a lot of frustration.
It's just an application of the Pareto principle. That last 20% of work to make perfect software costs a lot of time. Customers (and by extension, management) do not care how pretty your code is, how perfect your test coverage is (unless your manager is a former developer, then they might have more of an opinion), they care most that you ship it. Bugs are a minor irritation compared to sitting around waiting for functionality they need, as long as you're responsive in fixing the bugs that do come up.
Thanks. I thought that is what you meant but another possible take was that the last 20% is actually important. Getting something 80% finished is fast and then the long tail to get it to 100% is frustrating for everyone because the work, in theory is finished. I think that can happen as well.
Of course there are at least three dimensions to discuss here: internal quality, external quality and product/feature fit. Lower quality internal code eventually leads to slower future development and higher turnover as no one wants to work with the crappy code base. Lower external quality (i.e. bugs) can lead to customers not liking your product. Interestingly the relationship between internal and external quality is not as direct as one might think. Getting features out the door more quickly (at the expense of other things) can help with product fit. Essentially, like most things, this is an ongoing optimization problem and different approaches are appropriate for different problem domains.
That is interesting. I went in the other direction :)
I am tired of having to refactor shiny new things churned out at 10x the speed and that keep breaking in production. These days, if given a choice, I prefer writing them in Rust code, spending more time writing and less time refactoring everything as soon as it breaks or needs to scale.
When the pointer chasing (sometimes) comes in handy, is once you have a successful business with a lot of data and/or users, and suddenly the cost of all those EC2 instances comes to the attention of the CFO.
That's when rewriting the hot path in Go or Rust or Java or C or C++, can pay off and make those skills very valuable to the company. Making contributions to databases, operating systems, queueing systems, interpreters, Kubernetes etc. also fall into that category.
But yeah if you are churning out a MVP for a new business, yeah starting with Python or Ruby or Javascript is a better bet.
(Erlang/Elixir is also an interesting point in the design space, as it's very high level and concise, but also scales better than anything else, although not especially efficient for code executing serially. And Julia offers the concision of Python with much higher performance for numerical computing.)
If you're 'tired of chasing pointers', Rust's a lot closer to (and I'd argue better than) Python than say Go - it'll tell you where the issue is and usually how to fix it; Go will just blow up at run time. (Python (where applicable) will do something unexpected and wrong but potentially not error (..great!))
>and they don’t care - or know about - concurrency, memory efficiency, L2 cache misses due to pointer chasing.
Also if I (a programmer) want to write really really fast code I'm probably reaching for tools like tensorflow, numpy, or jax. So there's not much incentive for me to switch to a more efficient language when as near as I can tell the best tooling for dealing with SIMD or weird gpu bullshit seems to be being created for python developers. If you want to write fast code do it in c/rust/whatever, if you want to write really fast code do it in python with <some-ML-library>.
For a very specific definition of the word "fast" at least.
Your OS, the linear algebra libraries themselves, much of the user-facing software that you use (latency sensitive rather than throughput sensitive), image/video encoding/decoding, most of the language runtimes that you use, high volume webservers, high volume data processing (where your data is not already some nice flat list of numbers you're operating on with tensor operations), for some examples.
Really, for almost any X, somebody somewhere has to do X with strict performance requirements (or at very large scale, so better perf == savings)
Most of these python libraries are only fast for relatively large and relatively standard operations in the first place. If you have a lot of small/weird computations, they come with a ton of overhead. I've personally had to write my own fast linear algebra libraries since our hot loop was a sort of modified tropical algebra once.
They asked for examples of non-numpy/tf/had use cases and I gave some including my own experience? No disagreement, HPC Python in practice is heavily biased towards numpy and friends
> Also if I (a programmer) want to write really really fast code I'm probably reaching for tools like tensorflow, numpy, or jax. So there's not much incentive for me to switch to a more efficient language when as near as I can tell the best tooling for dealing with SIMD or weird gpu bullshit seems to be being created for python developers. If you want to write fast code do it in c/rust/whatever, if you want to write really fast code do it in python with <some-ML-library>.
Rather unfortunately, my current bugbear is that Pytorch is... slow. On the CPU. One of the most common suggestions for people who want stable diffusion to be faster is, wait for it, "Try getting a recent Intel CPU, you'll see a real uplift in performance".
This despite the system only keeping a single CPU core busy. Of course, that's all you can do in Python most of the time.
(You can also use larger batch sizes. But that only partially papers over the issue, and also it uses more GPU memory.)
Your comment is super interesting because it suggests Python has evolved in a direction opposite to the Python Paradox - http://www.paulgraham.com/pypar.html
Whereas before you could get smarter programmers using Python, now because of the exponential growth of Python, the median Python programmer is likely someone with little or no software engineering or computer architecture background who is basically just gluing together a lot of libraries.
Neat observation. I wasn't doing much programming in 2004, but, I'm guessing 2004 Python would be like today's Rust. People learn it because they love it.
I think more so Rust than even Python on 2004 since Rust has a pretty steep learning curve and does require a non-trivial amount of dedication to learning it.
Anything that doesn't require high performance that is. Is there any 3D game engine for python yet? I guess Godot has gdscript which is 90% python by syntax, but that doesn't quite count I think.
You won't get high performance out of Python directly, but there are a lot of Python libraries that use C or a powerful low level language underneath. The heavy lifting in so much of machine learning is CUDA, but most people involved in ML are writing Python.
Sure, but what's not really python per se. One could also call C++ libraries from java via JNI and pretend java is super fast.
If people write program logic in python it will run at python speeds. Otherwise you're not really writing python, like nobody says some linux native program is bash because it happens to be launched from a bash script.
Java is super fast though, it almost never uses JNI as it doesn’t need it as opposed to Python. It uses JNI for integrating with the C world (e.g. opengl bindings).
> Sure, but what's not really python per se. One could also call C++ libraries from java via JNI and pretend java is super fast.
But that's how every scripting language obtains good-not-just-decent performance. A strong culture of dropping down to C for any halfway-important library is why PHP's so hard to beat in real-world use, speed-wise (whatever its other shortcomings).
I completely agree - but you say that like it's a bad thing. I work as a developer alongside data scientists, who might have strong knowledge of statistics or machine learning frameworks rather than traditional programming chops.
For the most part they don't need to know about concurrency, memory efficiency etc, because they're using a library where those issues have been abstracted away.
I think that's what makes python ideal - it's interoperability with other languages and library ecosystem means less technical people can produce good, efficient work without having to take on a whole bunch of the footguns that would come from working directly in a language like c++ or Rust.
But this is a false dichotomy. The space of options isn't C++/Rust or Python. There are languages which attempt to give the best of both worlds, e.g. Julia.
> they're using a library where those issues have been abstracted away.
I work in Python, and while libraries like numpy have certainly abstracted away some of those issues, there's still so much performance left of the table because Python is still Python.
I've rewritten real world performance critical numpy code in C and easily gotten 2-5x speedup on several occasions, without having to do anything overly clever on the C side (ie no SIMD or multiprocessing C code for example).
Did you rewrite the whole thing or just drop into C for the relevant module(s)? Because the ability to chuck some C into the performance critical sections of your code is another big plus for Python.
But... pretty much any language can interoperate with C, it's calling conventions have become the universal standard. I mean, I still remember at $previousJob when I was deprecating a C library and carefully searched for any mention of the include file... only to discover that a whole lot of Fortran code depended on the thing I was changing, and I had just broken all of it (since Fortran doesn't use include files the same way, my search for "#include <my_library" didn't return any hits, but the function calls were there none-the-less).
Julia, to use the great-great-grand-op's example, seems to also have a reasonably easy C interop (I've never written any Julia, so I'm basing this off skimming the docs, dunno, it might actually be much more of a pain than it looks like here).
I’ve done the same but moved from vanilla numpy to numba. The code mostly stayed the same and it took a couple hours vs however long a port to C or Rust would have taken.
For a package whose pitch is "Just apply one of the Numba decorators to your Python function, and Numba does the rest." a few hours of work is a long time.
2-5x speedup is not a lot, I would say it is not worth it to rewrite from py to C if you don't have an order of magnitude improvement.
Because if you compare the benefit to the cost of rewrite from py to C and cost of maintaining/updating C code and possible C footguns like manual memory safety, etc - then there is no benefit left
I highly doubt that numpy can ever be a bottleneck. In typical python app - there are other things like I/O that consume resources and become bottleneck, before you run into numpy limits and justify rewrite in C.
I haven't personally run into IO bottlenecks so I have no idea how you would speed those up in Python.
But there's two schools of thoughts I've heard from people regarding how to think about these bottlenecks:
1. IO/network is such a bottleneck so it doesn't matter if the rest is not as fast as possible.
2. IO/network is a bottleneck so you have to work extra hard on everything else to make up for it as much as possible.
I tend to fall in the second camp. If you can't work on the data as it's being loaded and have to wait till it's fully loaded, then you need to make sure you process it as quickly as possibly to make up for the time you spend waiting.
In my typical python apps, it's 0.1-20 seconds of IO and pre-processing, followed by 30 seconds to 10 hours of number crunching, followed by 0.1-20 seconds of post processing and IO.
2-5x speedup barely seems worth re-writing something for, unless we're talking calculations that take literally days to complete, or you're working on the kernel of some system that is used by millions of people.
Oh, I'm familiar with numba and while it certainly helps, it has plenty of it's own issues. You don't always get a performance gain and you only find this out at the end of a refactoring. Your code can get less readable if you need to transport data in and out of formats that it's compatible with (looking at you List()).
To say nothing of adding yet another long dependancy chain to the language (python 3.11 is still not supported even though work started in Aug of last year).
I do wonder if the effort put into making this slow language fast could have been put to better use, such as improving a language with python's ease of use but which was build from the beginning with performance in mind.
Sure thing! Footguns might be the wrong word, and I know as a low level language Rust is insanely safe, but for a high level developer it's type system is gonna mean spending a lot of time in the compiler figuring out type errors, at least initially. That might not be a traditional footgun, but if you're just trying to, I dunno, build a crud api or something, its gonna nuke your development time.
Please don't read this as "rust is difficult and bad", I definitely don't think it is! But its a low level language, and working with it means dealing with complexity that for some tasks just might not be relevant.
I agree, but for something like the CRUD app example I made bringing in pydantic or something would solve that. Rust's type system is a lot stricter because it's solving problems in a space that doesn't touch a lot of Python developers.
> without having to take on a whole bunch of the footguns that would come from working directly in a language like c++ or Rust.
Don't forget the footguns of working with developers who do those things. Ask them to do something simple and you get something complex and expensive after months of back and forth about what is wanted. You're likely to a framework for a one off SQL query.
I hear it being said already, "You're using software developers wrong!" Well, maybe software developers shouldn't be so hard to use?
> maybe software developers shouldn't be so hard to use?
This whole take assumes bad intention on both sides. Nobody's job is easy in this situation. Leadership's job is to set everyone up for success. If things go off the rails and end up with months of back and forth leading to nobody being happy despite good intentions and honest effort, then the problem lies with leadership.
In fact, the development of the world is based on constant levels of abstraction, just think of assembly language and computer punch tape programming, those days are not long past.
> For the most part they don't need to know about concurrency [...]
In my opinion, this is the part that Go got mostly right. Concurrency is handled by the runtime, and held behind a very thin veil. As a programmer you don't really need to know about it, but it's there when you need to poke at it directly. Exposing channels as a uniform communication mechanism has still enough footguns to be unpleasant, though.
In an ideal world, I should be able to decorate a [python] variable and behind the scenes the runtime would automatically shovel all writes to it through an implicitly created channel. Instead of me as a coder having to think about it. Reads could still go through directly because they are safe.
If I could have Python syntax and stdlib, with Go's net/http and crypto libraries included, and have concurrency handled transparently in Go-style without having to think about it, that would be pretty close to an all-wishes-come-true systems language. Oh, and "go fmt", "go perf" and "go fuzz" as first-class citizens too.
Someone else in this thread brought up the idea of immutable data structures as a default. I wouldn't mind that. Python used to have frozenset (technically it still does but I haven't seen a performance difference for a while), so extending the idea of freeze()/unfreeze() to all data types certainly has appeal.
> It’s frequently the second best language but it’s the second best language for anything.
This myth wasn't even true many years ago, it certainly isn't true today. You can build a mobile app, game, distributed systems, OS, GUI, Web frontend, "realtime" systems, etc in Python, but it is a weak choice for most of those things (and many others) let alone the second best option.
When I switched from PHP to Python years ago I had the same feeling as the OP, then it became the third best, then the fourth, then situational when object-orientation makes sense, then for just scripting, and now... unsure beyond a personal developer comfort/productivity preference. TUIs and GUIs built on Python on my machine seem to be the first things to have issues during system upgrades because of the package management situation.
The saying does not mean that in a rigorous evaluation Python would be second best out of all programming ecosystems for all problems.
The saying means that for any given problem, there is a better choice, but second best is the language you know which has all of the tools to get the job done, so the answer is probably just a bunch of pip installs, imports, and glue code.
It’s kind of like “the best camera is the one you have with you” — it’s a play on the differing definitions of “best” to highlight the value of feasibility over technical perfection.
What worries me, though, is that the features that make Python quite good at prototyping make it rather bad at auditing for safety and security. And we live in a world in which production code is prototyping code, which means that Python code that should have remained a quick experiment – and more often than not, written by people who are not that good at Python or don't care about code quality – ends up powering safety/security-critical infrastructures. Cue in the thousands of developer-hours debugging or attempting to scale code that is hostile to the task.
I would claim that the same applies to JavaScript/Node, btw.
We still live in a world where many outward facing networked applications are written in C. Dynamic languages with safe strings are far from the floor for securable tools.
However, I hope that these C applications are written by people who are really good at C. I know that some of these Python applications are written by people who discovered the language as they deployed into production.
That’s a measure of programming prowess, not the actual security concern at hand.
If the masterful C developer still insists on using a language that has so many footguns and a weird culture of developers pretending that they’re more capable than they are, then their C mastery could very well’ve not been worth much against someone throwing something together in Python, which will at the very least immediately bypass the vast majority of vulnerabilities found in C code. Plus, my experience with such software is that the sort of higher level vulnerabilities that you’d still see in Python code aren’t ones that the C developer has necessarily dealt with.
What audits need most is some ability to analyze the system discretely and really "take it apart" into pieces that they can apply metrics of success or failure to(e.g. pass/fail for a coding style, numbers of branches and loops, when memory is allocated and released).
Python is designed to be highly dynamic and to allow more code paths to be taken at runtime, through interpreting and reacting to the live data - "late binding" in the lingo, as opposed to the "early binding" of a Rust or Haskell, where you specify as much as you can up front and have the compiler test that specification at build time. Late binding creates an explosion of potential complexity and catastrophic failures because it tends to kick the can down the road - the program fails in one place, but the bug shows up somewhere else because the interpreter is very permissive and assumes what you meant was whatever allows the program to continue running, even if it leads to a crash or bad output later.
Late binding is very useful - we need to assume some of it to have a live, interactive system instead of a punchcard batch process. And writing text and drawing pictures is "late binding" in the sense of the information being parsed by your eyes rather than a machine. But late binding also creates a large surface area where "anything can happen" and you don't know if you're staying in your specification or not.
There are many examples, but let's speak for instance of the fact that Python has privacy by convention and not by semantics.
This is very useful when you're writing unit tests or when you want to monkey-patch a behavior and don't have time for the refactoring that this would deserve.
On the other hand, this means that a module or class, no matter how well tested and documented and annotated with types, could be entirely broken because another piece of code is monkey-patching that class, possibly from another library.
Is it the case? Probably not. But how can you be sure?
Another (related) example: PyTorch. Extremely useful library, as we have all witnessed for a few years. But that model you just downloaded (dynamically?) from Hugging Face (or anywhere else) can actually run arbitrary code, possibly monkey-patching your classes (see above).
Is it the case? Probably not. But how can you be sure?
Cue in supply chain attacks.
That's what I mean by auditing for safety and security. With Python, you can get quite quickly to the result you're aiming for, or something close. But it's really, really, really hard to be sure that your code is actually safe and secure.
And while I believe that Python is an excellent tool for many tasks, I am also something of an expert in safety, with some experience in security, and I consider that Python is a risky foundation to develop any safety- or security-critical application or service.
I sometimes think about what Python would be like if it were written today, with the hindsight of the last thirty years.
Immutability would be the default, but mutability would be allowed, marked in some concise way so that it was easy to calculate things using imperative-style loops. Pervasive use of immutable instances would make it impossible for libraries to rely on mutating objects a la SQLAlchemy.
The language would be statically type-checked, with optional type annotations and magic support for duck typing (magic because I don't know how that would work.) The type system would prioritize helpful, legible feedback, and it would not support powerful type-level programming, to keep the ecosystem accessible to beginners.
It would still have a REPL, but not everything allowed in the REPL would be allowed when running code from a file.
There would be a strong module system that deterred libraries from relying on global state.
Support for at least one fairly accessible concurrency paradigm would be built in.
I suspect that the error system would be exception-based, so that beginners and busy people could write happy path code without being nagged to handle error values and without worrying that errors could be invisibly suppressed, but there might be another way.
I think free mutability and not really needing to know about types are two things that make the language easier for beginners.
If someone who's not familiar with programming runs into an error like "why can't I change the value of X" that might take them multiple hours to figure out, or they may never figure it out. Even if the error message is clear, total beginners often just don't know how to read them and use them.
They provide longer term advantages once your program becomes larger but the short term advantages are more important as a scripting language imo
The type system I want would just be a type system that tells you that your code will fail, and why. Pretty much the same errors you get at runtime. Hence the need for my hypothetical type system to handle duck typing.
I don't think mutability by default is necessary for beginners. They just need obvious ways of getting things done. There are two places beginners use mutability a lot. The first is gradual transformation of a value:
line = "The best of times, the worst "
line = line.trim()
line = line[:line.find(' ')]
This is easily handled by using a different name for each value. The second is in loops:
word_count = 0
for line in lines():
word_count += num_words(line)
I think in a lot of cases beginners will have no problem using a map or list comprehension idiom if they've seen examples:
word_counts = [num_words(line) for line in lines]
# or word_counts = map(num_words, line)
word_count = sum(word_counts)
But for cases where the immutable idiom is a bit tricker (like a complicated fold) they could use a mutable variable using the mutability marker I mentioned. Let's make the mutability marker @ since it tells you that the value can be different "at" different times, and let's require it everywhere the variable is used:
word_count @= 0
for line in lines():
word_count @= word_count + num_words(line)
Voila. The important thing is not to mandate immutability, but to ensure that mutability is the exception, and immutability the norm. That ensures that library writers won't assume mutability and rely on it (cough SQLAlchemy cough), and the language will provide good ergonomic support for immutability.
It's a common claim that immutability only pays off in larger programs, but I think the mental tax of mutability starts pretty immediately for beginners. We're just used to it. Consider this example:
Beginners shouldn't have to constantly wrestle with the difference between value semantics and reference semantics! This is the simplest possible example, and it's already a mind-bender for beginners. In slightly more complicated guises, it even trips up professionals programmers. I inherited a Jupyter notebook from a poor data scientist who printed out the same expression over and over again in different places in the notebook trying to pinpoint where and why the value changed. (Lesson learned: never try to use application code in a data science calculation... lol.) Reserving mutability for special cases protects beginners from wrestling with strange behavior from mistakes like these.
Julia is both dynamic and fast. It doesn’t solve all issues but uniquely solves the problem of needing 2 languages if you want flexibility and performance.
Exception error handling - and their extensive use in the standard library -is the fundamental design mistake that prevented Python becoming a substantial programing language.
Coupled with the dynamic typing and mutability by default, it guarantees Python programs won't scale, relegating the language to the role of a scratchpad for rough drafts and one off scripts, a toy beginner's language.
I have no idea why you say that it's a scratchpad or a toy language consdering that far more production lines of code are getting written in Python nowadays than practically any other language with the possible exception of Java.
But that's the same with Excel: massive usage for throwaway projects with loose or non-existing requirements or performance bounds that end-up in production. Python is widely used, but not for substantial programming in large projects - say, projects over 100 kloc. Python hit the "quick and dirty" sweet spot of programming.
This is absolutely not true. I’ve made my living working with Python and there’s an astounding amount of large Python codebases. Onstage and YouTube alone have millions of lines of code. Hedge funds and fintechs base their entire data processing workflows around Python batch jobs. Django is about as popular as Rails and powers millions of websites and backends.
None of those applications are toys. I have no idea where your misperception is coming from.
I guess I'm more than a little prejudiced from trying to maintain all sorts of CI tools, web applications and other largeish programs somebody initially hacked in Python in an afternoon and which grew to become "vital infrastructure". The lack of typing bytes you hard and the optional typing that has been shoehorned into the language is irrelevant in practice.
All sorts of problems would simply have not existed if the proper language was used from the beginning, as opposed to the one where anyone can hack most easily.
A popular opinion in game development is that you should write a prototype first to figure out what works and is fun, and once you reach a good solution throw away that prototype code and write a proper solution with the insigts gained. The challenge is that many projects just extend the prototype code to make the final product, and end up with a mess.
Regular sofware development is a lot like that as well. But you can kind of get around that by having Python as the "prototyping language", and anything that's proven to be useful gets converted to a language that's more useful for production.
I’ve been a Python developer for 15 years, and Python might have been the second best language for anything when I started my career, but there are so many better options for just about any domain except maybe data science. Basically for any domain that involves running code in a production environment (as opposed to iterating in a Jupiter notebook) in which you care about reliability or performance or developer velocity, Python is going to be a pretty big liability (maybe it will be manageable if you’re just building a CRUD app atop a database). Common pain points include performance (no you can’t just multiprocess or numpy your way out of performance problems), packaging/deployment, and even setting up development environments that are reasonably representative of a production environment (this depends a lot on how you deploy to production—I’m sure lots of people have solved this for their production environment).
Python isn't a joke either. I'm a full-on programmer who started with C and branched out to several other languages, and I'd still pick Python for a lot of new tasks, even things that aren't little scripts. Or NodeJS, which has similar properties but has particular advantages for web backends.
It's a fine article and makes many of the standard points. But I would expect a slightly more data driven analysis from GitHub given the large amount of data available to them. For example, what percentage of first repositories (new GitHub users) are in Python? What percentage of GitHub Python repositories use a data science library? Etc.
I have been a heavy Python user now about 15 years, but for me now I'm increasingly reaching for modern JavaScript and particularly TypeScript to do the things I would have traditionally done with Python.
ES modules, fat arrow expressions, and all the other nice new syntax and library features have made the language so more pleasant to use. In many ways the ergonomics of TypeScript in particular are far superior to Python now, and I find that really surprising.
I haven't yet found a replacement for Django (particularly the ORM/Admin/Forms combination) it is just so incredibly productive to use. So, I don't think I will be moving off it, but it's certainly not the growth language for me anymore.
I am going this way too. I recently tried to onboard a few contractors with limited python experience. It took several deep sessions to work out why they couldn't get an environment set up. After 10+ years I've never come across the install certificates command. There's still no way to get a dev environment with a specific version of Python working with one command that works reliably everywhere. pyenv isn't even packaged for Linux, you have to install from source.
This, plus Typescript's superior type system makes it very tempting for application development.
However I'll probably wait until there's a really good Jupyter Notebook equivalent...
Oh, I wouldn't say TypeScript is more pleasant than Python, but it is more pleasant than old JavaScript. But I increasing precise the productivity I have with TypeScript and some of the ergonomics than Python.
I still love Python. Just feel like I'm cheating on it with this new younger model...
Same. I analyzed a little why I find JS (I don't like TS) easier to deal with for some tasks:
- Particular support for web frontends or backends, both very broad categories.
- JS concurrency is easier to deal with. Focused entirely on promises with the nice async/await syntax on top, unlike Python which slapped on too many different ways to do this.
- By far easier package management and imports. Python's is so annoying that any project you download is gonna have you spin up a Docker container for it.
- ES6 added a lot of array/etc managing that JS was lacking before.
- Freeform objects with {key: value} syntax are convenient, despite maybe seeming weird at first. Python OOP somehow got really complicated over the years.
- Inline functions (I use fat-arrow but regular way is also fine). I never got why Python, despite being common in function-oriented programming, didn't let you do an inline def.
"Python 2 is unsupported by the Python Foundation since 2020-01-01" according to Debian docs. Ubuntu defaulted to Py2 until around then. So I'd say that's when the drama ended, then again it's probably not the last I've seen of Py2.
If you're looking for a Django replacement, check out Adonis. Laravel in PHP was heavily inspired by Django but is much better and Adonis is an exact clone of Laravel. I think you'd like it.
I'm a bit surprised to see this article on GitHub blog, it feels more like something from dev.to - looking at the surface, with little actual insights.
Most of the provided reasons behind Python's popularity are true also for other languages - portable, open source, productive, big community. This can be also said about PHP, Ruby, or Perl back in 2000s. Why isn't Perl as popular as Python?
I don't think it's all about readability or productivity, but about tools that were built over the last 30 years that have been used in academia and now with the boom in ML/AI/Data Science, they made Python an obvious choice to use for the new generation of tools and applications.
Imagine that the boom in ML/AI didn't happen - would Python be #1 language right now?
As commented somewhere else in this thread, Python was clearly more ergonomic than Python, hand had a lot of mindshare exactly for this reason. I remember when Python was new and the not that professional choice, Perl was at that time for that niche. Now still I don't see a contender for a language where speed doesn't matter. Ruby has some Perlisms that really make it weird, PHP is tight to the web, and equally weird, these $s and @s are really bad for normal people. Python wins clearly when teaching somebody programming.
I’d say that Ruby and even Perl are a lot nicer for scripting than Python (due to the extremely low-effort unix interop). Python can do it but it’s a while lot more verbose and difficult for a beginner to learn than “anything inside a pair of backticks is run as a system command and you can interpolate variables”.
Python was friendlier for beginners than Ruby the first time I took a real stab at learning to code during a CNY holiday in 2008, but it wasn’t about the language itself. Ruby was harder then because many of the popular libraries and many of the tutorials were written by people who considered Windows support as an afterthought. It’s hard to express how frustrating it was to have my vacation days ticking down, hitting issues in one tutorial after another and having people suggest I install linux on a VM (a process where I hit still more snags).
People learning Python and PHP didn’t hit that hurdle. I ended up learning Flash on my Asus laptop a couple of years later and getting my start that way and not coming back to Ruby until six years later when I was a much more experienced dev.
I don't think there is a single reason, but it sure didn't help that the community self-destructed by trying to make an entirely new language after version 5 and still call it Perl. It took a lot of years to resolve that nonsense, and in the meantime many people moved on.
It also does not help that Perl is a creative language, useful but very much open to many different interpretations. Hiring a perl guy and expecting them to read someone else's code is a crapshoot. The upside to Python's strong cultural opinions on coding style makes it easier for one developer to pick up someone else's code.
> Imagine that the boom in ML/AI didn't happen - would Python be #1 language right now?
Probably not. But it wouldn't be perl, either. Javascript most likely. But the core usage of python for scripting was never predicated on ML popularity, so it would still be a pretty commonly used language. and javascript has many annoying warts too, so I think plenty of people would still choose to write django apps instead of node, whether ML existed or not.
Perl was significantly more popular at one point, but it slowly lost traction while Python gradually gained traction over the years.
Better ecosystem for numeric computing is definitely a big reason for the success of Python, by the question is why Python gained a foothold in that niche in the first place. It think it is because Python is just a lot more accessible to people with different backgrounds. Perl really grew out of shell scripting as a supercharged alternative to Bash and Awk, but retaining many of the quirks for familiarity. Python on the other hand grew out of research in teaching programming to beginners.
The article assigns significance to the language when the real work is done by the libraries. And many good libraries inspire even more good libraries. At that moment the language no longer matters. But it matters in the beginning when people have to create the initial environment and this is what should've been underscored. Python lets you do so many things in so many ways that minor mistakes do not matter and everyone has the freedom to experiment, achieve results, discover that they've done something stupid, but at that moment you already feel confident to do it again, but better.
5his is right in the spirit of: The Python community is a bunch of people learning programming together.
Someone else made this remark once, when the indroduction of type annotations was discussed as the python community discovering the benefits of static typing.
Funny enough this sort of sentence, i.e one with a missing word, is a very common ML training task for large language models like Gpt3: the models objective is to predict the correct word from a set of possible answers.
Getting python bootstrapped is less easy than getting make bootstrapped, plus python isn't a GNU tool so I don't see them adding a dependency on it to all their packages.
GNU Make is really far more sophisticated than people give it credit for and it's not that easy to replace it.
Well as I pointed out, it does happen e.g. in Meson.
Make doesn't do any of the things you mention - it just works out what targets need to be rebuilt and the code to do it is in the rules that you have to write in a shell language like bash.
I admit you can use builtin implicit rules for C and get away with that up to a point but only a fairly low point.
python is far harder to use than bash when it comes to running processes and doing things with them and doing the file-system manipulation that is usually wanted when building.
There are modules that make it easier but so far as I've seen it's a crap choice plus python takes a long time to start up so there is a cost to running a "clean" environment for every build step but OTOH if you use the same interpreter for the whole build you can introduce all sorts of ordering problems when a build runs on a different machine and doesn't work because it's executing build tasks in a different order.
If you really want to build by "writing a script" (groan - because that's the age old horrible solution) why bother with python?
>python is far harder to use than bash when it comes to running processes and doing things with them and doing the file-system manipulation that is usually wanted when building.
Can easily be solved by having libraries that does. Which is far better solution that hacking/chaining 5 different obscure unix tools to do string manipulation.
>you can introduce all sorts of ordering problems when a build runs on a different machine and doesn't work because it's executing build tasks in a different order.
Surely it is better than make? Even if make is "better" because it "just works". This is basically Hyrum's Rule waiting to be unleashed.
>If you really want to build by "writing a script" (groan - because that's the age old horrible solution) why bother with python?
Because I personally cannot decipher make and I think this is the experience of many developers as well. And how many make derivations are there? Its basically an arcane spell that needs to be chanted with every project (configure -> make -> make install).
I remember the evolution of Python. When I started with my brand new Ubuntu installation when Ubuntu was also very new, there were two languages, Perl and Python. Because Perl was more popular at the time I check some books in the library, and wrote some scripts. But already then the consensus was, that even though libraries are lacking (!), it is the better, cleaner language. Another big push was the bad enterprise guys, that like static typing, so C, C++, Java, vs. the hipster dynamic language guys. Some tenets were broken, speed is not as important as language concepts, that enabled fast iteration. Agile was new, and Python was very agile. There was no real alternative to Python at that time, that was as sane, and beautiful. That's the reason why we have now such a nice variety of libraries, because it is just a sane language to build on, if speed is not your business. Ruby was a contender, but not that much better in the end. Now Javascript kind of took the edge away from Python, even it wasn't for the browser and the web, Python would dominate much more. Javascript as a language is worse than Python because of its cruft, so I don't see a big rewrite of ML tooling into Javascript. Maybe Typescript.
Perl was very popular and I think Python got its boost from being seen as the more sane replacement to it. At that time nothing else really offered to solve the pain of Perl in the same way except Ruby and Ruby was even slower
It's interesting that people keep claiming that indentation-based blocks makes Python easier to learn and read. I've been teaching programming to students in banking, finance and insurance for a few years, using Python, and my experience with them is the opposite. They have been struggling with that a lot. They didn't pay too much attention to white spaces and tabs, probably because they are "invisible", and couldn't see why a statement was not executed at the end of the block, because it was not indented correctly. For me, it was obvious, because as a professional programmer, I've been trained to pay attention to details. But for most of them, it made no sense. Explicit block markers are much easier to teach.
> Explicit block markers are much easier to teach.
The point of communication is to express something in a way that the listener understands it. Simply expressing something that is comfortable or normal to the speaker is simply not useful for either party of the goal is communication.
You're right. What matters is that explicit block markers are easier to learn (from the student's perspective). Being easier to teach (from the teacher's perspective) doesn't matter as much.
Sure, but without the white space defined blocks it is entirely on you to teach them about the importance of formatting their code to ensure readability.
My point was to express that your students need to learn to communicate with the programming language in a way that the language understands, not in a way that is comfortable for them as the “speaker”. You as the teacher should be setting that expectation.
On the other hand, back in college I was a TA for an intro to programming course that used Java. This:
> They didn't pay too much attention to white spaces and tabs, probably because they are "invisible"
can get so much worse than people imagine, when the language doesn't enforce it. It was that experience that made me lean towards python being a good introductory language, to help people get used to correctly indenting code in general.
IMO, the thing about beginners and relevant white space is that it forces them to learn how to indent their code, not that it makes it any easier to learn the language.
Code is still completely unreadable with explicit block markers, if it is without appropriate indentation and newlines. That's like trying to read minified javascript. And python forces you to have these, to some extent.
I used to think the same, but go fmt totally changed my mind on this. Most languages now have a code formatted that is integrated in all popular text editors and IDEs.
Python is the ideal glue layer for low-level languages, now mostly Fortran/C/C++ but there are already many projects exposing Rust libraries through Python, which for me is a great combo of performance and usability.
Sure, but that's because there was a good library available, little to do with Python vs C++.
If there was an equally good C++ image library available then it would have been equally simple in C++.
Of course the thing is that, in general, there may well NOT have been such a C++ library, and this is what makes languages such as Java and Python popular - because of the breadth of libraries available.
For C++ image manipulation I've found libvips to be decent, but the point still stands that for a given problem there may not be decent libraries available.
I think one reason for Python growing popularity is because it's become the default tool in some domains whether it is the best tool or not.
This week our Director ordered a total rewrite of two years of work in Python. His rationale: it's what everyone else uses in this space. No reason specific to our use case, just simply to follow the herd. I realise that a large community translates into easy hiring and rich ecosystems, but I despise the mentality as it promotes a monoculture.
... and Python _became_ the default tool because the de facto developer consensus (after years of competing languages) is that an interpreted language should
= be usable, and
= provide a set of data structures that an educated programmer *expects to find when scripting*.
Python literally sucked less than the alternatives.
Python gets introduced to students, so for many people it's the first language they learn. Half the programming community have less than 5 years of experience. I question their ability to evaluate suckage, lol.
What are you rewriting from? At director level focus is usually more on things like how easy is it to staff / get support for something. Python is strong here - you can find programmers globally who can do pretty well with it.
> This week our Director ordered a total rewrite of two years of work in Python
WTF? Unless your system is originally written in a proprietary language that literally no one outside your company knows, I'll say it's a good sign that you need to change team (or change job). Don't work under a director like that.
Sorry if it sounds too cynical, but that's probably the intended effect of a rewrite from what the team knows into Python: staffing changes. A bunch of the (expensive) old guard will leave and they can be replaced with cheap grads, who all know Python.
Yes. The entire reason I like using Java at my day job is that the rest of the company uses it the most and supports it well. I would never use Java on my own, but that's a different situation.
It reads as a little bit of a tautology. "People are using it because people use it". I get that from a hireability standpoint it's a real thing to consider, but the statement doesn't say anything about whether or not Python is actually a good language to use
I think people embrace Python because they think it's easy to learn and use. While that might be true, the real complexity lies not in learning and using a language (unless it's Rust) but in learning the libraries, frameworks, idioms, dos, don'ts, tooling.
I remember when Twitter rewrote the code in Java because Ruby wasn't suited for the task. Probably this will happen to many large Python code bases in the future.
Perhaps, but it seems the shelf-life of anything AI/ML related (huge part of Python usage) is so short that it'll become obsolete before it ever becomes worth rewriting.
So impressive to see all trading solutions/bots made with Python! What do you think about Julia in this field: I see it as a viable alternative even if it lacks the huge amount of Python libraries?
This is a strange article. It's got the talking point about Python that we were hearing about 10 years ago - "tired of those pesky curly brackets in Java, try this new language you might not have heard of: Python!". Who reading the GitHub blog has not heard of Python?
Also, that snippet used in the "What is Python commonly used for" section is strange:
import antigravity
def main():
antigravity.fly()
if __name__ == '__main__':
main()
It's overly verbosely written (especially given the example just above about how you don't need main function in Python) and refers just to an insider joke/Easter egg. I can't see that it's going to convince anyone to try Python, only make them feel that they're on the outside of a joke.
It then ends with what seems like it might have been the point of the article, an advert for Copilot. It seems the way to get started writing Python is to write a short comment and then spam <TAB> and let the AI auto-complete your project.
(Also, and perhaps less importantly, looking at the author's GitHub profile I can't see a single instance of Python there. Though I'm not doing a deep-dive as that feel overly picky and there's plenty of private contributions that could well be Python.)
>It's got the talking point about Python that we were hearing about 10 years ago - "tired of those pesky curly brackets in Java, try this new language you might not have heard of: Python!"
That was a talking point closer to 20 years ago, at this point.
Yes, I know. But some people still have the reflex from Python 2 and feel bitter when the error message says "I know what you want, and I'm not giving it to you."
I read the article and had the same feeling that it's a fluff piece without substance. If you've to compare anything, compare it with the vibrant JVM ecosystem. Using the same tired argument of `System.but.Println()` shows the author has no original idea. Python is great but JVM is no lackey, it is a marvelous piece of battle tested engineering.
In the end it is just an ad for Github products and not worthy of being on HN frontpage.
> Who reading the GitHub blog has not heard of Python?
The GitHub blog became a strange place recently(-ish). It went from a factual blog describing fancy new GitHub features and interesting technical stuff (as it was a decade ago) to mostly a place full of incoherent marketing fluff like this post (with some real technical content interspersed).
Curly brackets or brackets at all aren't mentioned by name in the article - is that your interpretation of the first code example?
IMO, that example is there to show that there's less required boilerplate required in python compared to java when doing the same thing. And, in particular, none of that boilerplate really matters to what you want to do - print hello world - emphasizing the point of python being simple.
The section that explains why python is good for AI talks about pybrain, a library that seems to date from 10 years ago. I’m pretty well versed in most ml frameworks and never heard of it. Last update to the website looks to be 2010. Weird to feature that and PyTorch as examples of ML libraries. No mention of sklearn which is vastly more popular
458 comments
[ 3.0 ms ] story [ 346 ms ] threadHaving JavaScript in the front end and backed both can reduce resource requirements as well.
You're not forced to dip into the OO side of things though and I've heard people suggest to me that Python is a language where no rules apply and no structure exists. They feel that they can dive in without the care they would have to take in Java or C++. I think they are very wrong but you can certainly write terrible code if you want and perhaps this "all-things-to-all-people" aspect of it is part of the success.
Understandable code is maintainable code. Abstraction makes comprehension possible — imagine if every call to print were an inline block of assembly instead! — and Python’s modules provide a very quick and easy way to break up large code into small modules.
And it’s not very portable the second dependencies with native code (which is common since pure Python is too inefficient for many tasks) are used.
I think Python is popular for two reasons:
- it’s believed to be beginner friendly compared to other languages. I’m not really sure why - maybe the whitespace?
- it has an enormous set of libraries available to use
In other words, Python has momentum. The language itself isn’t great, but that is a secondary concern to most, if they are even aware of this at all!
I am leaning more towards it looking like pseudocode
> - it has an enormous set of libraries available to use
I believe that to be the actual reason. I've long held that python is a kinda bad language, but with FANTASTIC libraries
ftfy - it's the only way a tragically slow language like Python can keep up.
Edit: didn't forget FORTRAN
That is simply not true. Even libraries like numpy, scipy and scikit-learn are majority python code.
https://stackoverflow.com/questions/1825857/how-much-of-nump...
For the vast majority of use cases, performance just isn't a priority. Doubly so for Python, that shines for simple automation, command line applications, and perhaps some serveless computing.
Being easy to write, having a good ecosystem of libraries, and being widely known is typically good enough. I wouldn't use Python to write a robust backend server side application, mostly because the language doesn't lend itself well for it.
If it was too slow, we'd be doing all of this in Java, the C# or maybe doing it in C/Fortran. But because of some early design decisions (Guido being on the matrix-sig helped), the history behind Numeric/Numarray and finally NumPy and SciPy being based on those efforts allowed it to thrive.
Those were your words, not mine. I need not make any assumptions.
I just replied listing use cases where Python shine due to its strengths, performance being mostly irrelevant. I didn't even mention data science.
And although it's beyond the point, if I was to use Python, why should I care in which language a library was written? If the language allows libraries written in other languages, this is actually a nice feature.
That’s actually my primary use case for python, playing with C/C++ libraries in a repl because they don’t natively have one.
Sure, it takes work to wrap a library but that’s something I enjoy doing.
Just wait until they learn the python interpreter is written in C… <grabs popcorn>
the best teaching language (imo) is one with few features or surprises. I think Scheme might come out top here.
Depends what you are trying to teach. If you're trying to teach computer science and programming fundamentals, then sure. If you're trying to teach people how to get 'real work' done quickly and efficiently then Scheme will only get in the way and slow people down.
For example when I've taught programming, one of the tasks I taught fairly beginner programmers was to grab some satellite images between certain dates, try to detect if there is a forest fire, measure the spread of the fire and plot the spread on a map.
With python (and its excellent libraries) this is quite quick and easy, and most people are up and running and hacking around with their programs in pretty short order. They find it really cool and inspiring and makes them quickly realise that programming could be something useful in their day to day job. Trying to start with chapter 1 of SICP and Scheme and working up from there to solving the above problem would probably lead to most of these people giving up on programming very quickly. That being said the few people that made it all the way through that would no doubt be much much better programmers because of it.
It looks like English. That's all pseudocode is, and they know that well enough.
If we want to measure developer productivity we should try to compare actual real world usage that goes beyond "hello world" using Notepad. The author did provide more examples in the article, but I think we should just retire the basic minimal hello world example for these types of discussions.
If their argument is that Python is simpler for quick scripts and programs, I agree. The same applies to ML / AI where Python has lots of great tools. Once you start looking at other areas (Django and FastAPI), there are many alternatives based on other languages where you can be just as productive.
Most of the time consuming work is spent doing business logic where the differences between languages doesn't matter as much. They all have advantages and disadvantages. Personally I prefer static typing for larger projects and teams.
If their argument is that Python is better in general, they need to provide better arguments.
Good bait. I'll take it.
- dynamic, weak (really "duck") typing, meaning users don't have to worry about conversion between things. Want to print() a dictionary of whatever? Sure!
- no semicolons to terminate a statement. End of line, that's it
- rich standard library, so you can actually get going on things without having to go on a quest to find the right library for your thing. JSON? It's right there. argument parsing? argparse. Http? http.client works...
- also yeah the "pseudocode" thing. Python is light on extraneous syntax.
Now eliti^H^H^H advanced programmers will frown on many of these same things that make it beginner-friendly...
Dynamic typing means that complex designs have to be careful with their APIs, lest you can get tangled up in deep type errors if you're not careful (I particularly hate the "mix-in" pattern). No semicolons mean, uh, long statements need to escaped? The standard library is "where lib go to die" because they can't evolve as much. "Significant whitespace!? What is this, COBOL!?!11"
Ability to do structural printing of arbitrary structures is not restricted to dynamically typed languages.
However, I actually think a good beginner language could be dynamic or static.
Also, say I want to make a nice plot of some tabular data (.tsv), I do:
Boom, what a plot (or large grid of plots actually), so much information. Can anyone show me how to do this in some other language (but R)? Idk, maybe I'm just not so smart, I just learned to program at 35 after always being a biologist, but any venture into any language has me thinking: Why does this have to be so complicated?(Btw, I'm putting 4 spaces in front of the code, why is it not rendered as code?)
When I just started I used Python to read, sort and transform images and output them to a PowerPoint file. One can use CSS like synthax to format the slides. Boom PPT with 120 slides with images and data from some Excel file that convinced a lot of people with a lot of data that fluorescent images next to H&E stains + metadata can be nice. Is it the best way to present such data? Meh. But man was it cool and easy and time saving.
Maybe it also doesn't help that out there, in the C++, Rust, Javascript, Go world I'm going to run into people with usernames like "ihatepython".
quote_from_user_ihatepython == """ ihatepython 1 day ago | parent | context | prev | next [–] | on: Moscow Metro launches longest metro circle line
"""It is however what a lot of people who 'program' at work do every day. Python actually lets working professionals solve real problems they actually have at work quicker and easier than any other programming language.
But, with a user name like that, I know I’m just wasting my time responding to you.
But what if your web request fails? Do you deal with it in time or use some_value_I_want which might contain junk?
Python does have a great data processing ecosystem. But that isn't really a property of the language.
Some languages have a good integrated tool chain (Rust, Go) but are not as approachable and forgiving.
Some are approachable but lack the friction-free story for locating and installing 3rd party dependencies.
Someone else identified Ruby's fatal flaw as not having good C tooling, and I think that's probably accurate.
Python definitely found footing based on its pseudocode readability. It’s gotten a bit worse recently with too many colons and features but already made it.
It shows an obvious reason why a lot of beginners choose python, even when learn Java basics in university first.
1. Near zero boilerplate. Python's boilerplate is usually no more than setting up a class and then calling a method. Often you can skip the class and just call it directly. This is probably the biggest strength; to give you an example, my standard test for a languages approachability is "how much work do I need to put in to get a very simple JSON file from a web URL" (nothing fancy like POST, just an HTTP GET). With python, a call to urllib.request.urlretrieve and then a call json.loads are all you need. In Java, you need to manually implement a bunch of boilerplate code that looks horrible, need to think about the size of your response in memory and often need to pass in configuration options that should be a case of "works by default" but aren't because the standard is ages old so it needs to be manually toggled on. Part of that is that the Java stdlib consists largely of reference implementations rather than actual implementations, which means that anyone who wants to implement something in Java will usually end up falling back to the stdlib interfaces, which in turn suffer from being stdlib interfaces, so a lot of "should really be on by default" expectations aren't on by default since the stdlib is where code goes to die, so you instead start piling on features.
2. Interop with C. I hold the opinion that C is a great language that doesn't work very well once you get to any form of scaling. Python allowing developers to take the slowest parts of their code and writing it in C to speed it up is one of the easiest speed gains to make and it avoids the biggest bumps that come with using C as your primary language in terms of project structuring.
3. Library support is as you say, good. If you need it, there's probably a package for it. Pypi is dependency-wise kind of a disaster but if you know how to set up requirements.txt, it works really well. Most libraries ship with "sane" defaults too.
All of these combine to a language that's easy to prototype, easy to expand
It's important to keep in mind that pythons biggest successes aren't in the speed-focused, low-memory environments where every speedgain is necessary. Its success lies in conventional desktops and servers, which have much more processing power and often have more leniency in being a bit slower.
With python you can write something in 3 hours what would take a day in another language, at the cost that instead of being lightning fast and done in 10 seconds, you need to wait 30 seconds. That's an issue for some environments but in 99% of the cases that's not a problem.
That isn't to say the language is perfect (no language is), but speed of development at the cost of slightly slower execution time is the main reason why python got popular.
In C# you just have to do this: var things = await httpClient.GetFromJsonAsync<List<Thing>>("url");
Most languages nowadays tend to implement some variation of this specific convenience because it's just one of the most frequently needed things; setting up a separate parser and then calling it might be the "cleaner" option, but it's also more boilerplate and the industry has largely moved to try and avoid that.
0: https://learn.microsoft.com/en-us/dotnet/csharp/programming-...
Well yes it it does now, since C# 10 or, but in Python is have been a one liner since the beginning 30 years ago, and over the decades it has build a following. And even with recent efforts to cut down on boilerplate, C# still has more cryptic syntax than Python.
I'd argue C# is more maintainable in the long run due to the static typing, but there is no way it is as accessible to beginners.
Experienced programmers tend to forget how the experience was as a beginner programmer. But Python grew out of actual usability research into teaching programming to beginners. Logo is another language with the same background, but it never escaped that niche. The success of Python is because it attractive to beginners but remains a powerful tool as the programmer becomes more experienced.
Whitespace is definitely a factor - or rather the lack of redundant braces. Beginner programmer seem to really struggle with the lack of correspondence between the visual structure and the logical structure in most languages. Even experienced programmers get tripped up by erroneous indents. Python just solves this once and for all.
The lack of type annotations is also a factor. For beginners this is just an additional layer of complexity.
Scheme is great for teaching computer science students, but if you are just a regular Joe researcher wanting to get the job done, you want to write "2 + 2" like everybody else in the world, not "(+ 2 2)"
Most languages are designed to attract experienced programmers, e.g. by having C-like syntax which they may already be familiar with. But it has a cost for beginners.
That's not a sales pitch for python, it's a sales pitch for the concept of a scripting language; it's like saying "you should really buy a Ford, it comes with four wheels".
I wouldn't say it's friendlier than the alternatives, Perl and Shell scripts sure, but not when compared to Javascript, Ruby or Lua.
Now, if you're talking about libraries, support, etc. then sure, Python wins hands down, but that doesn't make it a better language in itself. I'd say Ruby and Lua are a little bit better as languages.
But then again, I don't care much for the language in itself, so Python is enough for most of my use cases.
In my job I've seen some beginners starting with R, and quickly hating it because they don't feel they can do much on their own, but copy-pasting and then modifying from the examples and the tutorials. And it the changes go too far, everything collapses with cryptic errors. When you show them Python as an alternative, pointing that they shouldn't use it over R for statistics and graphics, they like that they can build ideas from the scratch. That beginner is hooked for life.
The story of Ruby is altogether different: they made the fatal mistake of not defining a C foreign function interface in the standard, otherwise I imagine we'd be seeing numerical computation and ML libraries with a Ruby interface today. Still, Ruby lives on in Metasploit, and in Sorbet and Crystal.
This to me is extremely plausible, and sad.
- Lua's standard library is so weak that it makes most other batteries-not-included languages look like they have large, robust, and helpful standard libraries.
- It's got a bit of the quirkiness and gotcha-ability of JavaScript but without its being a language that's impossible to avoid due to capture of a mega-popular platform, which is what propelled JavaScript to ubiquity despite its being kinda shit and unpleasant to work with.
- Tooling's not as good as many other languages.
(FWIW sometimes I write Lua regardless, because it's the right tool for the job)
In fact I'd argue that python enjoying the success it has, despite probably the worst handling of a version bump in any language (2->3), is a testament to its popularity.
Ruby would have a much bigger market share these days if library path dependence was that powerful.
Chiefly, because big money corp invested massively elsewhere.
Also by now it’s easier to find developers who are cheaper and already (only can) use javascript/python?
I think that actual technical merits are dwarfed compared to other forces at stake here.
https://github.com/elixir-nx/nx
I don't see how it could ever overtake Python, but it could establish itself as a viable niche alternative.
It’s just that you don’t have to write testable, type-safe code if you don’t want to.
Some of it is the language too, but modern Java can definitely be reasonably terse. Record classes has done a lot for the language, so has lambdas and streams.
Modules/Project Jigsaw is arguably an even better step in the right direction. It arguably allows a much saner granularity of visibility and access.
Example:
I would argue that there is nothing simple about the last if-statement and will probably be very confusing for new developers.The only thing that isn’t perfectly straightforward is “what is __name__”, but once you know how __name__ is defined…
OTOH, that example is much more complicated than needed. As a simple executable script, that would do the same thing if antigravity. Fly() actually did anything, all that is needed is:
The rest is unnecessary boilerplate to provide a module that can operate either as a library or a script, which is superfluous here. Moreover, since the actual functionality it is demonstrating is all in an import hook, all you actually need (the rest just produces an error message) is:Its…not, actually. A more typical, and testable, example of python would be:
Your version:* Isn’t testable because it runs the sys.exit() call unconditionally on import, so there is no way to import the module and call main() from test code.
* Fails with an error, because main has a required argument that it isn’t supplied in the unconditional main() call inside the sys.exit() call.
* Isn’t following a typical python idiom by returning a value from a function and have it be unconditionally 0; if the only options are a fixed return and a exception somewhere, the convention is to return None (which doesn’t require an explicit return.)
* Since a normal exit is the default response of ending the main program, sys.exit() is unnecessary here; “Exit with 0 if main() returns successfully, and with an abnormal exit on exceptions in main” is achieved by just calling main() and having nothing after it in the main program.
By the time a python programmer has «graduated» to learning a second language, exponential growth has created a bunch of new python programmers, most of which don’t consider themselves programmers.
There are more non-programmers in this world, and they don’t care - or know about - concurrency, memory efficiency, L2 cache misses due to pointer chasing. These people all use python. This seems to be a perspective missing from most hackernews discussions, where people work on high performance Big corp big data web scale systems.
If I write something in C++ that does a task in 1 second and it takes me 2 days to write, and I write the same thing in Python that takes 2 seconds but I can write it in 1 day, the 1 day of extra dev time might just pay for throwing a more high performance server against it and calling it a day. And then I don't even take the fact that a lot of applications are mostly waiting for database queries into consideration, nor maintainability of the code and the fact that high performance servers get cheaper over time.
If you work at some big corp where this would mean thousands of high performance servers that's simply not worth it, but in small/medium sized companies it usually is.
I have code running that reads ~20 bytes, checks the internal status on an hashmap and flips a bit.
Would it be faster in C? Of course.
Would it have taken me much longer to write to achieve absolutely no benefit? Yes.
This is the first line in most scientific code:
What slows Python down is generally the "everything is an object" attitude of the interpreter. I.e. you call a function, the interpreter has to first create an object of the thing you're calling.
In C++, due to zero-cost abstractions, this usually just boils down to a CALL instruction preceded by a bunch of PUSH instructions in assembly, based on the number of parameters (and call convention). This is of course a lot faster than running through the abstractions of creating some Python object.
Nah, it’s the interpreter itself. Due to it not having JIT compilation there is a very high ceiling it can not even in theory surpass (as opposed to things like pypy, or graal python).
Python is primarily slowed down because of the fact that each attribute and method access results in multiple CALL instructions since it's dictionaries and magic methods all the way down.
How so? Python is dynamically typed after all and even type annotations are merely bolted on – they don't tell you anything about the "actual" type of an object, they merely restrict your view on that object (i.e. what operations you can do on the variable without causing a type error). For instance, if you add additional properties to an object of type A via monkey-patching, you can still pass it around as object of type A.
Also, if you are interested, “shapes” are the primitives of both Javascript and python jit compilers instead of regular types.
Commonly it creates the .pyc files, so it doesn't really re-parse your code as a string every time. But it does check the file's dates to make sure that the .pyc file is up to date.
On debian (and I guess most distributions) the .pyc files get created when you install the package, because generally they go in /usr and that's only writeable by root.
It does include the full parser in the runtime, but I'd expect most code to not be re-parsed entirely at every start.
The import thing is really slow anyway. People writing command lines have to defer imports to avoid huge startup times to load libraries that are perhaps needed just by some functions that might not even be used in that particular run.
That doesn’t really take any significant time though on modern processors.
And OCaml (which offers a productivity comparable to Python) is sensibly slower than Rust or C++.
But yes python is slow.
However I've seen good python code be faster than bad C code.
I was talking specifically of pure python code (except the python's standard library itself, where it really is unavoidable).
But sometimes, you do end up writing that compute heavy piece of code. At that stage, you have to learn how to write your own native library :)
Speaking of which, I've written some Python modules in Rust using PyO3, its' a very agreeable experience.
Code is "compute heavy" (could equally be memory heavy or IOPs heavy) if it's deployed into many servers or "the cloud" and many instances of it are running serving a lot of requests to a lot of users.
Then the finance people start to notice how much you are paying for those servers and suddenly serving the same number of users with less hardware becomes very significant for the company's bottom line.
The other big one is reducing notable latency for users of your software.
With Python I can write and test on a Mac or Windows and easily deploy on Linux. I can iterate quickly and if I really need "performance" I can throw bigger or more VPSes at the problem with little extra cognitive load.
I do not have anywhere near the same flexibility and low cognitive load with C++. The better performance is nice but for almost everything I do day to day completely unnecessary and not worth the effort. My case isn't all cases, C++ (or whatever compiled language you pick) will be a win for some people but not for me.
E.g. people who once wrote "robust" code in Rust but were "outcompeted" left and right by coworkers who churn out shiny new things at 10x the speed.
With Python I can get things up and going very quickly with little boilerplate, but I find that I'm often stumbling on edge cases that I have to debug after the fact and that these instances necessarily happen exactly when I'm focused on another task. I also find that packaging for other users is a major headache.
With Rust, the development time is much higher for me, but I appreciate being able to use the type-system to enforce business logic and therefore find that I rarely have to return to debug some issue once I have it going.
It's a tough trade-off for me, because I appreciate the velocity of Python, but Rust likely saves me more time overall.
Sounds like a classic web-dev perspective, my customers hate when we ship broken tools because it ruins their work, new feature velocity be dammned. We love our borrow checker because initially you run at 0.5x velocity but post-25kSLOC you get to run at 2x velocity, which continues to mystify managers worldwide.
People use Python in financial applications, Data Engineering and AI/ML pipelines, infrastructure software etc and the 10x speed can be real.
With Python, testing, good hygiene and a bit of luck you can write core that is maybe 99% reliable. It is very, very hard to get to (100-eps)% for eps < 0.1% or so. Rust seems better suited to that.
Anything else, especially if there isn't a huge premium on speed, meh - Python is almost always sufficient, and not in the way.
I don't need rust specifically. It's just that its memory and thread management really help me to continue what I do in python: focusing on my core business instead of technical stuff.
The less I code the better I feel :-)
Is a coping canard invented by programmers who can't into more powerful programming languages.
By far the slowest language for developing is PHP, it's even worse than plain C in that regard.
I'm sure Rust was the wrong tool for the job in your case but I find this type of get shit done argument unpersuasive in general. It overestimates the value of short-term delivery and underestimates how quickly an investment in doing things "the right way" pays off.
Business owners definitely prefer less bugs, less customer complaints, less support burden, less outages, less headaches. Corner cutting doesn't make economic sense for most businesses and good engineering leadership doesn't have much trouble communicating this up the chain. The only environment where I've seen corner cutting make business sense is turd polishing agencies whose business model involves dumping their mistakes on their clients and running away so the next guy can take the blame.
What business owners actually don't like at all is how long is takes traditional software development to actually solve problems - which then don't really fit the business after wasting a few years of ressources... and the dumping and running away is worse in Java and other compiled software. With Python you can at least read the source in production if the team ran away...
Java (and dotnet, the two big "VM" languages) is somewhat of a strange example for that; JVM bytecode is surprisingly stable and reverse engineering is reasonably easy unless the code was purposely obfuscated - a bad sign on any language anyways.
> I'm sure Rust was the wrong tool for the job in your case but I find this type of get shit done argument unpersuasive in general.
Unless you're working on a fire-and-forget project with a tiny time horizon get shit done arguments are blatantly short-termist.
If you're dealing in areas with short time limits then Python is great, because you can't sell a ticket for a ship that has sailed.
And I've seen "the right way" which, again, depending on the business may result in a well designed product that is not what's actually needed (because people are really bad at defining what they want)
What's brilliant with Python compared to other hacky solutions that it does support test, type hints, version control and other things. It just doesn't force you to work that way. But if you want to write stable, maintainable code, you can do it.
That means you can write your code without types and add them later. Or add tests later once your prototype was been accepted. Or whenever something goes wrong in production, fix it and then write a test against that.
Oh and I totally agree you should certainly try to "do things the right way", if the business allows it.
The only thing that can compete with it for productivity in the science space is R.
For an early stage start up this is almost the only relevant factor for success.
> underestimates how quickly an investment in doing things "the right way" pays off.
What time horizon should a startup optimize delivery for? Minutes, hours, days, weeks? Say you're a startup dev in a maximalist "get shit done now" mindset so you're skipping types, tests, any forethought or planning so you can get the feature of the week done as fast as possible. This makes you faster for one week but slower the week after, and the week after, and the week after that.
Say a seed stage startup aims for 12 months runway to achieve some key outcomes. That's still a marathon. It still doesn't make sense to sprint the first 200 meters.
It's just an application of the Pareto principle. That last 20% of work to make perfect software costs a lot of time. Customers (and by extension, management) do not care how pretty your code is, how perfect your test coverage is (unless your manager is a former developer, then they might have more of an opinion), they care most that you ship it. Bugs are a minor irritation compared to sitting around waiting for functionality they need, as long as you're responsive in fixing the bugs that do come up.
Of course there are at least three dimensions to discuss here: internal quality, external quality and product/feature fit. Lower quality internal code eventually leads to slower future development and higher turnover as no one wants to work with the crappy code base. Lower external quality (i.e. bugs) can lead to customers not liking your product. Interestingly the relationship between internal and external quality is not as direct as one might think. Getting features out the door more quickly (at the expense of other things) can help with product fit. Essentially, like most things, this is an ongoing optimization problem and different approaches are appropriate for different problem domains.
I am tired of having to refactor shiny new things churned out at 10x the speed and that keep breaking in production. These days, if given a choice, I prefer writing them in Rust code, spending more time writing and less time refactoring everything as soon as it breaks or needs to scale.
That's when rewriting the hot path in Go or Rust or Java or C or C++, can pay off and make those skills very valuable to the company. Making contributions to databases, operating systems, queueing systems, interpreters, Kubernetes etc. also fall into that category.
But yeah if you are churning out a MVP for a new business, yeah starting with Python or Ruby or Javascript is a better bet.
(Erlang/Elixir is also an interesting point in the design space, as it's very high level and concise, but also scales better than anything else, although not especially efficient for code executing serially. And Julia offers the concision of Python with much higher performance for numerical computing.)
(Fwiw I use all three, Python professionally.)
Also if I (a programmer) want to write really really fast code I'm probably reaching for tools like tensorflow, numpy, or jax. So there's not much incentive for me to switch to a more efficient language when as near as I can tell the best tooling for dealing with SIMD or weird gpu bullshit seems to be being created for python developers. If you want to write fast code do it in c/rust/whatever, if you want to write really fast code do it in python with <some-ML-library>.
For a very specific definition of the word "fast" at least.
Very limited view which ignores way too many areas.
Really, for almost any X, somebody somewhere has to do X with strict performance requirements (or at very large scale, so better perf == savings)
Most of these python libraries are only fast for relatively large and relatively standard operations in the first place. If you have a lot of small/weird computations, they come with a ton of overhead. I've personally had to write my own fast linear algebra libraries since our hot loop was a sort of modified tropical algebra once.
Rather unfortunately, my current bugbear is that Pytorch is... slow. On the CPU. One of the most common suggestions for people who want stable diffusion to be faster is, wait for it, "Try getting a recent Intel CPU, you'll see a real uplift in performance".
This despite the system only keeping a single CPU core busy. Of course, that's all you can do in Python most of the time.
(You can also use larger batch sizes. But that only partially papers over the issue, and also it uses more GPU memory.)
Whereas before you could get smarter programmers using Python, now because of the exponential growth of Python, the median Python programmer is likely someone with little or no software engineering or computer architecture background who is basically just gluing together a lot of libraries.
The libraries are the killer feature for me.
Anything that doesn't require high performance that is. Is there any 3D game engine for python yet? I guess Godot has gdscript which is 90% python by syntax, but that doesn't quite count I think.
If people write program logic in python it will run at python speeds. Otherwise you're not really writing python, like nobody says some linux native program is bash because it happens to be launched from a bash script.
But that's exactly the strength of Python: it's an interface language. It's meant to make pretty sophisticated things like CUDA easy for everyone.
But that's how every scripting language obtains good-not-just-decent performance. A strong culture of dropping down to C for any halfway-important library is why PHP's so hard to beat in real-world use, speed-wise (whatever its other shortcomings).
For the most part they don't need to know about concurrency, memory efficiency etc, because they're using a library where those issues have been abstracted away.
I think that's what makes python ideal - it's interoperability with other languages and library ecosystem means less technical people can produce good, efficient work without having to take on a whole bunch of the footguns that would come from working directly in a language like c++ or Rust.
> they're using a library where those issues have been abstracted away.
I work in Python, and while libraries like numpy have certainly abstracted away some of those issues, there's still so much performance left of the table because Python is still Python.
Julia, to use the great-great-grand-op's example, seems to also have a reasonably easy C interop (I've never written any Julia, so I'm basing this off skimming the docs, dunno, it might actually be much more of a pain than it looks like here).
https://docs.julialang.org/en/v1/manual/calling-c-and-fortra...
Plus that you now need to write performant (and safe) C code, which (to me) defeats part of the reason to use Python in the first place.
Because if you compare the benefit to the cost of rewrite from py to C and cost of maintaining/updating C code and possible C footguns like manual memory safety, etc - then there is no benefit left
#-of-users * total-time-saved-per-user > time-spent-optimizing
Then it's worth it. You can even multiply by cost of user per time unit and cost of developer per time unit, to see how much money was saved.
Even in cases where its the same person on both sides, it can still work out. There's an xkcd comic about it, even.
And the parent was talking about numpy code which is better than stock python, who knows how far back normal python would send you.
They're comparing numpy (SIMD plus parallelism) with straightforward C code and getting a 2-5x improvement.
But there's two schools of thoughts I've heard from people regarding how to think about these bottlenecks:
1. IO/network is such a bottleneck so it doesn't matter if the rest is not as fast as possible.
2. IO/network is a bottleneck so you have to work extra hard on everything else to make up for it as much as possible.
I tend to fall in the second camp. If you can't work on the data as it's being loaded and have to wait till it's fully loaded, then you need to make sure you process it as quickly as possibly to make up for the time you spend waiting.
Not everything can be pushed into numpy, and you can still be left with lots of loops in python.
[0] https://numba.pydata.org/
To say nothing of adding yet another long dependancy chain to the language (python 3.11 is still not supported even though work started in Aug of last year).
I do wonder if the effort put into making this slow language fast could have been put to better use, such as improving a language with python's ease of use but which was build from the beginning with performance in mind.
Could you expand on this in the context of Rust?
Please don't read this as "rust is difficult and bad", I definitely don't think it is! But its a low level language, and working with it means dealing with complexity that for some tasks just might not be relevant.
Don't forget the footguns of working with developers who do those things. Ask them to do something simple and you get something complex and expensive after months of back and forth about what is wanted. You're likely to a framework for a one off SQL query.
I hear it being said already, "You're using software developers wrong!" Well, maybe software developers shouldn't be so hard to use?
This whole take assumes bad intention on both sides. Nobody's job is easy in this situation. Leadership's job is to set everyone up for success. If things go off the rails and end up with months of back and forth leading to nobody being happy despite good intentions and honest effort, then the problem lies with leadership.
In my opinion, this is the part that Go got mostly right. Concurrency is handled by the runtime, and held behind a very thin veil. As a programmer you don't really need to know about it, but it's there when you need to poke at it directly. Exposing channels as a uniform communication mechanism has still enough footguns to be unpleasant, though.
In an ideal world, I should be able to decorate a [python] variable and behind the scenes the runtime would automatically shovel all writes to it through an implicitly created channel. Instead of me as a coder having to think about it. Reads could still go through directly because they are safe.
If I could have Python syntax and stdlib, with Go's net/http and crypto libraries included, and have concurrency handled transparently in Go-style without having to think about it, that would be pretty close to an all-wishes-come-true systems language. Oh, and "go fmt", "go perf" and "go fuzz" as first-class citizens too.
Someone else in this thread brought up the idea of immutable data structures as a default. I wouldn't mind that. Python used to have frozenset (technically it still does but I haven't seen a performance difference for a while), so extending the idea of freeze()/unfreeze() to all data types certainly has appeal.
This myth wasn't even true many years ago, it certainly isn't true today. You can build a mobile app, game, distributed systems, OS, GUI, Web frontend, "realtime" systems, etc in Python, but it is a weak choice for most of those things (and many others) let alone the second best option.
The saying means that for any given problem, there is a better choice, but second best is the language you know which has all of the tools to get the job done, so the answer is probably just a bunch of pip installs, imports, and glue code.
It’s kind of like “the best camera is the one you have with you” — it’s a play on the differing definitions of “best” to highlight the value of feasibility over technical perfection.
What worries me, though, is that the features that make Python quite good at prototyping make it rather bad at auditing for safety and security. And we live in a world in which production code is prototyping code, which means that Python code that should have remained a quick experiment – and more often than not, written by people who are not that good at Python or don't care about code quality – ends up powering safety/security-critical infrastructures. Cue in the thousands of developer-hours debugging or attempting to scale code that is hostile to the task.
I would claim that the same applies to JavaScript/Node, btw.
However, I hope that these C applications are written by people who are really good at C. I know that some of these Python applications are written by people who discovered the language as they deployed into production.
If the masterful C developer still insists on using a language that has so many footguns and a weird culture of developers pretending that they’re more capable than they are, then their C mastery could very well’ve not been worth much against someone throwing something together in Python, which will at the very least immediately bypass the vast majority of vulnerabilities found in C code. Plus, my experience with such software is that the sort of higher level vulnerabilities that you’d still see in Python code aren’t ones that the C developer has necessarily dealt with.
How could we check?
But I fear that the same folks that decried the use of excel by "the masses" are now just as horrified by the widespread usage of Python! :-)
What's an example that makes it bad? Is it a case of the wrong tool for the job?
For example I understand that garbage collection languages shouldn't be used with real time systems like flight controllers.
Python is designed to be highly dynamic and to allow more code paths to be taken at runtime, through interpreting and reacting to the live data - "late binding" in the lingo, as opposed to the "early binding" of a Rust or Haskell, where you specify as much as you can up front and have the compiler test that specification at build time. Late binding creates an explosion of potential complexity and catastrophic failures because it tends to kick the can down the road - the program fails in one place, but the bug shows up somewhere else because the interpreter is very permissive and assumes what you meant was whatever allows the program to continue running, even if it leads to a crash or bad output later.
Late binding is very useful - we need to assume some of it to have a live, interactive system instead of a punchcard batch process. And writing text and drawing pictures is "late binding" in the sense of the information being parsed by your eyes rather than a machine. But late binding also creates a large surface area where "anything can happen" and you don't know if you're staying in your specification or not.
This is very useful when you're writing unit tests or when you want to monkey-patch a behavior and don't have time for the refactoring that this would deserve.
On the other hand, this means that a module or class, no matter how well tested and documented and annotated with types, could be entirely broken because another piece of code is monkey-patching that class, possibly from another library.
Is it the case? Probably not. But how can you be sure?
Another (related) example: PyTorch. Extremely useful library, as we have all witnessed for a few years. But that model you just downloaded (dynamically?) from Hugging Face (or anywhere else) can actually run arbitrary code, possibly monkey-patching your classes (see above).
Is it the case? Probably not. But how can you be sure?
Cue in supply chain attacks.
That's what I mean by auditing for safety and security. With Python, you can get quite quickly to the result you're aiming for, or something close. But it's really, really, really hard to be sure that your code is actually safe and secure.
And while I believe that Python is an excellent tool for many tasks, I am also something of an expert in safety, with some experience in security, and I consider that Python is a risky foundation to develop any safety- or security-critical application or service.
Immutability would be the default, but mutability would be allowed, marked in some concise way so that it was easy to calculate things using imperative-style loops. Pervasive use of immutable instances would make it impossible for libraries to rely on mutating objects a la SQLAlchemy.
The language would be statically type-checked, with optional type annotations and magic support for duck typing (magic because I don't know how that would work.) The type system would prioritize helpful, legible feedback, and it would not support powerful type-level programming, to keep the ecosystem accessible to beginners.
It would still have a REPL, but not everything allowed in the REPL would be allowed when running code from a file.
There would be a strong module system that deterred libraries from relying on global state.
Support for at least one fairly accessible concurrency paradigm would be built in.
I suspect that the error system would be exception-based, so that beginners and busy people could write happy path code without being nagged to handle error values and without worrying that errors could be invisibly suppressed, but there might be another way.
If someone who's not familiar with programming runs into an error like "why can't I change the value of X" that might take them multiple hours to figure out, or they may never figure it out. Even if the error message is clear, total beginners often just don't know how to read them and use them.
They provide longer term advantages once your program becomes larger but the short term advantages are more important as a scripting language imo
I don't think mutability by default is necessary for beginners. They just need obvious ways of getting things done. There are two places beginners use mutability a lot. The first is gradual transformation of a value:
This is easily handled by using a different name for each value. The second is in loops: I think in a lot of cases beginners will have no problem using a map or list comprehension idiom if they've seen examples: But for cases where the immutable idiom is a bit tricker (like a complicated fold) they could use a mutable variable using the mutability marker I mentioned. Let's make the mutability marker @ since it tells you that the value can be different "at" different times, and let's require it everywhere the variable is used: Voila. The important thing is not to mandate immutability, but to ensure that mutability is the exception, and immutability the norm. That ensures that library writers won't assume mutability and rely on it (cough SQLAlchemy cough), and the language will provide good ergonomic support for immutability.It's a common claim that immutability only pays off in larger programs, but I think the mental tax of mutability starts pretty immediately for beginners. We're just used to it. Consider this example:
Beginners shouldn't have to constantly wrestle with the difference between value semantics and reference semantics! This is the simplest possible example, and it's already a mind-bender for beginners. In slightly more complicated guises, it even trips up professionals programmers. I inherited a Jupyter notebook from a poor data scientist who printed out the same expression over and over again in different places in the notebook trying to pinpoint where and why the value changed. (Lesson learned: never try to use application code in a data science calculation... lol.) Reserving mutability for special cases protects beginners from wrestling with strange behavior from mistakes like these.Julia is both dynamic and fast. It doesn’t solve all issues but uniquely solves the problem of needing 2 languages if you want flexibility and performance.
Coupled with the dynamic typing and mutability by default, it guarantees Python programs won't scale, relegating the language to the role of a scratchpad for rough drafts and one off scripts, a toy beginner's language.
None of those applications are toys. I have no idea where your misperception is coming from.
All sorts of problems would simply have not existed if the proper language was used from the beginning, as opposed to the one where anyone can hack most easily.
It’s already what you get with python and mypy. Using typing.Protocol or Unions.
Regular sofware development is a lot like that as well. But you can kind of get around that by having Python as the "prototyping language", and anything that's proven to be useful gets converted to a language that's more useful for production.
Do we agree that this somewhat orthogonal to what I'm writing, though?
ES modules, fat arrow expressions, and all the other nice new syntax and library features have made the language so more pleasant to use. In many ways the ergonomics of TypeScript in particular are far superior to Python now, and I find that really surprising.
I haven't yet found a replacement for Django (particularly the ORM/Admin/Forms combination) it is just so incredibly productive to use. So, I don't think I will be moving off it, but it's certainly not the growth language for me anymore.
This, plus Typescript's superior type system makes it very tempting for application development.
However I'll probably wait until there's a really good Jupyter Notebook equivalent...
https://github.com/n-riesco/ijavascript
I used this for interactive debugging AWS API calls using the AWS Javascript SDK. It worked great!
Closest thing to this is Docker nowadays. It's so annoying. npm just works.
I still love Python. Just feel like I'm cheating on it with this new younger model...
Take a look here https://adonisjs.com
- Particular support for web frontends or backends, both very broad categories.
- JS concurrency is easier to deal with. Focused entirely on promises with the nice async/await syntax on top, unlike Python which slapped on too many different ways to do this.
- By far easier package management and imports. Python's is so annoying that any project you download is gonna have you spin up a Docker container for it.
- ES6 added a lot of array/etc managing that JS was lacking before.
- Freeform objects with {key: value} syntax are convenient, despite maybe seeming weird at first. Python OOP somehow got really complicated over the years.
- Inline functions (I use fat-arrow but regular way is also fine). I never got why Python, despite being common in function-oriented programming, didn't let you do an inline def.
- Curly braces. Indentation shouldn't affect logic flow.
- No Python3 vs 2 drama.
Specific use cases have been Express backends with node-postgres and lots of SQL, and React (or RN) frontends.
Most of the provided reasons behind Python's popularity are true also for other languages - portable, open source, productive, big community. This can be also said about PHP, Ruby, or Perl back in 2000s. Why isn't Perl as popular as Python?
I don't think it's all about readability or productivity, but about tools that were built over the last 30 years that have been used in academia and now with the boom in ML/AI/Data Science, they made Python an obvious choice to use for the new generation of tools and applications.
Imagine that the boom in ML/AI didn't happen - would Python be #1 language right now?
Python was friendlier for beginners than Ruby the first time I took a real stab at learning to code during a CNY holiday in 2008, but it wasn’t about the language itself. Ruby was harder then because many of the popular libraries and many of the tutorials were written by people who considered Windows support as an afterthought. It’s hard to express how frustrating it was to have my vacation days ticking down, hitting issues in one tutorial after another and having people suggest I install linux on a VM (a process where I hit still more snags).
People learning Python and PHP didn’t hit that hurdle. I ended up learning Flash on my Asus laptop a couple of years later and getting my start that way and not coming back to Ruby until six years later when I was a much more experienced dev.
I don't think there is a single reason, but it sure didn't help that the community self-destructed by trying to make an entirely new language after version 5 and still call it Perl. It took a lot of years to resolve that nonsense, and in the meantime many people moved on.
It also does not help that Perl is a creative language, useful but very much open to many different interpretations. Hiring a perl guy and expecting them to read someone else's code is a crapshoot. The upside to Python's strong cultural opinions on coding style makes it easier for one developer to pick up someone else's code.
> Imagine that the boom in ML/AI didn't happen - would Python be #1 language right now?
Probably not. But it wouldn't be perl, either. Javascript most likely. But the core usage of python for scripting was never predicated on ML popularity, so it would still be a pretty commonly used language. and javascript has many annoying warts too, so I think plenty of people would still choose to write django apps instead of node, whether ML existed or not.
One of the few programming language jokes I've liked enough to repeat is "Python in Perl for people who can't bring themselves to write Perl code"
Perl was significantly more popular at one point, but it slowly lost traction while Python gradually gained traction over the years.
Better ecosystem for numeric computing is definitely a big reason for the success of Python, by the question is why Python gained a foothold in that niche in the first place. It think it is because Python is just a lot more accessible to people with different backgrounds. Perl really grew out of shell scripting as a supercharged alternative to Bash and Awk, but retaining many of the quirks for familiarity. Python on the other hand grew out of research in teaching programming to beginners.
Someone else made this remark once, when the indroduction of type annotations was discussed as the python community discovering the benefits of static typing.
> Using Python for and artificial intelligence
for what and AI?
https://mesonbuild.com/index.html
Getting python bootstrapped is less easy than getting make bootstrapped, plus python isn't a GNU tool so I don't see them adding a dependency on it to all their packages.
GNU Make is really far more sophisticated than people give it credit for and it's not that easy to replace it.
> GNU Make is really far more sophisticated than people give it credit for and it's not that easy to replace it.
Since its turing complete, it should be able to do everything make can. For example, to compile a C project involves something like:
1. grab all the C files inside `src/`. 2. join the filenames with whitespace delimiter 3. execute `gcc <join result>`
It can also be used to figure out the OS, where to find dependencies and what order compilation should happen or whatever.
But why doesn't this happen?
Make doesn't do any of the things you mention - it just works out what targets need to be rebuilt and the code to do it is in the rules that you have to write in a shell language like bash.
I admit you can use builtin implicit rules for C and get away with that up to a point but only a fairly low point.
python is far harder to use than bash when it comes to running processes and doing things with them and doing the file-system manipulation that is usually wanted when building.
There are modules that make it easier but so far as I've seen it's a crap choice plus python takes a long time to start up so there is a cost to running a "clean" environment for every build step but OTOH if you use the same interpreter for the whole build you can introduce all sorts of ordering problems when a build runs on a different machine and doesn't work because it's executing build tasks in a different order.
If you really want to build by "writing a script" (groan - because that's the age old horrible solution) why bother with python?
Can easily be solved by having libraries that does. Which is far better solution that hacking/chaining 5 different obscure unix tools to do string manipulation.
>you can introduce all sorts of ordering problems when a build runs on a different machine and doesn't work because it's executing build tasks in a different order.
Surely it is better than make? Even if make is "better" because it "just works". This is basically Hyrum's Rule waiting to be unleashed.
>If you really want to build by "writing a script" (groan - because that's the age old horrible solution) why bother with python?
Because I personally cannot decipher make and I think this is the experience of many developers as well. And how many make derivations are there? Its basically an arcane spell that needs to be chanted with every project (configure -> make -> make install).
It's not arcane - you just don't want to learn it. You can write your own system and rediscover all the issues and resolve them if you like.
If you started from understanding make, though, you could do a better job than it has done because you don't have to be compatible.
The point of communication is to express something in a way that the listener understands it. Simply expressing something that is comfortable or normal to the speaker is simply not useful for either party of the goal is communication.
> They didn't pay too much attention to white spaces and tabs, probably because they are "invisible"
can get so much worse than people imagine, when the language doesn't enforce it. It was that experience that made me lean towards python being a good introductory language, to help people get used to correctly indenting code in general.
https://github.com/ziglang/zig/issues/35
I had a BMP file which I wanted to transpose and turn into raw RGB 5-8-5 values. Using the Python Image Library made things absurdly easy.
That's Python's advantage in a nutshell. Things are just so absurdly easy.
If there was an equally good C++ image library available then it would have been equally simple in C++.
Of course the thing is that, in general, there may well NOT have been such a C++ library, and this is what makes languages such as Java and Python popular - because of the breadth of libraries available.
For C++ image manipulation I've found libvips to be decent, but the point still stands that for a given problem there may not be decent libraries available.
This week our Director ordered a total rewrite of two years of work in Python. His rationale: it's what everyone else uses in this space. No reason specific to our use case, just simply to follow the herd. I realise that a large community translates into easy hiring and rich ecosystems, but I despise the mentality as it promotes a monoculture.
There was a time when all you had to learn was Java or C++ or (language or tool here). Somehow, this time when we standardize, it will be different.
Can you say the same about your solution?
WTF? Unless your system is originally written in a proprietary language that literally no one outside your company knows, I'll say it's a good sign that you need to change team (or change job). Don't work under a director like that.
Better luck with the next one!
It's really hard to trace a Python bug based on codebase, due to its stricly bugly indentation sensitive for space/tabs.
I rather work on a JS codebase than a python codebase.
But i consider Python coders genius, because they can go deep into rabbit hole really well.
That may only prove that you are not proficient in either.
This is a strange article. It's got the talking point about Python that we were hearing about 10 years ago - "tired of those pesky curly brackets in Java, try this new language you might not have heard of: Python!". Who reading the GitHub blog has not heard of Python?
Also, that snippet used in the "What is Python commonly used for" section is strange:
It's overly verbosely written (especially given the example just above about how you don't need main function in Python) and refers just to an insider joke/Easter egg. I can't see that it's going to convince anyone to try Python, only make them feel that they're on the outside of a joke.It then ends with what seems like it might have been the point of the article, an advert for Copilot. It seems the way to get started writing Python is to write a short comment and then spam <TAB> and let the AI auto-complete your project.
(Also, and perhaps less importantly, looking at the author's GitHub profile I can't see a single instance of Python there. Though I'm not doing a deep-dive as that feel overly picky and there's plenty of private contributions that could well be Python.)
That was a talking point closer to 20 years ago, at this point.
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Hello, world!")?
https://stackoverflow.com/questions/6182964/why-is-parenthes...
https://github.com/python/cpython/blob/main/Lib/antigravity....
In the end it is just an ad for Github products and not worthy of being on HN frontpage.
The GitHub blog became a strange place recently(-ish). It went from a factual blog describing fancy new GitHub features and interesting technical stuff (as it was a decade ago) to mostly a place full of incoherent marketing fluff like this post (with some real technical content interspersed).
IMO, that example is there to show that there's less required boilerplate required in python compared to java when doing the same thing. And, in particular, none of that boilerplate really matters to what you want to do - print hello world - emphasizing the point of python being simple.