Pydantic is unreasonably slow for big datastructures, though. For simple performance tests with simple requests/response it's probably fast. But with lots of the data marshalling happening in python some things are really slow in real life use cases.
Yeah i have a lot of things to do, to minimize this types of things. But Caching Tools that never cross GIL will boost a lot of performance in real life scenarios too. See https://github.com/cirospaciari/socketify.py/issues/60
I plan to create an native Redis cache and memcache too.
typedload is faster and written in pure python (unlike pydantic).
I know they are rewriting it in rust… but I don't know what will come out of it. After all my pure python implementation already manages to beat pydantic's binary, and when unions are in use, also apischema's binary.
The perf part of the tests just seems to be a microbenchmark for seeing how fast the various frameworks can parse a 30000x300 dict of strings representing numbers [1].
If that is all one's application does, and can use your library in their organization/team, that's great. However a 2-3x performance boost for the parsing stage for a use case like an API call might not matter when that could be overshadowed by validation and/or upstream API calls. A realistic app would likely use a validation library like Pydantic's [2] to throw a custom typed-error that can be processed, e.g., localization, before returning it downstream.
> a microbenchmark for seeing how fast the various frameworks can parse a 30000x300 dict of strings representing numbers [1].
Did you somehow miss all the other tests, even thought they are higher on the page? The important test is shown first: loading objects.
I'm not trying to benchmark my wifi or my disk. The IO time is not included there on purpose. Of course a bigger application that does other things wouldn't see this huge difference in performance. But I'm testing the performance of a library here.
typedload can use validators, but since it doesn't reimplement attrs/dataclass, I had no interest in testing those… it'd be a race between libraries I didn't write.
typedload's exceptions contain enough detail to tell the end user what went wrong.
when encountering errors in lists, typedload slows down due to keeping compatibility with python 3.7. However in my test with loading objects, despite the slowdown it remains faster than pydantic.
I read all the perftests in the repo. I think they nearly all parse a structure that contains a repetition of the same or similar thing a couple hundred thousand times times and the timing function returns the min and max of 5 attempts. I just picked one example for posting.
Not a Python expert, but could the Pydantic tests be possibly not realistic and/or misleading because they are using kwargs in __init__ [1] to parse the object instead of calling the parse_obj class method [2]? According to some PEPs [3], isn't Python creating a new dictionary for that parameter which would be included in the timing? That would be unfortunate if that accounted for the difference.
Something else I think about is if a performance test doesn't produce a side effect that is checked, a smart compiler or runtime could optimize the whole benchmark away. Or too easy for the CPU to do branch prediction, etc. I think I recall that happening to me in Java in the past, but probably not happened here in Python.
The kwargs thing is true… but I didn't design the API of pydantic. And it only happens on the small top level dictionary, not on all of them (unless it internally does it all the time).
Java has JIT, I agree that in that case keeping the output value is important. cpython isn't that smart. I honestly never tried to benchmark using pypy. I guess it could be interesting to try that.
I don't really understand this criticism nor why it's "clickbait". I have 12 "unpure" python packages in production. If my app ever needs WSGI, I'll have 13.
The reason this is clickbait is because you can wrap libuv in some other language and get the same results. So what’s the purpose of this article? That a language can wrap libuv is not a new or interesting idea.
Why do people only get upset about this sort of thing when it comes to web servers? I don't see the same nitpicks when its a BLAS library, or when it uses a CUDA kernel under the hood. Obviously python is a slow language and needs the C implementation to be fast. You could say the same thing for the vast majority of the python code base.
Probably should add that it's faster with PyPy as it might not be obvious on how an interpreted language like Python can beat an AOT compiled language like Go.
Yeah, I'm converting to HPy and it probably will be faster in CPython too, but like you said AOT is hard to beet without JIT at least. But with CPython is faster than Golang Gin :D
Others point out that calling it a Python framework might be disingenuous, but only if your goal is to compare programming languages.
If you're a web developer, trying to pick a framework for websockets, you probably don't care that the Python framework is a libuv wrapper, while to Go framework is native.
So well done, I guess? I am really happy to see library authors taking performance seriously!
EDIT: This is assuming the benchmark is actually fair. I haven't looked at it, but it's not uncommon for benchmarks to be comparing apples and oranges.
Python framework because is an framework for python, like most performance focused frameworks for python, this uses a lot of native code.
uvicorn uses uvloop that is an wrapper for libuv.
Web developer dont care if the framework is written in native or it is an wrapper, developers just want something that works in a nice way :D
Go developers might though. A go library that requires native is a pretty big reason to avoid. Similarly a headline “c framework that uses libuv is faster than go fiber” wouldn’t cause an eye blink.
So the concern is the title you used to get to the front page not the actual implementation.
I see, but like i said any performance focused Python framework are native (C/C++), so i will never be able to compare any Python framework with Golang framework ever.
The title seems accurate and appropriate to me. If I were selecting a framework for performance I might chose this framework, which I would use through Python, rather than a Go framework that I would program in Go. It's a framework, you use it with Python, and the performance is good (apparently, I didn't check).
> EDIT: This is assuming the benchmark is actually fair. I haven't looked at it, but it's not uncommon for benchmarks to be comparing apples and oranges.
well, a real world test would be both servers parsing 100kb json payload in each message
that would be an JSON benchmark not right? raw throughput is more useful to test WebServers, but JSON performance is also important, but i think is another benchmark. PyPy do very well in JSON too take a look https://www.techempower.com/benchmarks/#section=test&runid=1...
> So well done, I guess? I am really happy to see library authors taking performance seriously!
> EDIT: This is assuming the benchmark is actually fair. I haven't looked at it, but it's not uncommon for benchmarks to be comparing apples and oranges.
What is the point of this negativity? Why are you implying that the benchmarks might not be fair because you "haven't looked at it"?
I actually read their comment as positive, with a small caveat about the benchmarks. It's pretty common to see benchmarks here, where the load is very artificial and hence doesn't translate to real world usage, for instance. So I think it's an okay disclaimer.
Good point! However, you bring in a common misconception I'm fighting with within my company for a long time. Python is not an interpreted language. There is no such thing as an "interpreted language", a language is just a set or rules and keywords. Everything you can fit into Backus–Naur form is already a language even if it doesn't have any implementation nor compiler neither interpreter.
When people refer to a language, most often they are using it as a synecdoche to refer to the whole language ecosystem and not just the the formal language definition.
If you look at the Python ecosystem, it is most definitely interpreted.
Did you look up how to pronounce it? I was mispronouncing it for like a year after seeing “Synecdoche New York” (it’s kinda pretentious, not for everyone). My coworker laughed at me when I said the name to him. I’d only ever seen it written, and did not look at a pronunciation chart. Wah wah.
Just in case anyone else is getting it wrong like I was …
Personally, I often find myself saying things like "the <lang> ecosystem" to avoid this.
I think it is fair to have a discussion about terminology and arguing with the actual meaning of terms. When there is a technical difference, it would be good to acknowledge that difference between terms. It does not hurt anyone to do that. One can still say something like: "When we communicate I will use the word x for meaning y, because it is shorter and more convenient."
For all practical purposes, a programming language is defined by it's general real world implementation, and not it's specification. Maybe not so much for computer scientists, but for programmers, Python is interpreted with some JIT alternatives and C++ is compiled.
This is false. Making a new implementation is a practical purpose. Any new implementation will have to work with the spec, instead of idiosyncracies of other implementations. Ergo there are practical purposes which go by how the language is defined in its specification.
Some languages have more of a proper spec than others, of course. If we look at for example Scheme dialects, then they often change their implementation for better conformance with the various rnrs standards. New Scheme dialects often early on try to conform very much to the standard to avoid issues later on.
> Making a new implementation is a practical purpose.
Not yet it isn't. At least not for Python. I mean there are plenty of alternative implementations of Python but they aren't practical because they don't work with CPython modules (as far as I know anyway) so approximately nobody uses them. The de facto spec is "what CPython does".
PyPy work well with CPython modules but the performance is not that great, with CFFI performance is really good in PyPY, HPy project will change this https://hpyproject.org/
Last I checked, Pypy wasn’t compatible with many important packages in the Python ecosystem. As of a couple years ago, there was no trustworthy, supported package for talking to a Postgres database with Pypy because psycopg2 wasn’t supported.
postgresql can work with psycopg2 but is not the same performance as psycopg2cffi , i will integrate some databases to PyPy, and also HPy project will bring any Cython library to PyPy with basically the same performance or better.
I heard that very argument about GCC. C++ without __attribute__ is not practical for embedded development therefore C++ for PLCs is what GCC does. And yet, the general consensus now is that C++ is ISO/IEC 14882, and what GCC does is just an implementation.
With Python, you don't even have to go "alternative" to use LLVM compiler infrastructure. You can use CPython + Numba and compile your kernels for both CPU and GPGPU on the go. Which is very practical.
Which is ironic, to compile a CUDA kernel in C++, you would have to use an alternative NVidia implementation of the language.
A theoretical difference without practical impact; nobody uses these - sure you can implement a C++ interpreter in hardware if one so wishes, but if the industry doesn't use it, what's the good of it ?
By this they mean the real-world-implementation that is in general use, which in this case is interpreted.
Yes, Python can be compiled and C++ can be interpreted. 99% of the time, they're not though.
When somebody says X language is interpreted, they mean that the most common and widely implementation is an interpreter. That's a useful bit of information to know about a language. "Correcting" them and saying that all languages can be interpreted OR compiled, doesn't really add anything of value to the discussion, other than to label yourself as a pedant.
Counterexample. Numba is a Python compiler for heterogeneous computing. We adopted it last year and stopped rewriting code in C++ to run it on GPGPU.
On one hand, the conception from the 90s based on the most common Python implementation, on the other - a compiler that saves us tons of money. Which one adds value again?
I literally posted the links to the AOT Python compiler and to the C++ interpreter.
That would be important in high-school debate club. In the real world, most people don't care about obscure edge cases and pedantry. In the industry vernacular, Python is an "interpreted language" and C++ is a "compiled language". Arguing against that in anything other than a highly (highly!) specialized context is just lighting candles in the wind.
Numba is a Python compiler for heterogeneous programming. It builds kernels with the same back-end NVCC does. With the equivalent AST, the kernel code it also equivalent.
We adopted it last year, and we don't rewrite code in C++ for GPGPU anymore. We save tons of effort on rewriting and even more on support.
That’s the old way of language design; used to be that to create a “programming language”, you’d write a spec and call it a day. That just doesn’t fly anymore. These days, you actually have to implement a parser and compiler, and not only that, you have to write the entire standard library as well, along with a toolchain complete with debugger, profiler, package manager, language server implementation, etc.
All this is to say that a programming language these days is really the sum of its parts, not a spec.
Even if there isn't literally such a thing as an "interpreted language" there are absolutely design choices made in Python that you wouldn't make if you wanted it to be fast at runtime. To be fair, this is also true of JavaScript, but in Python the choices were literally intentional, as simplicity of CPython and elegance of the language were long valued over performance.
OTOH, interpreting C++ is actually a pain in the ass due to design choices it makes specifically regarding the semantics of translation units. You can see similar pain when trying to say, make a Go REPL: it's just not tuned for this.
It is of course technically possible to compile almost anything ahead-of-time. But, if you effectively wind up having to ship an interpreter or JIT into the resulting binary to run some code anyways, it's almost for naught. Both Python and JavaScript have eval. Python also has several other cases like Pickle where this can be an issue. PyPy being a JIT makes a lot of sense because it wants to be a drop-in replacement, and that makes the most sense for a language with these constraints. Codon can do AOT, but for practical reasons it is not nearly a drop-in replacement for CPython, just like the other Python AOT implementations.
A practical Python AOT is not compatible with loads of things that people associate with Python, like Django. PyPy is compatible with more, but even it is far from a drop-in today, and that is a problem.
> A language is just a set or rules and keywords. Everything you can fit into Backus–Naur form is already a language even if it doesn't have any implementation nor compiler neither interpreter.
This is objectively true. However, in practice, there are a finite number of available toolchains for any given language that exist today, and creating new ones is a non-trivial endeavor, especially production-quality ones. A language is just a set of rules and keywords. However, when people use Python and write Python, they are not merely writing Python to fit into those keywords and rules. They're writing Python to be executed and solve a problem, typically using CPython. Python is a language, but it's also an ecosystem.
In the same token, if something like Codon doesn't even support all of the things you can fit into Backus-Naur form about Python as it is in CPython and PyPy, can it even be called Python in this sterile technical sense?
>> However, you bring in a common misconception I'm fighting with within my company for a long time. Python is not an interpreted language.
My advice to you: Stop fighting that "misconception". See the other responses to back up the idea that Python is interpreted. The existence of a Python compiler is also not relevant here, as the linked project does not list that as a supported Python variant.
Let's assume that by some technicality you are not wrong. The "misconception" is so common that it is generally a fact in peoples mind. Rather than convince them, you are destroying your own credibility and probably making a fool of yourself in their eyes. And remember, I prefaced this with you not being wrong. It's worse for you if you are wrong. Just stop for your own good.
While I also find arguing for the sake of arguing tiresome, the comment your responding to makes a good point. The linked project beats Go fibers using PyPy, which is JIT compiled, where hot codepaths are not interpreted. A language is a set of rules (hopefully described in a specification...) - you can have interpreted C++, or compiled Python. The popular implementations are just that: implementations, of which others exist.
> Rather than convince them, you are destroying your own credibility and probably making a fool of yourself in their eyes.
I'd understand where you were coming from had the parent comment not explained what they meant and given examples.
Sorry, I didn't make it clear before. I'm winning. My credibility is well protected by our AWS bills.
Last year we adopted Numba for GPGPU programming. It is a Python compiler with LLVM back-end and full CUDA support. It generates the same code that NVCC (CUDA compiler for C++) does only it compiles Python code and not a C++ .cu dialect.
No one in our organization writes heterogeneous code in C++ anymore.
> There is no such thing as an "interpreted language", a language is just a set or rules and keywords.
Sure there is. Even if it is technically possible to write an interpreter for a "compiled" language, or a compiler for an "interpreted" language, language design really pushes you towards one or the other. "Technically possible" can also represent a very real challenge, as that set of rules and keywords might make ahead-of-time compilation either impossible or pointless.
E.g. any language that exposes `eval` with an arbitrary argument will still need to ship an interpreter as part of the runtime even if you compile some of the code ahead of time. Monkey-patching is common in Python and Ruby, and is pretty hostile to compilation. Codon has several documented limitations around this. Perl 5 famously can't be statically parsed without evaluating the code as you go, which completely defeats any attempt at compiling it.
Inversely, You can write an interpreter for C++, but features like `consteval` and `constexpr` are meaningless without ahead-of-time compilation.
> Inversely, You can write an interpreter for C++, but features like `consteval` and `constexpr` are meaningless without ahead-of-time compilation.
No they're not, `constexpr` and `consteval` indicate where you can use an expression, not when in the compilation/interpretation pipeline they need to be evaluated. The spec does not say that C++ has to be compiled, nor does the spec say "A compiler must execute `constexpr` expressions prior to entering `main()`".
If you think this is just being pedantic, well, that's kind of the point of a spec.
Language is used to communicate. And not needing to be absurdly verbose in everything that one says, is a virtue and a need for efficient transmission of concepts and idea.
So when someone says "Python is an interpreted language" they are pleading to an implicit preexisting shared knowledge about the nature of the most commonly found usage of the concept.
Granted, when this shared knowledge doesn't match, that's where misunderstandings can happen, but that's the nature of human language. In this case I'd be hard pressed to think that any misunderstanding could have been possible for the immense majority of people reading this.
There are language features of dynamic languages like python that prevent them from being compiled to machine code. That is to say, the compiler can’t know what assembly to output from the code itself, it also has to know some type information which isn’t available until runtime. I would encourage you not to fight the classification of interpreted language as your primary argument. You will likely lose credibility with the listener before you have a chance to make any good points. You could for instance say “just because the reference implementation is slow doesn’t mean there is no hope for the language becoming fast”. There is some truth to the argument. JavaScript has become a reasonably fast language with all the engineering effort spent on engines for it. But I would also spend some time learning about what makes dynamic languages hard to compile. I think you will find that some features of the language must be dropped to achieve fully aot compilation. The classic example would be eval() which invokes an interpreter from inside the language, but there are usually other features.
Fiber is built on FastHTTP. If you know Go top "frameworks" performance characteristics then you would know that you are actually comparing HTTP implementations - ala whether framework is based on FastHTTP or Go standard library HTTP implementation. Framework own % there is for the top 5 almost non-existent.
I would assume this is same case where - what ever Socketify is based on (pythons transport layer), is compared to FastHTTP. and mention of Fiber is click-bate.
I looked at the tests. There is nothing significant going on when creating response in http server for example. Just spit "hello world". So it appears to be a test of C++ uWebSockets and uSockets libraries vs for example native Go implementation. Do something serious in request handler using Python and then see what happens.
Actually uWebSockets and uSockets will perform better than this (2x at least in my local tests), I need to do a lot of copying, instancing and crossing Python GIL.
Yeah this test basically shows that Python backed by uWS is crazy fast, but is not an direct comparison to uWS to Go.
This test is just an troughput test, with is very useful to measure raw performance.
More tools like caching tools, a better database client etc is need to construct an complete scenario, and i'm working on it! (Maybe in 1 week or 2 weeks will be done)
>"Python backed by uWS is crazy fast, but is not an direct comparison to uWS to Go."
Correction. It shows that uWS is crazy fast. Sure if python us used as a thin glue between something like "select username from user" and uWS or send file over uWS then the result is fast. Do something meaningful in Python itself an it will be slow.
This is not to diminish your work. Python is mostly glue anyways so in average the end result should be quite ok
This is kind of cool, but it all makes sense if your app is a very simple CRUD that spends 99% of time just passing json around. I have this feeling that when you add some more logic there you’ll start noticing that you’re not in a compiled language any more. Your requests will get slower and your memory usage will explode.
Saying that Python is faster than Go with this as a proof looks like overreaching for
Me. It only proves that wrapping C code in Python is fast. It’s an achievement, sure, but your app probably won’t be faster when you add thousands of Python lines between request and response.
Claiming python has blazing speed while literally being CPython is insanity. This is like saying "Im a natural body builder who only uses Test, HGH, and Dianabol". Great drop the natural part, similarily just advertise speed as CPython.
> Claiming python has blazing speed while literally being CPython is insanity
And it is not claimed anywhere. Besides, cpython is not really know to be fast, as it is an interpreter? Why would its implementation language matter much?
Technically (not sure what you mean with this but with the context I will use it too as it seems to match with what I want to say) you could never talk about Python (the most common and original implementation) without making reference to C then.
And if you are not making reference to a specific implementation then the comparison does not make any sense.
Isn't there a difference between interpreting Python code with CPython (which I would call "Python") and running compiled C from Python (which I would call "C")?
Then it kind of make sense to talk about " pure Python" (the interpreted one) and "wrapped C", doesn't it?
> you could never talk about Python (the most common and original implementation) without making reference to C then.
Good. Because otherwise people would be mislead to think that Python itself is fast. The Python plus C combo has opposite tradeoffs in many dimensions.
But that isn't the question. The question is what tool delivers the best performance. If I want to build some end-use-case software, should I adopt "Python Framework X" or "Golang Framework Y" to build this system? One consideration amongst many is performance. If you assume that the overall system with Go will be faster than the overall system with Python (and c modules) then you'd be wrong in this case. Other factors would inform your overall decision, but having more knowlege about tradeoffs enables one to make better engineering decisions.
I'm reading some arguments and counter-arguments in this thread, and in my opinion it kinda boils down to what point of view you're having.
If you look at it as a framework that minimises the networking overhead, then fine, it's an interesting piece of software.
If on the other hand you look at it like a "fast" web framework then things start to change and the discussion gets a bit more complicated.
So for example, you look at the source code of the applications being benchmarked (example: https://github.com/cirospaciari/socketify.py/blob/main/bench...) and you immediately see it's simply returning the string "hello world"). Which means that it's almost 100% of the time running in the fast path / best case.
My guess is that as soon as you start doing any kind of computation in the request handler in normal non-super-optimized python (trival example: validating some headers and/or checking some signatures as you would do with jwt tokens for example) then the python-vs-golang gap will start to go back to favour golang.
And then again, it boils down to what you're doing: anything io-intensive might benefit from the unetworking/uwebsocket beneath, anything cpu-intensive will benefit from the golang compiler producing native executable code.
Actually the benchmark is on https://github.com/TechEmpower/FrameworkBenchmarks with is basically this with more headers, but you are right, is not enought, i used TechEmPower because is very popular. I have some issues (new features comming) open to create an better JWT token support, database and much more, i will post these in the future!
sure it's faster.. but go gin and fiber are really slow. they are literally 90% slower than the fast go frameworks. Not that im a go fanboy, but to put things into context here, the actual context here is: python framework 5% faster than fiber and 85% slower than gnet/silverlining/gearbox.
For sure have room to improve but is very competitive, i will try to archive the 7 mi inthe future, this is a very young framework, i will work harder!
in the case of socketify is CPython and PyPy running the same code, but both are really fast, in the case of fiber is without prefork and with prefork variants.
How is fiber slow? Can you please provide some source? According to the techempower fortune branchmark, it's the 3rd (prefork) and 6th (normal) fastest go framework and even in the all-language list it's 24th and 34th.
Learn as you develop. I've been there, I've felt like I'm constantly switching techs because my app should have the most bleeding edge stuff.. And in the end I always felt overwelmed at the sheer amount of frameworks/libs coming out every year, or at how the larning curve of each tech didn't allow me to actually develop because I couldn't get past the "Researching and trying out tech before marrying a technology" phase.
The thing is: You A) Don't need to learn the newest things if all you want is to create a cool app. And B) Won't get anything from learning about building apps without actually building them.
Maybe you do need to get off HN, but I recommend just browsing HN safely and knowing that there'll be time to learn these new things if you learn them through making cool stuff. If your priority is to make apps, these new frameworks shouldn't bother you so much. You can make a nice app just with Flask.
But I do feel old too. I used to think Angular was the hottest thing.. Now I hardly enjoy it lol but that's because my Angular-married colleagues use very outdated practices, which were "modern" a few years ago.
If every month comes out someone from nowhere with the latest and best, what this really means is that its is really not the best! People nowadays put together two shiny things and go around the world saying they have the best thing ever. It is just silliness. Stop paying attention to that and stick to well tested and proven technologies.
I feel you. Tooling for programming often seems to presume that programming is your primary focus and that you can afford to spend time every month keeping up with changes. For a small passion project the overhead of this churn eats a large fraction of the time budget, not to mention the demotivation of realizing how much energy you are spending to essentially tread water.
Whereas with art or math or writing projects I can come back to old notes decades later and basically pick up where I left off.
This helped me: Nobody asks a professional violin player to "just" switch to the cello, because the orchestra is missing one. Yes, there are many musicians who are proficient in multiple instruments, and yes, the underlying music theory is the same and maybe even the playing mechanics might be very similar (e.g. within strings or winds).
A few years ago I made a conscious decision that - at least during my remaining professional career - I will not switch to a new programming language anymore. Instead, I intend to keep specializing in Java. I embrace the fact that in 20 years I will be today's (Assembly|Fortran|C|...)-specialist. I do enjoy reading about other language features. I applaud the people at the cutting edge, experimenting with new stuff, and I am glad that Java is slowly incorporation some of those innovations. Every LTS release (2-3 years), I enjoy learning the curated set that Oracle's language architects decided to add.
This decision helped me tremendously with the fear of missing out on the latest and greatest.
I think the .so files are built from other source though? I'm not sure. In the makefile they are rm'd and then created. The linux workflow file, for example, has:
make linux
cd ../
git add libsocketify_linux_amd64.so
I assume that if this C-extension is faster than Golang Fiber, it is way faster than asyncio!
And it would have been the same with Python 2.7. This demonstrates yet again that the main value of Python has always been C-extensions.
The efforts of the old boys to add bloat in Python 3 (mostly written by other people of course, the old boys talk and rarely develop) have been misguided.
Sorry for all the hate you seem to be recieving, I think it looks great!
I can't join the discord however since I was banned for unknown reasons and cannot create a new account without giving them my phone number, which I refuse to do.
But remember many great things get hated upon in the beginning, take dropbox as an example. People on HN thought it was a shitty idea and they seem to be doing just fine :)
Yet another "X faster than Y" where inside X all hot jobs are done by really good-tuned C or C++. Facepalm.
Without real problem-solving (business logic) written in Python, this is only lightweight wrapper on C/C++. When the number of Python code start growing in the hot path, then these blazing-amazing thruput numbers tremendously will be going down.
Why is it a sham? It's useful to know that x is faster. As a user of x I don't really care if the reason it's faster is it's C++ under the hood. That's really an implementation detail for me.
What happens when you use X because it's fast at one thing in a microbenchmark, but it locks you into an environment that is slow at the rest of your program?
Because it's measuring a scenario that's not representative of real usages of X and is usually tweaked for the framework/tool the author is trying to showcase.
For example: all the deno/bun benchmarks often use the Node.js frameworks in an unnecessarily slow way and don't measure apples to apples. (like using uWS in bun but not uWS for Node in the benchmark).
This one in particular just measures the speed of "hello world" in plaintext and JSON with very specific parameters.
The real thing will be some mix of your code and the library's code, and in cases like this where you'd be writing all your logic in Python vs Golang, that will have a significant effect on the results.
A while back I implemented a game rules engine and MCTS in Torch (the Lua library). The training loop spent like half of its time running the rules engine. Writing it in Python would have been a disaster in comparison. To really make good use of the hardware and spend more time in the optimized machine learning code provided by libraries, one would have to write their rules engine in some language other than Lua or Python.
Because the moment you actually write any server or business logic in Python performance drops off a cliff and it doesn't differ significantly from any of the other Python web frameworks.
I think this is correct and does demonstrate that these comparisons are not all that useful.
But I also think that writing Python and then dropping down into C or C++ for performance is perfectly valid and often a great idea. Of course, there's nothing wrong with just using a different language that's a sensible middle ground between Python and C (like Go). But hold on to the baby when that bathwater is being dumped: there's nothing wrong with writing a blend of Python and C++ for real uses.
> Yet another "X faster than Y" where inside X all hot jobs are done by really good-tuned C or C++. Facepalm.
This is the case for basically every programming language or performant library that exists. Yet, I never see this when discussions exists about node libraries or other languages for that matter. I mean you could argue that node itself is just a thin wrapper around fast C++ libraries.
Who the fuck cares if the request goes down to some compiled C++ library? I still write my logic in Python and get this benefit anyway. This is what makes it great.
> Who the fuck cares if the request goes down to some compiled C++ library?
The way this is titled, you know someone will read this as "Python can be faster than go" and influence their decision when it comes to performance ; that's actually why I clicked on that, thinking "how the hell did they achieve faster speed with Python".
Well, it could be true IMHO. If I run my code that calls down to some ultra fast, mega well done C++ library that is much better than anything in the Go community my code then completes requests faster than the Go alternative.
If that is the case, that should influence the decision when it comes to performance. If performance is a very important deal in your app, you have to test this yourself so that you reach the performance you need.
Perhaps you can write C++ extensions to Python for the perf-heavy parts yourself instead of having to write the entire application in a language that takes much longer time to develop in?
> Yet, I never see this when discussions exists about node libraries or other languages for that matter.
Huh? It's the first comment for pretty much any of these stupid performance comparisons. Does someone really think node performance is great, or pick node because of its performance?
> I still write my logic in Python and get this benefit anyway.
That's part of the point. Writing your logic in Python is fine, but then your app performance isn't winning any comparisons. No one cares about the performance this article is explicitly talking about.
NodeJS might be slow, but it still is orders of magnitude faster than any interpreted or JIT Python implementation. I agree that no one really chooses Node because it is fast, but I think they choose it because it is fast _enough_ whereas if using Python you'd probably find multiple performance bottlenecks that could be show-stoppers.
I think the issue is that there is three different types of people when it comes to performance.
There is the 1% that are in a need of hard realtime performance, that thinks about GC performance as an issue that needs to be adressed. Sure I can get that for these people, these kind of discussions seems stupid and confusing since they have to develop in C/C++, Rust etc to get the pure performance they need. These people probably aren't in this thread because they know that Python is not a language for them and will never be.
Then there is the 99% crowd, which I am a part of, that could use slow-as-shit frameworks like Rails or Laravel because the performance is just a nice-to-have. These people, like myself, compares Python perf to perhaps PHP or Node because they are fast enough languages and in this kind of league a framework like OPs can make a huge difference. No matter what language you pick here you are basically always using something "beneth" it. Node has been picked by many because it's faster than for example Python and Ruby.
Then there is the people who are part of the 99% crowd but are just have to comment and point the needs of the 1% out because they are negative people who probably produce little themselves and have to hate on what other people create and distribute for free. This group is where most haters on HN belong in. :)
I started in PHP, when I went to Node in the early beginning I was deeply impressed by the great performance and nice features it had. Sure it was built upon C++ libraries but I am never gonna use C++ to write my web api. Still, I could use Node and get 2-3x AT LEAST the performance in all of my endpoints because of this and all my code was in javascript.
What I am trying to say is that I don't care if you pack my code and run it on a computer in Minecraft or your calculator, if it runs faster for very little to no pain on my side that is pure joy to me.
Would golang interfaced with the C++ implemenation be as fast? IMO one of the nice features of Python is being able to seamlessly drop to numba, or a C++ module when you need an optimized fastpath, but the rest of your application, the bulk of it, need not.
From uWS docs basically "sends into one single syscall/SSL block", in my understanding corking is basically minimizing to much sends, so it's batching calls.
191 comments
[ 3.0 ms ] story [ 260 ms ] thread[0] -- https://github.com/pydantic/pydantic-core
Pydantic is indeed really slow (https://ltworf.github.io/typedload/performance.html)
typedload is faster and written in pure python (unlike pydantic).
I know they are rewriting it in rust… but I don't know what will come out of it. After all my pure python implementation already manages to beat pydantic's binary, and when unions are in use, also apischema's binary.
If that is all one's application does, and can use your library in their organization/team, that's great. However a 2-3x performance boost for the parsing stage for a use case like an API call might not matter when that could be overshadowed by validation and/or upstream API calls. A realistic app would likely use a validation library like Pydantic's [2] to throw a custom typed-error that can be processed, e.g., localization, before returning it downstream.
[1] https://github.com/ltworf/typedload/blob/37c72837e0a8fd5f350...
[2] https://docs.pydantic.dev/usage/validators/
Did you somehow miss all the other tests, even thought they are higher on the page? The important test is shown first: loading objects.
I'm not trying to benchmark my wifi or my disk. The IO time is not included there on purpose. Of course a bigger application that does other things wouldn't see this huge difference in performance. But I'm testing the performance of a library here.
typedload can use validators, but since it doesn't reimplement attrs/dataclass, I had no interest in testing those… it'd be a race between libraries I didn't write.
typedload's exceptions contain enough detail to tell the end user what went wrong.
when encountering errors in lists, typedload slows down due to keeping compatibility with python 3.7. However in my test with loading objects, despite the slowdown it remains faster than pydantic.
Not a Python expert, but could the Pydantic tests be possibly not realistic and/or misleading because they are using kwargs in __init__ [1] to parse the object instead of calling the parse_obj class method [2]? According to some PEPs [3], isn't Python creating a new dictionary for that parameter which would be included in the timing? That would be unfortunate if that accounted for the difference.
Something else I think about is if a performance test doesn't produce a side effect that is checked, a smart compiler or runtime could optimize the whole benchmark away. Or too easy for the CPU to do branch prediction, etc. I think I recall that happening to me in Java in the past, but probably not happened here in Python.
[1] https://github.com/ltworf/typedload/blob/37c72837e0a8fd5f350...
[2] https://docs.pydantic.dev/usage/models/#helper-functions
[3] https://peps.python.org/pep-0692/
The kwargs thing is true… but I didn't design the API of pydantic. And it only happens on the small top level dictionary, not on all of them (unless it internally does it all the time).
Java has JIT, I agree that in that case keeping the output value is important. cpython isn't that smart. I honestly never tried to benchmark using pypy. I guess it could be interesting to try that.
I don't understand this this logic.
Interesting results nevertheless.
https://www.techempower.com/benchmarks/#section=test&runid=1...
If you're a web developer, trying to pick a framework for websockets, you probably don't care that the Python framework is a libuv wrapper, while to Go framework is native.
So well done, I guess? I am really happy to see library authors taking performance seriously!
EDIT: This is assuming the benchmark is actually fair. I haven't looked at it, but it's not uncommon for benchmarks to be comparing apples and oranges.
Web developer dont care if the framework is written in native or it is an wrapper, developers just want something that works in a nice way :D
The benchmarks are in https://github.com/TechEmpower/FrameworkBenchmarks
EDIT: some preliminary results https://www.techempower.com/benchmarks/#section=test&runid=1...
So the concern is the title you used to get to the front page not the actual implementation.
well, a real world test would be both servers parsing 100kb json payload in each message
> EDIT: This is assuming the benchmark is actually fair. I haven't looked at it, but it's not uncommon for benchmarks to be comparing apples and oranges.
What is the point of this negativity? Why are you implying that the benchmarks might not be fair because you "haven't looked at it"?
Just as a piece of evidence, there are interpreted C++ (http://www.artificialworlds.net/wiki/IGCC/IGCC) and AOT Python (https://github.com/exaloop/codon) implementations.
When people refer to a language, most often they are using it as a synecdoche to refer to the whole language ecosystem and not just the the formal language definition.
If you look at the Python ecosystem, it is most definitely interpreted.
Or at least, that's how I'm hearing it from Google.
Just in case anyone else is getting it wrong like I was …
https://youtube.com/watch?v=v-n1vGeVIXo
(But really, https://youtu.be/u8_LDxZReTc)
I think it is fair to have a discussion about terminology and arguing with the actual meaning of terms. When there is a technical difference, it would be good to acknowledge that difference between terms. It does not hurt anyone to do that. One can still say something like: "When we communicate I will use the word x for meaning y, because it is shorter and more convenient."
Some languages have more of a proper spec than others, of course. If we look at for example Scheme dialects, then they often change their implementation for better conformance with the various rnrs standards. New Scheme dialects often early on try to conform very much to the standard to avoid issues later on.
Not yet it isn't. At least not for Python. I mean there are plenty of alternative implementations of Python but they aren't practical because they don't work with CPython modules (as far as I know anyway) so approximately nobody uses them. The de facto spec is "what CPython does".
https://pypi.org/project/psycopg2cffi/
I have run it with pandas+numpy and postgres+timescale without issue for the last year.
With Python, you don't even have to go "alternative" to use LLVM compiler infrastructure. You can use CPython + Numba and compile your kernels for both CPU and GPGPU on the go. Which is very practical.
Which is ironic, to compile a CUDA kernel in C++, you would have to use an alternative NVidia implementation of the language.
> general real world implementation
By this they mean the real-world-implementation that is in general use, which in this case is interpreted.
Yes, Python can be compiled and C++ can be interpreted. 99% of the time, they're not though.
When somebody says X language is interpreted, they mean that the most common and widely implementation is an interpreter. That's a useful bit of information to know about a language. "Correcting" them and saying that all languages can be interpreted OR compiled, doesn't really add anything of value to the discussion, other than to label yourself as a pedant.
On one hand, the conception from the 90s based on the most common Python implementation, on the other - a compiler that saves us tons of money. Which one adds value again?
That would be important in high-school debate club. In the real world, most people don't care about obscure edge cases and pedantry. In the industry vernacular, Python is an "interpreted language" and C++ is a "compiled language". Arguing against that in anything other than a highly (highly!) specialized context is just lighting candles in the wind.
We adopted it last year, and we don't rewrite code in C++ for GPGPU anymore. We save tons of effort on rewriting and even more on support.
How is this computer science?
I bet that's fun.
> There is no such thing as an "interpreted language"...
In the common vernacular, yes, there are.
All this is to say that a programming language these days is really the sum of its parts, not a spec.
E.g. many languages’ syntax is context-free, yet the languages themselves are almost always Turing-complete.
OTOH, interpreting C++ is actually a pain in the ass due to design choices it makes specifically regarding the semantics of translation units. You can see similar pain when trying to say, make a Go REPL: it's just not tuned for this.
It is of course technically possible to compile almost anything ahead-of-time. But, if you effectively wind up having to ship an interpreter or JIT into the resulting binary to run some code anyways, it's almost for naught. Both Python and JavaScript have eval. Python also has several other cases like Pickle where this can be an issue. PyPy being a JIT makes a lot of sense because it wants to be a drop-in replacement, and that makes the most sense for a language with these constraints. Codon can do AOT, but for practical reasons it is not nearly a drop-in replacement for CPython, just like the other Python AOT implementations.
A practical Python AOT is not compatible with loads of things that people associate with Python, like Django. PyPy is compatible with more, but even it is far from a drop-in today, and that is a problem.
> A language is just a set or rules and keywords. Everything you can fit into Backus–Naur form is already a language even if it doesn't have any implementation nor compiler neither interpreter.
This is objectively true. However, in practice, there are a finite number of available toolchains for any given language that exist today, and creating new ones is a non-trivial endeavor, especially production-quality ones. A language is just a set of rules and keywords. However, when people use Python and write Python, they are not merely writing Python to fit into those keywords and rules. They're writing Python to be executed and solve a problem, typically using CPython. Python is a language, but it's also an ecosystem.
In the same token, if something like Codon doesn't even support all of the things you can fit into Backus-Naur form about Python as it is in CPython and PyPy, can it even be called Python in this sterile technical sense?
My advice to you: Stop fighting that "misconception". See the other responses to back up the idea that Python is interpreted. The existence of a Python compiler is also not relevant here, as the linked project does not list that as a supported Python variant.
Let's assume that by some technicality you are not wrong. The "misconception" is so common that it is generally a fact in peoples mind. Rather than convince them, you are destroying your own credibility and probably making a fool of yourself in their eyes. And remember, I prefaced this with you not being wrong. It's worse for you if you are wrong. Just stop for your own good.
> Rather than convince them, you are destroying your own credibility and probably making a fool of yourself in their eyes.
I'd understand where you were coming from had the parent comment not explained what they meant and given examples.
Last year we adopted Numba for GPGPU programming. It is a Python compiler with LLVM back-end and full CUDA support. It generates the same code that NVCC (CUDA compiler for C++) does only it compiles Python code and not a C++ .cu dialect.
No one in our organization writes heterogeneous code in C++ anymore.
Sure there is. Even if it is technically possible to write an interpreter for a "compiled" language, or a compiler for an "interpreted" language, language design really pushes you towards one or the other. "Technically possible" can also represent a very real challenge, as that set of rules and keywords might make ahead-of-time compilation either impossible or pointless.
E.g. any language that exposes `eval` with an arbitrary argument will still need to ship an interpreter as part of the runtime even if you compile some of the code ahead of time. Monkey-patching is common in Python and Ruby, and is pretty hostile to compilation. Codon has several documented limitations around this. Perl 5 famously can't be statically parsed without evaluating the code as you go, which completely defeats any attempt at compiling it.
Inversely, You can write an interpreter for C++, but features like `consteval` and `constexpr` are meaningless without ahead-of-time compilation.
No they're not, `constexpr` and `consteval` indicate where you can use an expression, not when in the compilation/interpretation pipeline they need to be evaluated. The spec does not say that C++ has to be compiled, nor does the spec say "A compiler must execute `constexpr` expressions prior to entering `main()`".
If you think this is just being pedantic, well, that's kind of the point of a spec.
So when someone says "Python is an interpreted language" they are pleading to an implicit preexisting shared knowledge about the nature of the most commonly found usage of the concept.
Granted, when this shared knowledge doesn't match, that's where misunderstandings can happen, but that's the nature of human language. In this case I'd be hard pressed to think that any misunderstanding could have been possible for the immense majority of people reading this.
I would assume this is same case where - what ever Socketify is based on (pythons transport layer), is compared to FastHTTP. and mention of Fiber is click-bate.
The benchmarks are TechEmPower plaintext https://github.com/TechEmpower/FrameworkBenchmarks
preliminary results here: https://www.techempower.com/benchmarks/#section=test&runid=1...
https://github.com/django/django
https://github.com/mopemope/meinheld
So django meinheld is basically saying that i used Django served by meinheld in that benchmark.
Yeah this test basically shows that Python backed by uWS is crazy fast, but is not an direct comparison to uWS to Go.
This test is just an troughput test, with is very useful to measure raw performance.
More tools like caching tools, a better database client etc is need to construct an complete scenario, and i'm working on it! (Maybe in 1 week or 2 weeks will be done)
https://github.com/TechEmpower/FrameworkBenchmarks
https://www.techempower.com/benchmarks/#section=test&runid=1...
Correction. It shows that uWS is crazy fast. Sure if python us used as a thin glue between something like "select username from user" and uWS or send file over uWS then the result is fast. Do something meaningful in Python itself an it will be slow.
This is not to diminish your work. Python is mostly glue anyways so in average the end result should be quite ok
It is positive. You've created very nice "Web Python". No arguments here.
Saying that Python is faster than Go with this as a proof looks like overreaching for Me. It only proves that wrapping C code in Python is fast. It’s an achievement, sure, but your app probably won’t be faster when you add thousands of Python lines between request and response.
I used https://github.com/TechEmpower/FrameworkBenchmarks and in the future i will write fortunes and other benchmarks with have more things going on!
> It only proves that wrapping C code in Python is fast.
Yes, it proves that for this specific use case Python is faster.
And it is not claimed anywhere. Besides, cpython is not really know to be fast, as it is an interpreter? Why would its implementation language matter much?
And if you are not making reference to a specific implementation then the comparison does not make any sense.
Then it kind of make sense to talk about " pure Python" (the interpreted one) and "wrapped C", doesn't it?
Good. Because otherwise people would be mislead to think that Python itself is fast. The Python plus C combo has opposite tradeoffs in many dimensions.
If you look at it as a framework that minimises the networking overhead, then fine, it's an interesting piece of software.
If on the other hand you look at it like a "fast" web framework then things start to change and the discussion gets a bit more complicated.
So for example, you look at the source code of the applications being benchmarked (example: https://github.com/cirospaciari/socketify.py/blob/main/bench...) and you immediately see it's simply returning the string "hello world"). Which means that it's almost 100% of the time running in the fast path / best case.
My guess is that as soon as you start doing any kind of computation in the request handler in normal non-super-optimized python (trival example: validating some headers and/or checking some signatures as you would do with jwt tokens for example) then the python-vs-golang gap will start to go back to favour golang.
And then again, it boils down to what you're doing: anything io-intensive might benefit from the unetworking/uwebsocket beneath, anything cpu-intensive will benefit from the golang compiler producing native executable code.
Nice work anyways.
https://www.techempower.com/benchmarks/#section=test&runid=1...
It is as fast as the fastest C++ / Rust implementation.
https://github.com/maurice2k/tcpserver
I went back to Flask recently and keep finding everything I know is deprecated, or not best practice any more.
Then I read every week that some new framework is out that is better.
I'm tired of it. I end up spending more time learning new things than I do building new things! Maybe I should get off hn for a while
https://www.youtube.com/watch?v=UNQcFjIOfCE
The thing is: You A) Don't need to learn the newest things if all you want is to create a cool app. And B) Won't get anything from learning about building apps without actually building them.
Maybe you do need to get off HN, but I recommend just browsing HN safely and knowing that there'll be time to learn these new things if you learn them through making cool stuff. If your priority is to make apps, these new frameworks shouldn't bother you so much. You can make a nice app just with Flask.
But I do feel old too. I used to think Angular was the hottest thing.. Now I hardly enjoy it lol but that's because my Angular-married colleagues use very outdated practices, which were "modern" a few years ago.
Whereas with art or math or writing projects I can come back to old notes decades later and basically pick up where I left off.
Anyway what worked fine yesterday isn't necessarily a bad choice today.
A few years ago I made a conscious decision that - at least during my remaining professional career - I will not switch to a new programming language anymore. Instead, I intend to keep specializing in Java. I embrace the fact that in 20 years I will be today's (Assembly|Fortran|C|...)-specialist. I do enjoy reading about other language features. I applaud the people at the cutting edge, experimenting with new stuff, and I am glad that Java is slowly incorporation some of those innovations. Every LTS release (2-3 years), I enjoy learning the curated set that Oracle's language architects decided to add.
This decision helped me tremendously with the fear of missing out on the latest and greatest.
"Any sufficiently large Flask project contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Django."
Just sticky with Django. It's stable, well-maintaned and includes most batteries out-of-the-box.
[0] https://github.com/cirospaciari/socketify.py/tree/main/src/s...
And it would have been the same with Python 2.7. This demonstrates yet again that the main value of Python has always been C-extensions.
The efforts of the old boys to add bloat in Python 3 (mostly written by other people of course, the old boys talk and rarely develop) have been misguided.
I can't join the discord however since I was banned for unknown reasons and cannot create a new account without giving them my phone number, which I refuse to do.
But remember many great things get hated upon in the beginning, take dropbox as an example. People on HN thought it was a shitty idea and they seem to be doing just fine :)
Without real problem-solving (business logic) written in Python, this is only lightweight wrapper on C/C++. When the number of Python code start growing in the hot path, then these blazing-amazing thruput numbers tremendously will be going down.
Real world benchmarks take more time to prepare, we get that, but let’s be honest: a bold claim _needs_ bold evidence.
For example: all the deno/bun benchmarks often use the Node.js frameworks in an unnecessarily slow way and don't measure apples to apples. (like using uWS in bun but not uWS for Node in the benchmark).
This one in particular just measures the speed of "hello world" in plaintext and JSON with very specific parameters.
A while back I implemented a game rules engine and MCTS in Torch (the Lua library). The training loop spent like half of its time running the rules engine. Writing it in Python would have been a disaster in comparison. To really make good use of the hardware and spend more time in the optimized machine learning code provided by libraries, one would have to write their rules engine in some language other than Lua or Python.
But I also think that writing Python and then dropping down into C or C++ for performance is perfectly valid and often a great idea. Of course, there's nothing wrong with just using a different language that's a sensible middle ground between Python and C (like Go). But hold on to the baby when that bathwater is being dumped: there's nothing wrong with writing a blend of Python and C++ for real uses.
This is the case for basically every programming language or performant library that exists. Yet, I never see this when discussions exists about node libraries or other languages for that matter. I mean you could argue that node itself is just a thin wrapper around fast C++ libraries.
Who the fuck cares if the request goes down to some compiled C++ library? I still write my logic in Python and get this benefit anyway. This is what makes it great.
The way this is titled, you know someone will read this as "Python can be faster than go" and influence their decision when it comes to performance ; that's actually why I clicked on that, thinking "how the hell did they achieve faster speed with Python".
If that is the case, that should influence the decision when it comes to performance. If performance is a very important deal in your app, you have to test this yourself so that you reach the performance you need.
Perhaps you can write C++ extensions to Python for the perf-heavy parts yourself instead of having to write the entire application in a language that takes much longer time to develop in?
Huh? It's the first comment for pretty much any of these stupid performance comparisons. Does someone really think node performance is great, or pick node because of its performance?
> I still write my logic in Python and get this benefit anyway.
That's part of the point. Writing your logic in Python is fine, but then your app performance isn't winning any comparisons. No one cares about the performance this article is explicitly talking about.
There is the 1% that are in a need of hard realtime performance, that thinks about GC performance as an issue that needs to be adressed. Sure I can get that for these people, these kind of discussions seems stupid and confusing since they have to develop in C/C++, Rust etc to get the pure performance they need. These people probably aren't in this thread because they know that Python is not a language for them and will never be.
Then there is the 99% crowd, which I am a part of, that could use slow-as-shit frameworks like Rails or Laravel because the performance is just a nice-to-have. These people, like myself, compares Python perf to perhaps PHP or Node because they are fast enough languages and in this kind of league a framework like OPs can make a huge difference. No matter what language you pick here you are basically always using something "beneth" it. Node has been picked by many because it's faster than for example Python and Ruby.
Then there is the people who are part of the 99% crowd but are just have to comment and point the needs of the 1% out because they are negative people who probably produce little themselves and have to hate on what other people create and distribute for free. This group is where most haters on HN belong in. :)
I started in PHP, when I went to Node in the early beginning I was deeply impressed by the great performance and nice features it had. Sure it was built upon C++ libraries but I am never gonna use C++ to write my web api. Still, I could use Node and get 2-3x AT LEAST the performance in all of my endpoints because of this and all my code was in javascript.
What I am trying to say is that I don't care if you pack my code and run it on a computer in Minecraft or your calculator, if it runs faster for very little to no pain on my side that is pure joy to me.
.NET were gaming their techempower benchmarks to advertise they were blazingly fast. Turns out, not so fast with real-world code.
https://docs.socketify.dev/corking.html
https://baus.net/on-tcp_cork/