Interesting and roughly in-line with what I've read elsewhere. One must note, however, that performance on a couple of benchmarks does not generalize to all problems. Still - perfect is the enemy of good, and it may be that when looking for a high-level language one might have to settle for "good enough."
C++ can be a nice language if you are disciplined with it, but being disciplined means not using every feature and using features only when they make code clearer or higher quality.
Unfortunately, it's not only you who must be disciplined, but all of your colleagues/collaborators and to a lesser extent the authors of any libraries you happen to depend upon (or will depend upon). Which is a shame, because writing C++ can be a whole lot of fun, but the aforementioned invariants rarely hold.
Treating sequences as strings of characters is a peculiarity, but it really isn't much like text. The operations you would expect to perform on text like uppercase, lowercase, tokenize at word boundaries, etc. are irrelevant. The operations that you perform on sequences like reverse, reverse complement, align, edit distance are separate. No reason why FORTRAN couldn't do great for that.
My first ever professional program was a source code formatter (for Jovial language) written in Fortran (77 I think). Writing a parser in Fortran (early 80's) is an exercise I am glad I never had to repeat.
I actually got to the end and said "never heard of Fortran?".
Modern Fortran would be a good option; 'right' is a different matter. Certainly, Fortran 2003/2008/2018 are expressive languages (way more than F77), has an extensive ecosystem with good integration with C, has compilers that generate very fast code, handles parallel computations and SIMD and first-class language features, is already well established in the STEM world, etc., etc.
The biggest downside of Fortran is it's called 'Fortran' and so many people are unwilling to believe it's changed since the 70s.
One note, Fortran 2018 compliance in the compilers is still evolving, so not all features are in all compilers yet. Fortran 95/2003 support should be solid and most all have all of 2008 in.
It is the right option for anything numerical and array-based. Any super-cool array-based feature that people see in MATLAB and Python was inherited from Fortran 90. Yes, that is true. Sadly, modern Fortran is treated quite unfairly within the field of computer science and engineering because of people's lack of knowledge of modern Fortran, which reminds me of the famous quote: "People hate each other because they do not know each other, and they do not want to know each other, because they hate each other".
Crystal is really exciting, it somehow 'fell under the radar' but when I heard 'fast, and ruby-like with types' I became very excited.
There's even a Rails-like [0] framework being developed!
Last weekend I spend a couple of hours getting Lucky up and running (it took some doing, I had to borrow a lot from people's Docker images to get it booted and working).
It's a big 'watch this space' situation. The macro system for metaprogramming is very easy to understand in Crystal [1] as well.
I'd recommend you check out https://amberframework.org too. I also like Crystal but Lucky kind of turned me off considering the steep learning curve coming from Rails whereas amber is much more similar.
I would qualify a language as high level if you're not expected to deal with pointers in 'normal' use. Yes, Rust has pointers, but using them is NOT normal.
C/C++/C#/D, etc are not high level by this criteria.
There are also of course degrees to all this. Rust is lower level than a lot of languages by virtue of having native pointer uses, even if it's frowned on.
There's also something to be said for language features, but I can't quite put my finger on it.
The difference between pointers and references can easily become muddled. If you have a language with pointers that are non-nullable, type- and memory-safe, is that not high-level?
I think a good real-world example that exposes the problem in your definition is Go. Go has pointers, using them is normal. However, go does not pointer arithmetic and outside of unsafe (like Rust) they are memory safe. I consider golang to be higher level than C/C++ for this reason, and many others (GC, channels, defer, etc, etc) -- I'd also consider it lower-level because of its non-answer/cop-out to error-handling.
But what is special about pointers compared to references? If you have a language with pointers that are type-safe and memory-safe how is this distinctive?
I mean, even Python has references to variables. You can't escape references (as opposed to values).
The difference is one of model. Pointers are an exposure of the underlying computer architecture. Whereas references are more of a property of common language design. In theory, you could not have pointers but still have references.
> Pointers are an exposure of the underlying computer architecture. Whereas references are more of a property of common language design.
Sorry, I just don't follow. How are the pointers in Go more exposing of the underlying architecture than a reference? (I'm using Go as an example to make it concrete, but any language with similar properties will do).
The syntax and some of the semantics of assignment and rebinding are different between say go pointers and python references, but that's the point I'm contesting, I don't see how one is necessarily higher level than the other. If you put automatic memory management, null pointer checks, removal of any "undefined behavior", pointers aren't necessarily low-level. It wasn't the pointer, it was the memory safety.
I personally think that once you tease it out that it becomes a semantics argument that unfortunately doesn't shed much light on what is "high-level".
I feel that defer is lower-level than C++ RAII which happens automatically in destructors without manually specifying. C has compiler-specific defers, and you can emulate defer using C++ RAII (though clunky).
I disagree. Rust still requires thinking about low-level details like lifetimes, aliasing, whether/when to use `Box` or `Rc` etc., Even when these are not explicitly spelled-out sometimes (like in lifetime elision), the programmer still has to be aware of these. Plus, modern C++ also abstracts away raw pointers by your definition, and has `unique_ptr` and `shared_ptr` which are the same as `Box` and `Rc` respectively. Bare pointers are frowned upon in modern C++ too. Furthermore, the OP talks about languages high-level enough to be used by biologists, while maintaining an acceptable performance.
Problems like the benchmark in the post are actually really simple to write in rust. They're all essentially streaming algorithms with some simple parsing.
Fwiw rust has completely replaces cpp for high performant bioinfx code in my workflows. Sooo much easier to write than cpp!
It can really depend how you write it. In some instances you can just throw things together and it can look a fair bit like you might write it in what many would consider high level languages.
On the other hand, there are times when you really need to get into the nitty gritty of things for performance reasons, and it ends up feeling quite low level.
It's longer than the Python version[0], I'm going out of my way to not allocate for every string. But, and maybe I'm biased here, it doesn't really feel like it's low-level code.
i think there needs to be a new set of terms. ah, that probably wouldn't help, because people would then mismatch eachother's idea of what terms mean and which set of terms correspond to which ideas. still, if people are calling c low level then what should assembly be called?
Imo the problem is that 'high' is a relative term. It's calibrated according to whatever the commonly used abstraction level is for the term-speaker.
The other problem is that level doesn't really mean anything, or it means something different in each language. Is it feature count? Abstraction potential? Memory-addressed vs objects? Statically or dynamically typed? Closeness of fit to the machine it's running on (imagine a lisp on a lisp machine, or x86 on an emulator - is that high or low level?)? Etc.
If "level" is ill-defined then low and high aren't relative terms, they're meaningless. But there is a lowest either way, the machine code. Then high level programming languages have the programmer writing code in a model independent of machine architecture. Those models are't one dimensional, so let's not bother trying to put them in order from low to high, let's talk about their features.
That's how i'd have it if it were for me to decide, but this isn't how language evolves.
Low-er? High and low are relative adjectives almost everywhere they're used. C is a low level programming language relative to most other programming languages; it has fewer layers of abstraction than say, Java. In a conversation discussing x86 assembly, LLVM IR, and C, then C would be high level. But that's usually not the conversation being had; there are a hundred times as many Java programmers as there are x86 assembly programmers.
We don't need a new set of terms. Relative terms are fine, they're just context sensitive.
OP had a specific high-levelness in mind, that Rust does not fall into. C is provided as performance baseline, which is a common benchmark. The languages are compared against Python - Nim and Crystal are closer to Python than Rust. For example GC is a requirement at language level implied by the author.
Rust is cool, but it does not have a horse in this race I'm afraid.
It's perhaps worth pointing out that programming languages are not themselves fast or slow; it is the programming language implementations that are fast or slow. (This is a particularly sore point in the programming languages research community.)
There are certainly languages that it is essentially impossible to implement to run fast.
Like, one can certainly write a slow C... but writing a fast Python (real Python, I mean, with full reflection, run-time introspection, etc..) would be...near impossible.
Yes, I know there are projects that make large subsets of Python run fast, but it's that last 5% that kills you.
Wouldn't that require some code to redefine `int.__add__` or `int.__radd__` between iterations of the loop? Which I would file under "bizarre shit that shouldn't normally happen." Before the loop starts, you'd have to override `int.__add__` to modify itself every time it's called, or something crazy.
If we're talking about custom classes and not ints, maybe it's a bigger problem. But if PyPy doesn't allow the required introspection to make this work, how does it run anything at all?
Most languages have a single implementation and definition of the language and especially its ecosystem is married to it. Some have more, but usually an ecosystem is popular for only one of them, because they are not entirely compatible.
Nowadays by a language people usually mean the whole package.
Forgive my lack of specificity, as I am quite new to Julia, however, I believe that after the initial compile takes place, you do experience significantly faster runtimes, so you can "forgive" the one time delay.
I think you can also have VS Code just load up the packages you expect to use on a regular basis at startup.
but I agree. If you complain that compile takes 11 seconds when the program runs in 30, then I wonder if that's really the use-case where you need every last bit of performance.
Now, on a program that runs for two days straight in Python or Matlab, and Julia reduces that time by half, I can deal with a bit of compile time.
Very interesting. If I understand correctly, this article is mainly focused on bioinformatics and performance of high level languages in that area.
I wonder why BioPerl (https://bioperl.org/, https://en.wikipedia.org/wiki/BioPerl) has not been included, it would have been interesting. Perl is considered surprisingly fast for certain classes of tasks (for an interpreted language of course, no point in comparing to C for example).
Perl isn't really an interpreted language. It's compiled to a sort of bytecode when you run the program, and executed on a sort of virtual machine. Similar to Java or C#, but much faster to compile. So fast that most people don't even know it's happening.
As for performance, it really helps if you write idiomatic Perl code. There may be more than one way to do it, but some ways are better. For example, an explicit for loop over a list has to be compiled to byte code that steps through the looping, but if you use map or grep the byte code calls a pre-written optimized function that loops as fast as code written in C would be. The more idiomatic your code is, the more optimized calls like that you get, and the faster your code will run.
'Interpreted' is often used to mean 'not compiled to machine code'. Perl fits that bill. So do, say, CPython and Lua, in contrast to JIT-compiling runtime systems like PyPy or LuaJIT or AOT-compiled languages like C or Rust.
I think it would be much better if someone asked a group of experts for each language to come up with their best implementation for the tasks. If I just spent a few hours implementing a benchmark in a handful of languages I wasn't proficient with then I think the benchmark rankings would be nearly random or merely dependent on my priors.
Without having dug into the specifics of this particular package, I can say generally speaking that average package quality has in my view been improving a lot over the past few years; two years ago the language was pre-1.0 and people were still figuring a lot of things out. Some packages of course were excellent even then, but those tend to still be actively maintained.
This does not seem to be a "real" library. It has a github, but no readme. It seems to be a julia implementation of some other c library, probably experimental.
I reviewed the first Julia benchmark. The second one is based on this not-very-thorough "experimental" lib I've never heard of from a person not very well-versed in Julia, so I don't think that's representative at all.
For the uncompressed FASTQ benchmark, my relative timings differ from his, with Julia being ~30% faster. It's probably because he installed an old version of FASTX.jl, that would also explain his seemingly outdated comment about the FASTX.jl source code. Another 30% time can be shaven off by disabling bounds checks, which would put Julia as the fastest program on the list, around the speed of C. However, I don't think the speed increase is worth it, since FASTQ files are basically aæways compressed in real life.
For the compressed FASTQ file, it seems this is entirely explained by CodecZlib.jl being 2x slower than whatever his C solution is using. However, when profiling CodecZlib.jl, it just spends all its time calling `zlib` written in C (calling C has near-zero overhead in Julia). So I have no idea why his benchmark is slower there.
Trying to get biologists to care about computers is like telling someone to be excited that they need to get all their wisdom teeth pulled and have a cavity filled on the same day. I was pretty much the oddity throughout all my schooling and any of the biologists I met while working that actually knew much of anything about computers and programming beyond what needed to be done to write reports and enter data.
Most biologists I met were far more comfortable in the pissing rain, up the mountain, in the middle of nowhere collecting animal shit than in front of a keyboard.
If there were a Venn diagram of biologists and computer and technology enthusiasts the overlap would need a micrometer to be read.
Disclaimer: Please take this extremely generalized and likely offensive to one of those small few in that tiny overlap I mentioned, statement with a grain of salt. Please don't take this too seriously, it's just from my own narrow sampling of people i've interacted with which is may or may not be representative of the overall population.
I know about bioinformatics and the contributions of biology to computer science and vice versa. I've personally worked in both field sampling and data analysis. My overly offensive generalized statement was a light jab towards field biologists i've known, including my own friends who i've debated on this subject with, who tend to dislike computers and will go as far to avoid even excel and hand write their data while doing all math on a calculator. It really wasn't meant to be taken seriously and I suppose would go above the head of those who haven't spent a lot of time in the field with biologists.
Okay, sorry about the misinterpretation, but it should have been clear to you that ggp was referring to computational biologists or bioinformaticians, making your observations rather unrelated.
Bioinformaticians use Python,R,Java,Julia,Groovy,Js and more (I use kotlin and python mostly but that's in a niche of bioinformatics I don't touch genes nor proteins)... Same as in software dev, there are a lot of beliefs. So you have the club of the js people that hate java, the club of pythoners that hate R, and people that just use what allow them to solve their problem and they are the ones that achieve the most but also that you hear about the less.
And that is the problem. I understand Julia and Python there. I'd understand Haskell (as every biologist is a mathematician too). But to list JS and Lua, and not have C# and Java?
AFAIK a lot of chemistry equipment actually is built with Java and C#.
Both of these are good development languages, particularly for large projects. However, for bioinformatics or other science/academic projects, they have neither the wide range of libraries and ease of writing quick scripts that python has, nor the speed of something like C.
I think that for most academic uses, ease of development is really important. Remember, most of these people don't have CS backgrounds, which makes python a good choice because it's easy to learn and doesn't require much setup. It's main drawback is that it's slow, but this doesn't matter in many cases. When I was in grad school, I'd use a small dataset to try to figure out the best algorithm and then if I had to leave my computer running overnight to run on a larger dataset, it usually wasn't a big deal.
On the other hand, sometimes you just need something really fast. In that case, C is your darling.
Java/C# make lots of tradeoffs on both sides, which works well for software developers but not necessarily researchers (who don't have to worry about code being maintainable).
That looks super fun! Are they still using this? I was at MS a few years before that, still dealing with a few legacy things in A+. There were hopes that we'd switch to something like F#, but we ended up just re-doing most of the mortgage analytics in kdb/q, which was fun in its own way.
Yep still used (pre and post trade, processing millions of orders and billions of market data events per day).
I'd like to grow it into a successor to kdb, though kdb is very entrenched in finance and the company is extremely litigious (they threatened to sue me).
F# is great too, and all of the other ML variants. We'll get there eventually, it's inevitable, but there's a lot of institutional inertia.
The inclusion of javascript makes me want to dismiss this whole article. If that's a serious candidate then I know he didn't look very thoroughly. But no, go would be just as horrible.
If the goal here is maximum performance, that's going to require an AOT-compiled language, which implies static typing. Is that acceptable to scientists used to writing Python and R scripts?
Bioinformatician here. It's certainly a drawback. There is no quesiton that Python etc. are more expressive and quick to write in than C are. Some bioinformaticians swear to static languages for reasons other than speed (e.g. static type checking).
There is always a need for scripting languages. Not just for speed of development, but for interactive data manipulation and visualization. Static languages are a no-go in that regard.
My bet in on Julia. Although slower than C by itself, I think in practice, Julia code written by bioinformaticians will be faster than C code wrapped in Python, or C libraries used in inefficient workflows from the shell. That's certianly been my experience so far.
Why not take a look at Kotlin? High level, statically typed, fast iteration, good runtime performance, great support for parallelism (via Java's APIs for it).
I very well might! As I said, I really do need an interactive language for some (most?) tasks, but it would be nice to complement it with a static language. I've been drawn to Rust since it seems to be enjoyable and less full of arcane obscurities than C/C++, but I'll take a look at Kotlin.
There's a Kotlin REPL and IntelliJ has this feature called 'workspaces' that's meant for interactive use. But it's not the primary focus of the team, for sure.
Nah, that’s not true. Something like Common Lisp (AOT compiled, btw) you can write without types and then add them in later to bump performance by facilitating unboxing and removal of inline type checks. It is a tradeoff because you lose some safety, but it can lead to some pretty fast code.
This is sort of interesting but isn't informative without any idea as to why some languages are slower. What is the difference between the LLVM-IR/assembly produced by C and the one produced by Crystal/Nim/Julia?
Compilers aren't different from pretty much every other kind of software, for which it's generally painfully obvious that two programs that target the same data _format_ will not necessarily output the exact same _data_, especially when they have wildly differing end goals.
I don't think every article discussing benchmarks has to restate that the differences between programming languages are not just syntactical to be informative.
I'm not sure I follow. What I meant is that a benchmark over a specific task is not really informative without a comparison of why it is slower in some languages compare to other.
I know many people don't consider Common Lisp as particularly fast language, but actually in my experience you can make it quite fast depending on what you need.
When using SBCL, for example, you get your application to be compiled to native instruction set. Moreover, you get control over optimization level for each piece of the code separately. Even more, you get complete control over the resulting native assembly. Something you don't get to do with other higher level languages.
One of the production applications I did in Common Lisp was parsing a stream of XDP messages (https://www.nyse.com/publicdocs/nyse/data/XDP_Common_Client_...) with a requirement for very low latency. I made the parser in Common Lisp so that it generates optimal binary code from XML specification of message types, fields, field types, etc. using a bunch of macros.
The goal of application was to proxy messages to the actual client of the stream. The proxy was there so that it was possible to introduce changes to the stream in real time without requiring to restart any components. Using REPL I was able to "deploy" any arbitrary transformation on the messages. The actual client of the messages was a black box application that we had no control over but we sometimes had problems with when it received something it did not like.
I liked Common Lisp in particular because it does not force you to make your performance decision upfront. You can develop your application using very high level constructs and then you get the option to focus on parts that are critical for your performance. Macros allow you to present DSL to your application but then have full control over code that is actually working beneath the DSL.
If everything fails, calling C code is a breeze in Common Lisp compared to other languages.
I explored Common Lisp for high performance code once (I work in HFT), the biggest issue was that it didn't have native support for "arrays of structs" (where all the structs would be values stored "unboxed" adjacent in memory). I know it'd probably be possible to write a library to do that, but that'd be a huge amount of work compared to just using a language with existing support for unboxed arrays.
Well, I did some algorithmic trading (not really HFT but still monitoring the market and responding within 5 microseconds of incoming messages).
I would not use Common Lisp on the critical path because I would end up basically rewriting everything to ensure I have control over what is happening so that some kind of lazy logic does not suddenly interrupt the flow to do some lazy thing.
A large part of the application was basically about controlling the memory layout, cache usage, messaging between different cores, ensuring branch predictor is happy, etc which would be really awkward in Common Lisp (technically possible, practically you would have to redo almost everything). We have also experimented with Java with the end result being that the code looked like C but was much more awkward.
I have, however, successfully used Common Lisp to build the higher layer of the application that was orchestrating a bunch of compiled C code and also did things like optimizing and compiling decision trees to machine code or giving us REPL to interact with the application during trading session.
>giving us REPL to interact with the application during trading session
This to me is what the big appeal of Common Lisp for a trading system could be, particularly if it allowed live recovery from errors (dropping somebody into the debugger, rather than just core-dumping), which could save a lot of money by reducing downtime. But as you say it would require redoing everything to make the code fit latency constraints and be cache friendly, which would be a lot of work.
I would be curious to see what the performance is like in Go. I've been experimenting with it a bit lately, and coming from a mostly C and Python background, I have thus far found it easy to pick up. I haven't tested the performance for my own use cases very much yet, but I am told it compares very well.
I think the real challenge for scientific computing (I'm a graduate student, so this is most of the programming I do) is that there is already a huge network effect around NumPy + SciPy + matplotlib and friends. Golang just doesn't quite have the library ecosystem yet, although gonum[0] shows some potential.
In my limited experience so far, I think Go is good in an environment where most people are going to have experience with C and/or Python. It also makes it much harder to write truly crappy code, and it's much easier to get a Go codebase to build on n different people's workstations than C.
Having written a lot of Python, and relatively little Go, I think I would prefer to write scientific code in Go if the libraries are available for whatever I'm trying to do.
It's also much easier to integrate Go and C code, compared to integrating C and Python.
Personally I think numerical code should still mostly be written in C++, right now it still has by far the widest choice of options for doing so. It is also relatively easy to interface with python. For example both xtensor, libtorch/ATen, arrayfire have straightforward python interoperability via pybind11.
Finally no other language except for maybe FORTRAN has seamless parallelisation support and first class low level numerical primitives developed by vendors. Sometimes you will get a massive performance increase by #pragma omp parallel for.
Even for visualization some python libraries will suddenly fall off a cliff (Altair) once you reach a moderately large number of datapoints.
I would definitely agree that it depends on what kind of scientific computing you are doing.
For big numerical stuff and things that need to run on supercomputers, C/C++/FORTRAN are definitely very relevant and I don't see that changing. Likewise for edge stuff that has to run on bare metal or embedded, I think we're still going to be using C/C++ for a long time to come.
"Scientific computing" is a huge range of different use cases with very different levels of numerical intensity and amounts of data. I doubt very much that there would ever be a one-size-fits-all approach.
However in the context of the OP, I'm arguing that Go would be preferable to Python for the purpose of writing bioinformatics models, and certainly more suitable than Lua or JavaScript.
Of course Python can sometimes be very performant if you leverage NumPy/SciPy, since those are ultimately bindings into the FORTRAN numeric computing libraries of yore. But if we're talking about writing the inner loop, and the choices are Go, Python, Lua, and JavaScript, I think Go is going to win that on the performance and interoperability fronts handily (I omit Crystal, as I am not familiar with it).
Even in the context of bioinformatics my comment applies. With modern C++ libraries you are able to replicate the numpy user experience almost line by line. A baseline FASTQ parser in modern C++ would look nothing like the fastq.h C parser the author presented. Naive versions of sequence alignment algorithms like Needleman-Wunsch are easily implemented in C++ aswell and you can even do most of your development in a Jupyter notebook with cling/xeus.
I'll take your word for it; I haven't worked in that field.
I do still think it would interesting to see a comparative benchmark though. I know the Go compiler tries to use AVX and friends where available. I doubt it will ever beat a competent programmer using OpenMP to vectorize though (though Goroutnes might be competitive for plain multithreading).
A relevant consideration too -- OpenMP seems to be moving in the direction of supporting various kinds of accelerators in addition to CPUs*, so your C++ code has a better change of being performance-portable to an accelerator if you need it.
Note the go compiler is actually fairly immature compared to C++ compilers. It does not have any AVX autovectorization. Any AVX optimization is manual assembly by a library author.
I would expect Rust to be faster. I never claimed Go would be faster than Rust, but that it would be faster than Python.
I think for rust, the barrier of entry is high in that it is a difficult language to learn. Admittedly, I don't know rust, but people I know who do have said as much.
I think Go strikes a good balance of being easy to learn and use, having a rich standard library, and also being performant.
The author has a lot of familiarity with JVM based tooling for the operations he describes in the blog. I'm not aware of a Kotlin implementation but have seen him commenting on both Java and Scala implementations over the years. My assumption is the performance would be similar to those.
177 comments
[ 2.9 ms ] story [ 230 ms ] threadAnyway - good job!
Perhaps a well-written bio informatics library would be a nice solution.
http://www.stroustrup.com/tour2.html
Modern Fortran would be a good option; 'right' is a different matter. Certainly, Fortran 2003/2008/2018 are expressive languages (way more than F77), has an extensive ecosystem with good integration with C, has compilers that generate very fast code, handles parallel computations and SIMD and first-class language features, is already well established in the STEM world, etc., etc.
The biggest downside of Fortran is it's called 'Fortran' and so many people are unwilling to believe it's changed since the 70s.
There are quite a number of books. I recently got this one that is pretty good: https://www.amazon.com/Modern-Fortran-Explained-Incorporatin...
One note, Fortran 2018 compliance in the compilers is still evolving, so not all features are in all compilers yet. Fortran 95/2003 support should be solid and most all have all of 2008 in.
There's even a Rails-like [0] framework being developed!
Last weekend I spend a couple of hours getting Lucky up and running (it took some doing, I had to borrow a lot from people's Docker images to get it booted and working).
It's a big 'watch this space' situation. The macro system for metaprogramming is very easy to understand in Crystal [1] as well.
Can't wait for 1.0!
[0] https://luckyframework.org/ [1] https://crystal-lang.org/
C/C++/C#/D, etc are not high level by this criteria.
There are also of course degrees to all this. Rust is lower level than a lot of languages by virtue of having native pointer uses, even if it's frowned on.
There's also something to be said for language features, but I can't quite put my finger on it.
I think a good real-world example that exposes the problem in your definition is Go. Go has pointers, using them is normal. However, go does not pointer arithmetic and outside of unsafe (like Rust) they are memory safe. I consider golang to be higher level than C/C++ for this reason, and many others (GC, channels, defer, etc, etc) -- I'd also consider it lower-level because of its non-answer/cop-out to error-handling.
But what is special about pointers compared to references? If you have a language with pointers that are type-safe and memory-safe how is this distinctive?
The difference is one of model. Pointers are an exposure of the underlying computer architecture. Whereas references are more of a property of common language design. In theory, you could not have pointers but still have references.
Sorry, I just don't follow. How are the pointers in Go more exposing of the underlying architecture than a reference? (I'm using Go as an example to make it concrete, but any language with similar properties will do).
The syntax and some of the semantics of assignment and rebinding are different between say go pointers and python references, but that's the point I'm contesting, I don't see how one is necessarily higher level than the other. If you put automatic memory management, null pointer checks, removal of any "undefined behavior", pointers aren't necessarily low-level. It wasn't the pointer, it was the memory safety.
I personally think that once you tease it out that it becomes a semantics argument that unfortunately doesn't shed much light on what is "high-level".
Fwiw rust has completely replaces cpp for high performant bioinfx code in my workflows. Sooo much easier to write than cpp!
On the other hand, there are times when you really need to get into the nitty gritty of things for performance reasons, and it ends up feeling quite low level.
For example, here's a Rust program which does the same as fqcnt_py1_4l.py: https://gist.github.com/Measter/d31abe88b5e318ba98856bf6f047...
It's longer than the Python version[0], I'm going out of my way to not allocate for every string. But, and maybe I'm biased here, it doesn't really feel like it's low-level code.
[0] https://github.com/lh3/biofast/blob/master/fqcnt/fqcnt_py1_4...
Comparing scripting languages to compiled languages is apples and oranges. The advantage of scripting is supposed to be the convenience - not speed.
how times have changed
The other problem is that level doesn't really mean anything, or it means something different in each language. Is it feature count? Abstraction potential? Memory-addressed vs objects? Statically or dynamically typed? Closeness of fit to the machine it's running on (imagine a lisp on a lisp machine, or x86 on an emulator - is that high or low level?)? Etc.
That's how i'd have it if it were for me to decide, but this isn't how language evolves.
We don't need a new set of terms. Relative terms are fine, they're just context sensitive.
C (especially C99 and up) is far, far higher level than something like FORTH.
Rust is cool, but it does not have a horse in this race I'm afraid.
Jython does not have a GIL.
https://wiki.python.org/moin/GlobalInterpreterLock
Like, one can certainly write a slow C... but writing a fast Python (real Python, I mean, with full reflection, run-time introspection, etc..) would be...near impossible.
Yes, I know there are projects that make large subsets of Python run fast, but it's that last 5% that kills you.
to see the problem with Python, take this loop as an example:
There is no way, looking only at this part of the code to know if the `+` operator does the same operation every time it is executed.If we're talking about custom classes and not ints, maybe it's a bigger problem. But if PyPy doesn't allow the required introspection to make this work, how does it run anything at all?
Most languages have a single implementation and definition of the language and especially its ecosystem is married to it. Some have more, but usually an ecosystem is popular for only one of them, because they are not entirely compatible.
Nowadays by a language people usually mean the whole package.
I think you can also have VS Code just load up the packages you expect to use on a regular basis at startup.
but I agree. If you complain that compile takes 11 seconds when the program runs in 30, then I wonder if that's really the use-case where you need every last bit of performance.
Now, on a program that runs for two days straight in Python or Matlab, and Julia reduces that time by half, I can deal with a bit of compile time.
I wonder why BioPerl (https://bioperl.org/, https://en.wikipedia.org/wiki/BioPerl) has not been included, it would have been interesting. Perl is considered surprisingly fast for certain classes of tasks (for an interpreted language of course, no point in comparing to C for example).
My info could be outdated of course.
As for performance, it really helps if you write idiomatic Perl code. There may be more than one way to do it, but some ways are better. For example, an explicit for loop over a list has to be compiled to byte code that steps through the looping, but if you use map or grep the byte code calls a pre-written optimized function that loops as fast as code written in C would be. The more idiomatic your code is, the more optimized calls like that you get, and the faster your code will run.
Same is true for both CPython and certain sections of R code.
I would make a pull request, but their setup uses 1TB of RAM...
For the uncompressed FASTQ benchmark, my relative timings differ from his, with Julia being ~30% faster. It's probably because he installed an old version of FASTX.jl, that would also explain his seemingly outdated comment about the FASTX.jl source code. Another 30% time can be shaven off by disabling bounds checks, which would put Julia as the fastest program on the list, around the speed of C. However, I don't think the speed increase is worth it, since FASTQ files are basically aæways compressed in real life.
For the compressed FASTQ file, it seems this is entirely explained by CodecZlib.jl being 2x slower than whatever his C solution is using. However, when profiling CodecZlib.jl, it just spends all its time calling `zlib` written in C (calling C has near-zero overhead in Julia). So I have no idea why his benchmark is slower there.
The choice of languages for this benchmark is laughable.
Results in clear code, easy implementation, and good performance 95% of the time.
Most biologists I met were far more comfortable in the pissing rain, up the mountain, in the middle of nowhere collecting animal shit than in front of a keyboard.
If there were a Venn diagram of biologists and computer and technology enthusiasts the overlap would need a micrometer to be read.
Disclaimer: Please take this extremely generalized and likely offensive to one of those small few in that tiny overlap I mentioned, statement with a grain of salt. Please don't take this too seriously, it's just from my own narrow sampling of people i've interacted with which is may or may not be representative of the overall population.
Disclaimer: my background has absolutely no overlap with biology/bioinformatics.
AFAIK a lot of chemistry equipment actually is built with Java and C#.
In fact you might encounter some biologists using VB.NET after they outgrown their Excel VBA macros.
https://news.ycombinator.com/newsguidelines.html
I think that for most academic uses, ease of development is really important. Remember, most of these people don't have CS backgrounds, which makes python a good choice because it's easy to learn and doesn't require much setup. It's main drawback is that it's slow, but this doesn't matter in many cases. When I was in grad school, I'd use a small dataset to try to figure out the best algorithm and then if I had to leave my computer running overnight to run on a larger dataset, it usually wasn't a big deal.
On the other hand, sometimes you just need something really fast. In that case, C is your darling.
Java/C# make lots of tradeoffs on both sides, which works well for software developers but not necessarily researchers (who don't have to worry about code being maintainable).
https://github.com/Morgan-Stanley/hobbes
It’s kind of a structurally-typed variant of Haskell, integrates closely with C++, produces very fast code.
I'd like to grow it into a successor to kdb, though kdb is very entrenched in finance and the company is extremely litigious (they threatened to sue me).
F# is great too, and all of the other ML variants. We'll get there eventually, it's inevitable, but there's a lot of institutional inertia.
Anyway, it's generic and open source, so can be used for anything. I use it at home to make little video games for my son.
Why wasn't Go, Haskell, Java or OCaml even considered here? Seems much more appropriate than the likes of Javascript for scientific computations.
Types are child’s play for the crowd using Matlab and worrying about low-level performance like this though.
There is always a need for scripting languages. Not just for speed of development, but for interactive data manipulation and visualization. Static languages are a no-go in that regard.
My bet in on Julia. Although slower than C by itself, I think in practice, Julia code written by bioinformaticians will be faster than C code wrapped in Python, or C libraries used in inefficient workflows from the shell. That's certianly been my experience so far.
I don't think every article discussing benchmarks has to restate that the differences between programming languages are not just syntactical to be informative.
https://github.com/cncf/cnf-conformance/
Its been a dream.
You can pretty much write the ruby you are used to but it performs like a compiled language and you can make binaries.
Was SUPER skeptical at first when the team decided on it but its been a pleasant surprise.
When using SBCL, for example, you get your application to be compiled to native instruction set. Moreover, you get control over optimization level for each piece of the code separately. Even more, you get complete control over the resulting native assembly. Something you don't get to do with other higher level languages.
One of the production applications I did in Common Lisp was parsing a stream of XDP messages (https://www.nyse.com/publicdocs/nyse/data/XDP_Common_Client_...) with a requirement for very low latency. I made the parser in Common Lisp so that it generates optimal binary code from XML specification of message types, fields, field types, etc. using a bunch of macros.
The goal of application was to proxy messages to the actual client of the stream. The proxy was there so that it was possible to introduce changes to the stream in real time without requiring to restart any components. Using REPL I was able to "deploy" any arbitrary transformation on the messages. The actual client of the messages was a black box application that we had no control over but we sometimes had problems with when it received something it did not like.
I liked Common Lisp in particular because it does not force you to make your performance decision upfront. You can develop your application using very high level constructs and then you get the option to focus on parts that are critical for your performance. Macros allow you to present DSL to your application but then have full control over code that is actually working beneath the DSL.
If everything fails, calling C code is a breeze in Common Lisp compared to other languages.
I would not use Common Lisp on the critical path because I would end up basically rewriting everything to ensure I have control over what is happening so that some kind of lazy logic does not suddenly interrupt the flow to do some lazy thing.
A large part of the application was basically about controlling the memory layout, cache usage, messaging between different cores, ensuring branch predictor is happy, etc which would be really awkward in Common Lisp (technically possible, practically you would have to redo almost everything). We have also experimented with Java with the end result being that the code looked like C but was much more awkward.
I have, however, successfully used Common Lisp to build the higher layer of the application that was orchestrating a bunch of compiled C code and also did things like optimizing and compiling decision trees to machine code or giving us REPL to interact with the application during trading session.
This to me is what the big appeal of Common Lisp for a trading system could be, particularly if it allowed live recovery from errors (dropping somebody into the debugger, rather than just core-dumping), which could save a lot of money by reducing downtime. But as you say it would require redoing everything to make the code fit latency constraints and be cache friendly, which would be a lot of work.
I think the real challenge for scientific computing (I'm a graduate student, so this is most of the programming I do) is that there is already a huge network effect around NumPy + SciPy + matplotlib and friends. Golang just doesn't quite have the library ecosystem yet, although gonum[0] shows some potential.
In my limited experience so far, I think Go is good in an environment where most people are going to have experience with C and/or Python. It also makes it much harder to write truly crappy code, and it's much easier to get a Go codebase to build on n different people's workstations than C.
Having written a lot of Python, and relatively little Go, I think I would prefer to write scientific code in Go if the libraries are available for whatever I'm trying to do.
It's also much easier to integrate Go and C code, compared to integrating C and Python.
0 - https://www.gonum.org/
Finally no other language except for maybe FORTRAN has seamless parallelisation support and first class low level numerical primitives developed by vendors. Sometimes you will get a massive performance increase by #pragma omp parallel for.
Even for visualization some python libraries will suddenly fall off a cliff (Altair) once you reach a moderately large number of datapoints.
For big numerical stuff and things that need to run on supercomputers, C/C++/FORTRAN are definitely very relevant and I don't see that changing. Likewise for edge stuff that has to run on bare metal or embedded, I think we're still going to be using C/C++ for a long time to come.
"Scientific computing" is a huge range of different use cases with very different levels of numerical intensity and amounts of data. I doubt very much that there would ever be a one-size-fits-all approach.
However in the context of the OP, I'm arguing that Go would be preferable to Python for the purpose of writing bioinformatics models, and certainly more suitable than Lua or JavaScript.
Of course Python can sometimes be very performant if you leverage NumPy/SciPy, since those are ultimately bindings into the FORTRAN numeric computing libraries of yore. But if we're talking about writing the inner loop, and the choices are Go, Python, Lua, and JavaScript, I think Go is going to win that on the performance and interoperability fronts handily (I omit Crystal, as I am not familiar with it).
I do still think it would interesting to see a comparative benchmark though. I know the Go compiler tries to use AVX and friends where available. I doubt it will ever beat a competent programmer using OpenMP to vectorize though (though Goroutnes might be competitive for plain multithreading).
A relevant consideration too -- OpenMP seems to be moving in the direction of supporting various kinds of accelerators in addition to CPUs*, so your C++ code has a better change of being performance-portable to an accelerator if you need it.
I think for rust, the barrier of entry is high in that it is a difficult language to learn. Admittedly, I don't know rust, but people I know who do have said as much.
I think Go strikes a good balance of being easy to learn and use, having a rich standard library, and also being performant.
I have sat down to learn Rust several times investing many hours and still wouldn’t say I’m in any way competent at it.
One cannot even write a generic `reduce` function...