HN Explorer

Ask HN: Why can't we write a single system in multiple languages?

2 points by dennisy ↗ HN
I hate having to make a choice on language at the onset of a new project. Each language has various great features for certain tasks.

I was wondering today why we cannot have some platform which would allow me to call across languages. If we take a basic example and say that I want to have Node/TS and Python files in a single system - what is the most ergonomic / efficient way to do this available today?

Why is this concept not more popular? Is this a bad idea?

If this is a good idea, what do we need to improve upon what is available today?

8 comments

[ 3.4 ms ] story [ 23.6 ms ] thread
TotoHorner
Services Oriented Architecture is the most common way. Use different systems and have then communicate through HTTP or gRPC
dennisy
Yeah that is not really the idea here, that means a lot of overhead for introducing another system to have to call via http, deploy etc
PaulHoule
It is more common today, I think, for people to write the front end and back ends of a web system in different languages than to write them in the same language. For instance the front end might be Javascript or Typescript and the back end could be Go, Java, Rust, Python, C#, etc.

The same pattern can be used without the web browser by writing part of the system in one language and parts in another language and having them communicate via a "web service" or similar protocol.

Some runtime environments such as the JVM and .NET support multiple languages, so there is no problem with Clojure, Java, Jython and Scala in the same runtime or F# and VB.NET.

There are also methods for making calls between different languages in the same address space, particularly I would call out

https://cffi.readthedocs.io/en/latest/

which is increasingly being used as a model for other languages. That one above makes it very easy to call C functions from Python. When you do this though you run into problems which are sometimes subtle and sometimes not subtle. For instance I worked on a system which combined C++ with Java and used JNI for interoperation. The worst problem I ran into was that you could not use C threads. I also ran into the problem that the gdb debugger would always be catching segmentation faults from the Java runtime because segfaults in Java are a routine condition. So it took some configuring gdb to get into a place where I could run gdb and jdb on the same process at the same time so I could debug both the Java and C++ sides of the system at the same time.

Note there is more to it than just having a way to share access to functions and data, you also will need answers for how you will handle dependencies, do builds, run unit tests, run an IDE, for each environment.

dennisy
Is there no way to create a layer above the two runtimes which made it feel as though they are running on the same process?

> having a way to share access to functions and data

This is spot on, what I would like to achieve - to share data across the two languages by throwing variables around and the same with ops/functions on that data.

Under the hood, they could be running 100% separately but the code base would feel as though they are not.

PaulHoule
In the JVM world you can already write web service request handlers

   POST /some/where/{like_this}
that are discovered by the JAXB runtime no matter if you write them in Java, Scala, Groovy, Clojure, Jython, etc. It works very well because the JVM handles memory allocation and reuse systematically. It would be hard to do this in the C world because how C programs allocate memory is idiosyncratic to those programs. Despite that, Python's runtime (which is a C program that consumes libraries) works really well when glued to C and FORTRAN style code like scikit-learn.

DCOM, CORBA, and RMI were early attempts at what you are asking for. We really got vernacular JSON APIs, API gateways, message-oriented-middleware, enterprise service bus, API documentation tools, etc. Latency and reliability (the network flaking out) are big pains if you have a chatty communications protocol so it is just as important to batch transfers as it is to avoid unnecessary communications. Maybe you want to write map-reduce or something like SQL that consistently gets good-if-not-great performance.

If you have many servers running (more than 20 for sure) people will tell you that you need Kubernetes to manage your servers and no matter what you need a script that builds a working development system as quickly as possible.

You might really like Hazelcast

https://docs.hazelcast.com/hazelcast/latest/data-structures/...

which manages shared data structures seemingly transparently on moderate sized clusters, there is even an example of using it between Java and Python here

https://docs.hazelcast.com/hazelcast/latest/pipelines/python

pestatije
You mean something like this?:

https://en.wikipedia.org/wiki/List_of_JVM_languages

https://www.graalvm.org/22.2/graalvm-as-a-platform/language-...

dennisy
Hmm, I had not thought about it in this way - this is an interesting way of looking at it, to have a VM which allows various languages to I guess compile down to a shared code.

I guess WASM is also then in a way going to help with this?

Never heard of GraalVM, looking at the python example it does not feel hugely ergonomic, to pass in strings / files and specify a language...

Are people using this today?

lordkrandel
Hmmm, are you a developer? Generally, language is a small part of the choice. Much comes from the ecosystem: libraries, compilers or interpreters, testing environments, CI, CD, licenses, that tend to be different. Multiply the choice and you generally multiply the cost of maintenance. That's why the frontend and backend separation was born and why Node was actually built to try to solve. You have to maintain entire different toolchains.