Actually I doubt the article is right when it comes to performance. I mean nodejs could be faster than scala for some things, but scala could also be faster than nodejs for other things. It really depends on what you are doing.
Working with big lists and manipulate them is definitly faster in scala than in python, node, etc..
I've also been working with Play Framework and I'm pretty impressed at how awesome and easy to use everything is. The documentation is great, the performance is great.
I'm using it for just a web API, and I've had several scala developers tell me to use akka-http instead because it is simpler. I tried, but the route api is quite horrid in comparison to a simple file-based routing description, and the utilities (like the web service client) were a nightmare to work with and debug. And to top it off, the performance was still worse than play. Maybe all of that will get better, but as of right now, Play is hard to beat.
My only complaints so far:
1) Slick is bloated and kludgy and still produces bad SQL queries...I've resorted to using the plain SQL interface which is nice, but not great.
2) It could use some support for API description languages like swagger.
3) Testing your code using the dependency injection framework has been very unintuitive (I'm pretty new to dependency injection frameworks though, so maybe it's just me).
5) I wish I didn't have to deal with an opaque SBT plugin, and that the project just used a regular SBT project structure. I realize there are reasons for needing a plugin (like codegen for routes, etc.), but at least document how to work with the plugin better. SBT is like working with a black box as it is, no need to make it worse.
Yes there are issues, but what framework doesn't have them? I am too annoyed by 22 element tuple limit in context of slick and other stuff, but when you look around there are no better alternatives without major downsides. Try to find a decent framework written in functional strictly typed well scalable language and you end up with Play.
> 1) Slick is bloated and kludgy and still produces bad SQL queries...I've resorted to using the plain SQL interface which is nice, but not great.
You could check out quill [1]. Haven't used it in a real application yet but the codebase seems easier to understand and the sql it produced seemed reasonable. virtualwhys could probaly tell you more - if I remember correctly he is a contributor for quill (and was/is (?) a longtime user of scalaquery/slick).
On (3): DI can actually be quite practical. Use different implementation via configuration (bind at startup), use fakes for testing, benchmark two implementations before swapping them, etc. Develop to an interface, not to an implementation :)
Slick is good. Although we use the plain SQL interface because it offers the most flexibility. The docs for Slick should be much much better since it can get fairly complex. I would say its not very newbie friendly especially if you trying to learn Scala at the same time. When you figure out Slick it's very easy to use.
I have used Play Framework in production and its really good -- I really hope there is more adoption like we have seen with Node or Rails. The only complaint, is that there have been big changes from 2.0 to 2.5. Moreover, the documentation on the site is just OK, examples in the community can be out of date, and books are out of date as well.
What do you think of upgrading incrementally via each minor versions (e.g. 2.0 -> 2.1 -> 2.2, etc.)? It will be easier to reason (and debug) the changes between each version, and test compatibility on the dependent libraries.
PS. I have a Play 2.3 app already in production, 2.4 currently in development, and plan for a new app on 2.5. I'm upgrading from the oldest one first...
Upgrading minor versions is the way to go. It still seemed like a pain to upgrade since Play Framework has changed quite a bit in the 2.X branch -- more than you would think. I am not trying to trash Play Framework since when you get it up and running its simple the best. I'm a big fan of the JVM.
>LinkedIn is a prominent user. Their entire mobile stack is built on Node.js. They went from running 15 servers with 15 instances on each physical machine, to just 4 instances – that can handle double the traffic!
>eBay launched ql.io, a web query language for HTTP APIs, which uses Node.js as the runtime stack. They were able to tune a regular developer-quality Ubuntu workstation to handle more than 120,000 active connections per node.js process, with each connection consuming about 2kB memory!
I can remember when Git began to replace Subversion, but a lot of companies resisted the trend, because Git was so radically different. Subversion was comfortable: everything was centralized, and the revision numbers went up in an orderly fashion: 1, 2, 3, 4, 5, 6... You always knew where you were in history, with Subversion. It was clear that Git was better in some fundamental way, and yet it took effort to understand it, conceptually, at first.
I feel like the same thing is happening now with GraphQL versus REST. On some level it seems like GraphQL is altogether better than REST, but we don't see a mad rush to adopt GraphQL, because it represents a big shift in thinking, and those of us who just spent the last 10 years creating RESTful interfaces, we are a bit slow to throw out all of our work and start over with GraphQL. Also, the frontend needs to change as much as the backend, so it requires a full re-write of everything.
All the same, a title like "Scale vs. Node.js as a RESTful back end server" does strike me as an anachronism in 2016. Who cares? I doubt I will ever again build a RESTful interface. Rather, on the project I'm working on right now, we are getting rid of the RESTful interface and we are looking at our GraphQL options:
I have a question I have been trying to figure out for a while. All the examples of clients that use GraphQL seem to be javascript; is there any support for writing, for example, a Java client to talk to a GraphQL backend? This would really help make it easier to port infrastructure over piece by piece.
You can write the server in any language, and you can also write the client in any language. As a point of comparison, consider SQL databases such as MySQL or Oracle or MongoDB. There are clients written in dozens of different languages.
GraphQL defines the query language, but uses simple HTTP as the transport. It's not like like redis or mysql - those use custom protocols and so require special clients to use them. GraphQL backends are like Elasticsearch backends - you access them by hitting the exposed HTTP endpoint with your query in plain text. You could do this with any HTTP client library in any language, or even with curl.
Thats true, but in a typed, structured language, you generally build some kind of data structure to deserialize responses into. With REST endpoints, each response can be translated to a specific predefined object in a client library. GraphQL seems to, by design, ignore this kind of usage in favor of the "heterogenus untyped dictionary responses"... do I really have to write custom data types for every single different query?
I think get this now - with SOAP/RPC/REST we can establish the interaction contract by explicitly defining inputs and outputs. With that contract, we can draw clearly defined system boundaries, give better assurances to the developer that they are doing the right thing, and improve testing.
You're right - in the scale of app-level problems, GraphQL is just a low-level protocol. It requires language-specific libs to address these higher-level problems (provide interfaces, structs, query typing, etc). As a very basic example, programatically building a GraphQL query is unnecessary complex outside of JS because many languages don't yet have a QueryBuilder for the GraphQL syntax. There are some community efforts, but as of yet no heavy corporate players to drive GraphQL-specific dev (e.g. like Pivotal/Sensio/Sencha did for their respective fields).
How would it be different? This article could be retitled "Is Scala or Node the best option for building a backend server", without any other changes, and it would cover both use cases (plus plain old html server etc).
> I am yet to experience an enterprise customer that has jumped into Git.
My company recently did (1000+ employees), though from what I've seen, many of my coworkers familiar with only svn are still using git like svn. We also have a lot of legacy code still within svn (various reasons outside of my team's control), so the transition is pretty messy IMHO.
> JVM provides a great deal more APIs for creating and handling threads and processes. Not something you would typically do directly in many webapp backends but there if you need it.
You might not need it directly, but under the hood it allows each request to run in its own thread, which eliminates the need for asynchronous callbacks that pollute any node.js code. The absence of async callbacks not only makes the code cleaner, but also means that when you're debugging and hit a breakpoint, you can follow the call stack all the way back to the request entrypoint.
"Asynchronous Libraries" is listed as a pro for Node, but unless you're doing something very IO-intensive at 10K req/sec, asynchronicity is not going to matter one bit on the JVM since it's multithreaded. Going from sync to async at 10K req/s will save you a few MB of RAM, that's all. Assuming you've got that few MB to spare, it won't change your throughput or responsiveness at all. It's a premature optimization that makes coding and debugging more difficult. I'd rather state this as "Needs Async Libraries" as a con for Node.
Generally true. Though, in context of webapps, each thread usually works on one request. If your threads share (mutable) states in that case then something is VERY broken.
Kind of, but consider logging. In node you can just console.log. In JVM you have to use a multi threaded logging library or your logs could be jumbled. Lots of things like that.
> The absence of async callbacks not only makes the code cleaner
I agree async callbacks are not very pretty. However, neither is dealing with shared state between threads, or doing thread management in general. Personally I will take the ugly callbacks over buggy/unsafe thread management any day.
> The absence of async callbacks not only makes the code cleaner, but also means that when you're debugging and hit a breakpoint, you can follow the call stack all the way back to the request entrypoint.
Look for the "async call stack" in the v8 debugging tools. This lets you follow async callbacks around like a normal call stack.
> I'd rather state this as "Needs Async Libraries" as a con for Node.
Again, having a single-threaded run loop avoids entire classes of problems (like shared state), which is a huge plus in many cases.
That's the beauty of it though; the JVM web server does the thread management so your app code doesn't have to deal with it at all.
> avoids entire classes of problems (like shared state)
Shared state is an orthogonal thing though. You can have shared state in Node or go stateless in Scala. You can have race conditions in either. Node prevents the simple `shared.x = shared.x + 1` problems, but can't ensure anything across callback boundaries. Scala attacks this by making everything immutable by default. Either way, there shouldn't be shared state in a REST app anyway.
> "async call stack"
That is pretty cool. Actually once Javascript gets async/await added to core, then Node could be pretty viable.
I mostly agree. Should almost always be a project specific decision in my opinion. If you know that you will not scale to size XYZ than maintaining a backend based on Scala is just not worth it.
I like the storm of Scala.js comments on the article. It's definitely game-changing for the Scala ecosystem. You can run Scala on node.js now, and by extension you can package desktop apps written 100% in Scala.js using Electron.
I'm just starting to fiddle with it, but Scala.js on both the backend and frontend seems like a great combination – lets you use the same language and packages throughout your application, and frees you from the JVM.
Building cross-platform apps with the same codebase is gonna be ugly one way or another. This particular method is interesting to me personally because I'd rather depend on node.js than Oracle's JVM.
It's worth mentioning that Scala might get a native backend soon, partly built on the same infrastructure as Scala.js – that could give you a migration path from node.js/Electron to a truly native app with a simple WebView for UI. See https://www.reddit.com/r/scala/comments/4ftcx4/scala_native_...
I am not sure what makes you trust node.js (V8) more than the JVM. The JVM has been in solid for a long time, has higher performance, and more stable. The JVM GC is very good and highly-tunable.
Either of those technologies is good enough technically for my purposes.
However, I have much more confidence in node's new governance (and therefore its bright future) than in Oracle. Oracle essentially has a monopoly on Java. And they're not nice guys. I don't feel comfortable building a cross platform application on Java no matter how much momentum it has gained historically.
The Java ecosystem is fairly complex and it's hard to understand. Your statement of "Oracle essentially has a monopoly on Java" is not true, considering Java language is used to built Android apps, runs on diff VM and uses different bytecode -- outside the control of Oracle (about 2M andorid apps and 1.6B phone sold). Another thing to consider, the JVM is an open spec and most of it is open sourced, see OpenJDK. There is also many implementations of the JVM, see; https://en.wikipedia.org/wiki/List_of_Java_virtual_machines
Indeed, I don't have a good understanding of the JVM ecosystem. However:
Google is tentatively on the hook for $9B for... essentially implementing its own JVM... or something. Doesn't matter to me, end result is that the future of Java on Android is far from clear. They're allegedly looking into using Swift as the next generation Android language.
RoboVM, one of the better / more popular ways to write iOS apps in Java was close-sourced, then acquired by Xamarin, then Microsoft, then shut down.
Oracle charges licensing fees for using JVM in embedded devices and kiosks.
To summarize, there's an incredible amount of tools and libraries for Java, but I don't like where Java's mobile story is going in general.
As far as I know the number of viable JVM machines you could build a business on is maybe... 5 at most? They're all experimental and/or abandoned.
Meanwhile, node / JS / ASM.js / Emscripten and friends are only getting better, faster and more widespread outside of the web.
The reason I'm going with Scala.JS is to have UI code shared by web, mobile and desktop platforms. ScalaFX needs JVM for UI code, so it can only run on desktop and possibly Android.
50 comments
[ 3.2 ms ] story [ 110 ms ] threadWorking with big lists and manipulate them is definitly faster in scala than in python, node, etc..
I'm using it for just a web API, and I've had several scala developers tell me to use akka-http instead because it is simpler. I tried, but the route api is quite horrid in comparison to a simple file-based routing description, and the utilities (like the web service client) were a nightmare to work with and debug. And to top it off, the performance was still worse than play. Maybe all of that will get better, but as of right now, Play is hard to beat.
My only complaints so far:
1) Slick is bloated and kludgy and still produces bad SQL queries...I've resorted to using the plain SQL interface which is nice, but not great.
2) It could use some support for API description languages like swagger.
3) Testing your code using the dependency injection framework has been very unintuitive (I'm pretty new to dependency injection frameworks though, so maybe it's just me).
4) Action composition has been a lot harder than expected (for example: http://stackoverflow.com/questions/36683676/play-action-comp...)
5) I wish I didn't have to deal with an opaque SBT plugin, and that the project just used a regular SBT project structure. I realize there are reasons for needing a plugin (like codegen for routes, etc.), but at least document how to work with the plugin better. SBT is like working with a black box as it is, no need to make it worse.
Have you tried Spray? It still uses akka for actors, but I consider it easier to use than akka-http.
http://spray.io/
You could check out quill [1]. Haven't used it in a real application yet but the codebase seems easier to understand and the sql it produced seemed reasonable. virtualwhys could probaly tell you more - if I remember correctly he is a contributor for quill (and was/is (?) a longtime user of scalaquery/slick).
[1] http://getquill.io/
On (2): There is a number of Play Swagger plugins (e.g. https://github.com/swagger-api/swagger-play), each with different approaches.
On (3): DI can actually be quite practical. Use different implementation via configuration (bind at startup), use fakes for testing, benchmark two implementations before swapping them, etc. Develop to an interface, not to an implementation :)
PS. I have a Play 2.3 app already in production, 2.4 currently in development, and plan for a new app on 2.5. I'm upgrading from the oldest one first...
>LinkedIn is a prominent user. Their entire mobile stack is built on Node.js. They went from running 15 servers with 15 instances on each physical machine, to just 4 instances – that can handle double the traffic!
>eBay launched ql.io, a web query language for HTTP APIs, which uses Node.js as the runtime stack. They were able to tune a regular developer-quality Ubuntu workstation to handle more than 120,000 active connections per node.js process, with each connection consuming about 2kB memory!
I feel like the same thing is happening now with GraphQL versus REST. On some level it seems like GraphQL is altogether better than REST, but we don't see a mad rush to adopt GraphQL, because it represents a big shift in thinking, and those of us who just spent the last 10 years creating RESTful interfaces, we are a bit slow to throw out all of our work and start over with GraphQL. Also, the frontend needs to change as much as the backend, so it requires a full re-write of everything.
All the same, a title like "Scale vs. Node.js as a RESTful back end server" does strike me as an anachronism in 2016. Who cares? I doubt I will ever again build a RESTful interface. Rather, on the project I'm working on right now, we are getting rid of the RESTful interface and we are looking at our GraphQL options:
https://github.com/chentsulin/awesome-graphql
Is Scala or Node.js the best option for building a GraphQL server? That would be an interesting article.
You're right - in the scale of app-level problems, GraphQL is just a low-level protocol. It requires language-specific libs to address these higher-level problems (provide interfaces, structs, query typing, etc). As a very basic example, programatically building a GraphQL query is unnecessary complex outside of JS because many languages don't yet have a QueryBuilder for the GraphQL syntax. There are some community efforts, but as of yet no heavy corporate players to drive GraphQL-specific dev (e.g. like Pivotal/Sensio/Sencha did for their respective fields).
My company recently did (1000+ employees), though from what I've seen, many of my coworkers familiar with only svn are still using git like svn. We also have a lot of legacy code still within svn (various reasons outside of my team's control), so the transition is pretty messy IMHO.
You might not need it directly, but under the hood it allows each request to run in its own thread, which eliminates the need for asynchronous callbacks that pollute any node.js code. The absence of async callbacks not only makes the code cleaner, but also means that when you're debugging and hit a breakpoint, you can follow the call stack all the way back to the request entrypoint.
"Asynchronous Libraries" is listed as a pro for Node, but unless you're doing something very IO-intensive at 10K req/sec, asynchronicity is not going to matter one bit on the JVM since it's multithreaded. Going from sync to async at 10K req/s will save you a few MB of RAM, that's all. Assuming you've got that few MB to spare, it won't change your throughput or responsiveness at all. It's a premature optimization that makes coding and debugging more difficult. I'd rather state this as "Needs Async Libraries" as a con for Node.
Going multi-threaded and dealing with semaphores and mutexes also makes coding and debugging more difficult. It's all about trade-offs.
Also, remember that you can have unexpected re-entrancy at any callback boundary in the fully async model. It is not a panacea.
I agree async callbacks are not very pretty. However, neither is dealing with shared state between threads, or doing thread management in general. Personally I will take the ugly callbacks over buggy/unsafe thread management any day.
> The absence of async callbacks not only makes the code cleaner, but also means that when you're debugging and hit a breakpoint, you can follow the call stack all the way back to the request entrypoint.
Look for the "async call stack" in the v8 debugging tools. This lets you follow async callbacks around like a normal call stack.
> I'd rather state this as "Needs Async Libraries" as a con for Node.
Again, having a single-threaded run loop avoids entire classes of problems (like shared state), which is a huge plus in many cases.
That's the beauty of it though; the JVM web server does the thread management so your app code doesn't have to deal with it at all.
> avoids entire classes of problems (like shared state)
Shared state is an orthogonal thing though. You can have shared state in Node or go stateless in Scala. You can have race conditions in either. Node prevents the simple `shared.x = shared.x + 1` problems, but can't ensure anything across callback boundaries. Scala attacks this by making everything immutable by default. Either way, there shouldn't be shared state in a REST app anyway.
> "async call stack"
That is pretty cool. Actually once Javascript gets async/await added to core, then Node could be pretty viable.
Why do you think a Scala backend can be more trouble than it's worth?
I'm just starting to fiddle with it, but Scala.js on both the backend and frontend seems like a great combination – lets you use the same language and packages throughout your application, and frees you from the JVM.
It's worth mentioning that Scala might get a native backend soon, partly built on the same infrastructure as Scala.js – that could give you a migration path from node.js/Electron to a truly native app with a simple WebView for UI. See https://www.reddit.com/r/scala/comments/4ftcx4/scala_native_...
However, I have much more confidence in node's new governance (and therefore its bright future) than in Oracle. Oracle essentially has a monopoly on Java. And they're not nice guys. I don't feel comfortable building a cross platform application on Java no matter how much momentum it has gained historically.
Google is tentatively on the hook for $9B for... essentially implementing its own JVM... or something. Doesn't matter to me, end result is that the future of Java on Android is far from clear. They're allegedly looking into using Swift as the next generation Android language.
RoboVM, one of the better / more popular ways to write iOS apps in Java was close-sourced, then acquired by Xamarin, then Microsoft, then shut down.
Oracle charges licensing fees for using JVM in embedded devices and kiosks.
To summarize, there's an incredible amount of tools and libraries for Java, but I don't like where Java's mobile story is going in general.
As far as I know the number of viable JVM machines you could build a business on is maybe... 5 at most? They're all experimental and/or abandoned.
Meanwhile, node / JS / ASM.js / Emscripten and friends are only getting better, faster and more widespread outside of the web.
http://www.scalafx.org/
Now you can choose the language you like and transpile to Javascript.
I like it though. Especially as Scala and Scala.js are phenomenal :)
A better comparison would have been: Vert.x vs. Node.js