HN Explorer

Ask HN: Python vs Scala for a fast web-project

9 points by ajakhu ↗ HN
I was working on a web-project lately with a friend. We faced a situation where we realized that Python was being a bit slower than expected. Also, we were planning on using a database that was not native to Python (Python bindings used its REST interface).

So, we had to make a decision to either keep going in Python or port to Scala (Scala seemed apt for many reasons).

What would you have done in this case?

14 comments

[ 1.9 ms ] story [ 39.8 ms ] thread
gelnior
If you are comfortable with Scala and think it fits well for your project go for it. There are some frameworks like Lift or Play with which you can start quickly a web app.
ajakhu
Yes, we were planning for Scala/Lift only.
gaius
First profile the Python app - is it being slow in Python or is it waiting on something else? If it is being slow in Python, can you rewrite the critical section in C? And speaking of which, is there a C API to your DB that you can use from Python? The Python mentality has always been that it is easy to interface Python and C where necessary for performance.
pestaa
Or better yet, have a look at Cython or PyPy. Both give a performance advantage over the standard intepreter.
ajakhu
The API problem will still exist even if we switch to PyPy or Cython.
ajakhu
No, there isn't a C API for the DB. So, that put C out of the picture for us.
gaius
Out of curiosity, may I ask which DB this is?
playing_colours
Did you have any experience doing web development with Scala? I am now choosing stack for my new project and considered Scala/Play as an option. There are some problems both with Play2 framework atm(no Wax deployment support, long rebuilding after small changes, work with actions is confusing) and the language of Scala (complexity, version migration, SBT etc.) The situation is promised to get better in future but most need their work done now. If you are choosing bleeding edge, new shiny thing be ready to face additional problems which can slow your development. If you prefer just to get things done you'd review your Python code, find and fix bottlenecks. Maybe you can combine techs choosing what is appropriate for some work - say Python for web interface with critical parts in Java/Scala.
ajakhu
No, we didn't have any experience with Scala/Lift/Play. We were aware of the learning curve of scala. But what we thought was in the the best interest of project. I've read about the problems ingoing with Scala, but the benefits outweigh the problems most of the times. And yes, learning a new technology will definitely slow down the development (especially Scala).
manishsharan
Its not the language but your framework and architecture and your own proficiency in the language ( whatever that might be) that matters most for a fast web application.

Your code would spend most of its time fetching stuff from DB, external web services etc and rendering HTML. Does your framework allow you to easily incorporate database caches? Does it allow to to cache frequently rendered HTML segments ? Can you offload certain tasks ( such as sending email etc.) to a secondary process via a job queue ?

You could write a web application in C that could still suck big time unless you get the application architecture right.

ajakhu
We had integrated redis in our project for server side caching. And I'd like to clarify that Python was being slow for regular cases (atleast not as fast as we wanted it to be).

As we were newbies to web development (although we have a lot of experience in development), we had put in our best knowledge while designing the architecture. We kept independent things independent. That's why we chose flask, it gave us the flexibility we wanted. We were free to choose the extensions we wanted to integrate.

petervandijck
If your goal is to launch, stop fooling around with languages and rewriting your code. What exactly was slow?
ajakhu
1) A NoSQL database best suited our project. So, the non-nativeness was a hurdle. Neo4j bindings for Python used JPype, which was rather unmaintained. And I read at some places that it was a bad idea to mix python-java (especially using the unmaintained JPype).

2) Using Flask gave us so much flexibility as a framework. SQLAlchemy was probably the best thing that I've ever seen. But the work was made complex by the fact that Python was not able to give us the performance that we wanted. We tested the web-project running locally, and we noticed some exciting results. Even though we had done proper memory management from our side, the memory consumption was a little high than we had actually expected. I'm not an expert web-technologies, but as a programmer for last n years, I've learnt a lot about memory management. We projected that when the load would be actual, python performance would suffer.

3) We wanted a real-time performance application so that the user experience was the best. We implemented a part of the web-application and tested it. The results were unsatisfactory. Then we planned on shifting to scala.