Ask HN: What tech stack would you choose for a serious, long-term business app
If you were about to start building a very vital brand new application for your company that needs to be maintained long term what tech stack would you choose? Put aside what devs at your company may already be comfortable with and lay out what you think is best.
When I ask tech stack I'm talking the whole dam thing, for example:
DB engine : ie MySQL, Postgres
API layer : ie Node w/joy, Java w/spring, PHP w/laravel, Elixer w/phoenix
Frontend SPA : ie React, Vue (do you use typescript or flow, webpack etc)
Caching : ie Redis, Mongo
PaaS : ie AWS, Google, Azure
Interested to hear your thoughts, and don't want any flame wars stating x is always better than y etc.
Feel free to just specify the stack but better to also back up why you think it is best in a few sentences.
79 comments
[ 3.7 ms ] story [ 132 ms ] thread2. Any serious business system (not just one app) will have multiple tech stack.
3. In the long term tech stack has to change depending on the changing business needs.
Assuming a whole lot about above 3 points, choice could be postgres, python, react, aws. Purely because they are battle tested. Any discussions about their merits are moot without knowing requirements.
For serious long term business apps you want to use multiple technologies that all integrate with each other, where each are used for the tasks they suit best.
Real answer: it depends. You've told us nothing about your application's business requirements, operating environment, or staffing availability.
I'd also caution on the idea "put aside what your devs are already familiar with" - different tech can give you different velocities, but either you accept a much lower velocity for a long initial period while your team spins up OR you accept a significant recruiting cost to hire new devs & teach them enough about the business to make good decisions.
First point: nothing Microsoft. Done that show for 20 years and been through all the marketoid schizophrenia promoted rewrites for no business value. Now the new "start again" is .Net core and a big chunk of stuff just isn't portable and never will be. Plus the price gouging, licensing hell etc when you have to scale up. Entire ecosystem is a wasteland of unmaintained crap and automation prospects are terrible. Forget it at this point. I'm done.
At this point, I'd pick Java + Spring + Postgres + CentOS. Does most of the stuff you need out of the box, good support available, long product lifespan, well maintained, possible to automate everything. No SPA, just old fashioned dynamic pages with light weight in house JS framework that does partial page replacement, simple validation etc via DOM. Not even jQuery which has been nothing but hell to upgrade so I'm loathed to pick anything more complicated.
PaaS: none. Either in house or old fashioned colo, with HP hardware. If you have to go PaaS, then AWS which seems the most glacial in deprecation so far.
Edit: also desperately avoid any non portable abstractions and vendor specific features. They will ruin you. For example when someone suggests a database engine, hide it behind hibernate and only use the hibernate API!
Edit 2: it is important to assign an exit cost for every decision as well. What's true now is not going to be in a decade.
Interesting thing is this stack was doable probably 13+ years ago, so I see your emphasis on stability.
Anyway, it's similar to what I'm using and surprised to see it here. Biggest tweaks are that I'd consider Vue/Webpack if starting today and do believe PaaS to be far superior to on-prem/colo. Also, not big on Hibernate or too much abstraction at the data access layer.
I've never used vue/webpack so can't comment. I've been shot too many times by javascript framework churn over the years. I wrote a relatively small and powerful abstraction over DOM that doesn't hide it away, merely makes the hard things easy and less verbose and allows simple templates to be used.
Hibernate is essential if you ask me. One of the problems I see a lot is relying on features of the underlying storage engine. This, while it has advantages, has more disadvantages such as coupling and vendor lock in. These have been significantly costly when you have to scale something commercial up like SQL Server.
That is, I would advocate PaaS and IaaS over on-prem/colo, and how far you go towards PaaS is a matter of comfort.
I do think that comes down to control though, and I actually can't think of a much better basis for the decision.
No expert on the terminology here, though. Just how I understand the term.
I have actually tried to get things fixed on MSFT open source projects as well and even when you get up to Scott Hanselman poking it on github, nothing tends to happen. On top of that many crippling bugs remain unresolved for so far 9 years. We have unreliable tooling, portability problems, all sorts.
On a larger team and product, it's a non trivial time and cost sink.
What sort of projects did you have the most trouble with? Where were your pain points?
Pain points: large project performance, high concurrency projects, IDE reliability, cost, odd time sinks, bugs in the framework, lack of automation possibility for a vast number of project types (try building VSTO applications on a CI server), deprecations (WWF/WCF changes, AppFabric, Velocity etc) all sorts.
On tiny projects it seems fine but we're huuuuge.
In contrast I'd like to offer our competing ServiceStack framework as an alternative (commercial) platform [1] which has been in active development since 2008 that in contrast to the ever changing MS frameworks offers a single integrated Web Services [2] and Web [3] solution that has maintained the same clean, stable high-level message-based API throughout its lifetime. It was created out of frustration in the WCF/WebForms era [4] where we were the first to offer an integrated IOC, impl-free/testable multi-provider logging, caching, virtual files and configuration providers, clean/extensible typed user sessions - all using high-performance components. The first to run on .NET Framework and Mono, the first to create high-performance JSON and text serializers and fast typed ORM, which at the time they were developed were several times faster than anything MS offered [5].
All pre-dating Web API [6] and outlasting the continually changing fragmented MVC/Web API "framework in a framework" coupling as well as the rolling churn of abandoned OWIN/katana abstractions [7]. Instead we've maintained clean impl-free IRequest/IResponse interfaces which provides a lightweight abstraction over the underlying platforms concrete HTTP Request/Response abstractions where we're still the only framework where the same ServiceStack App can run on classic ASP.NET, self-hosting HttpListener (.NET Framework or Mono OSX/Linux), ASP.NET Core on .NET Core (Win/OSX/Linux) and ASP.NET Core on .NET Framework v4.7 [8].
As a result of our stable surface area we still have older v3 (BSD) Customers upgrading to our latest commercial version just so they can leverage their existing investments to run on ASP.NET Core which is relatively easy to port [9] where most of our .NET Live Demos [10] have been ported to run on .NET Core [11].
Unlike any of MS offerings, ServiceStack:
- Has never abandoned any .NET library, our libraries are continually enhanced and maintained, not rewritten or discarded
- The same Services implementation supports multiple formats out-of-the-box, can be called from multiple HTTP / MQ / SOAP endpoints, on multiple hosting options [12]
- Provides native, end-to-end typed development experience for C#, TypeScript/JS, Java, Kotlin, Swift, F# and VB.NET [13]
- Has no prejudice towards MS products, our libraries work equally well on all platforms/back-end providers, e.g. OrmLite [14] provides a high-level typed DB agnostic API across all major RDBMS's. Whilst SQL Server is our most popular dialect (courtesy of marketing in .NET), we personally prefer/use PostgreSQL and its a popular configuration to configure Unit tests to use SQLite for faster Integration tests.
- Provides clean abstractions of multiple providers for AWS [15] and Azure [16]
- Has its own flexible Auth Provider model [17] which hasn't had a schema change and never needed a "migration" whilst being continually updated to support new providers
- Has an aversion to complexity, instead of blindly adopting over-engineered solutions we develop our own cleaner alternatives. Some examples of this include AutoQuery vs OData [18] Add ServiceStack Reference vs WCF Service Reference [19], Templates [20] vs Razor [21]
- Offers a Simple and "pure Cloud App" solution for developing .NET Core Apps with built-in hot-reloading and without any compilation or builds [22]
From 2013 we've adopted a sustainable dual licensing model (commercial for closed-source or OSS/Free for OSS projects) which has allowed ServiceStack to be actively supported and maintained and continually updated with new releases [23].
Anyway I'd like to offer ServiceStack a stable competing framework for .NET and .NET Core that's being developed from a different mindset/values which has evolved gracefully over the years and stand...
The only desirable feature was Redis sentinel but we can live without that.
Honestly we can just switch parts of the system to another platform for the same cost and get an ROI over a couple of years.
It's a different world with a much improved development model that I find vastly superior compared to traditional classic ASP.NET Framework Web Apps - which I don't see myself ever creating greenfield Apps for again.
Honestly I echo the Java suggestion plus a rdbms. You have a big talent pool, it's well supported and "enterprise stable", and it's multiplatform.
API: Ruby on Rails
Frontend: React, the react_on_rails gem to set up webpack, Flow, Redux, Immutable.js
Background Jobs: Sidekiq and Redis
Caching: Memcached
PaaS: convox (on AWS). See: https://convox.com and https://github.com/convox/rack (But you should start with Heroku.)
I recommend the Rails stack. It's not the "hot new thing", but it's very stable, secure, and productive.
[1] http://www.mikeperham.com/2015/09/24/storing-data-with-redis...
And then you need to look around yourself at the resources you have now and what they may be 5 years from now. There's no use writing something up in a little used language hoping you'll be able to hire for it later and adding risk by writing something in a language or technology only you know in your company is creating an unnecessary bus factor of 1.
A safe place to start would be Linux + Postgres + Java Spring or PHP Laravel.
My experience is the market has more Java or PHP engineers (compared to say, Elixir).
That's a quantity over quality argument. Assuming there isn't an extreme shortage, I'd opt for a language and/or framework that tends to attract higher quality candidates. Personally, Ruby/Rails feels like it hits the sweet spot there.
DB: A hybrid of Postgres & MongoDB
API: Hand rolled w/ SparkJava
Front End: intercooler.js (SPAs blow out your complexity budget, which should be spent on business logic)
Caching: Redis
PaaS: the system should be designed to deploy on anything, including internal servers
For a SaaS app, I would look at rails due to the ecosystem and stability, but otherwise it would remain the same.
This is a combination of things I'm most familiar with and things I'd build shit in if not for the fact that other devs at my company aren't familiar with it. i.e. We use MySQL everywhere, so I can't just spin up a new project using Postgres for the sake of consistency.
Heavily data dependent / ETL applications have different requirements than CRUD applications. There is not a one size fits all stack, you need to consider the user cases.
For example:
- We use DynamoDb and ElasticSearch on the back-end because our case is very key-value oriented but also needs faceted search in some use cases.
- We use Node.js for our user-facing REST API because it was fast to develop and quick to iterate on but we use GoLang for our internal gRPC APIs and front-end because in our case those are the hot paths and where the performance differences between node and Go make a difference.
For an activity feed API (hundreds of thousands of activities a day) I've used Redis with huge success.
The point being is if your app is truly enterprise you'll probably have more than one microservice, and you should pick the right stack for each microservice.
In absence of a hard business requirement that is significantly different between languages choose the framework your team can develop the fastest in and get to market fastest (like our hot path example).
My personal go to is...
Backend: Postgres, Redis, Elasticsearch, DynamoDb
Front-end server and APIs: Node.js with express (no framework), focusing on as few lines of code per microservice as possible
OS: Definitely Linux
Cloud Provider: AWS
Cache: Cache close to the app when possible, if you don't need cache to be synchronized across machines there is no need for a central cache. With that said, I personally love Redis
On the cloud platform note, I'm an AWS guy but they all have their benefits. I try to use as many AWS services as possible to decrease dev time / costs but write the code in such a way where it is abstracted so getting away from vendor lock-in would be possible (painful but at least possible).
All the cloud providers have a service catalog that can shorten your development time if you take the time to learn them use them right. I happen to like AWS value added services best.
With all that said:
- The most important things you should worry about is getting to market and solving the customer's use case. The stack is not important in 90%+ of cases. If you structure as microservices you can always replace individual services cheaply on a case by case basis once you have customers.
- Second, "When I ask tech stack I'm talking the whole dam thing" is very unprofessional and I almost didn't answer because of that. I suggest perhaps an attitude adjustment when asking the community for free consulting advice.
Backend/API layer would be one or more of Ruby, Python, or Java, depending on what my team felt comfortable with. Large, mature communities, healthy ecosystems, overall decent tooling, good support for testing and deployment automation.
Nothing Microsoft. Not going down that road again if I can help it.
AWS hands-down. Google tends to have this pathological distaste towards providing support for their customers, and I don't trust them to not just shutter any of their non-core products randomly one day.
I am very sold on GraphQL or Protobufs as the interchange format over REST.
Databases, hard to go wrong with PostgreSQL. Redis and Memcache for caching.
Long term maintenance is easier if the tools are relatively stable. I think that argues against most (all?) Javascript frontends. Eg, if you started this 3 years ago, your current JS devs would want to throw away your entire JS framework and tooling stack. Odds are if you do it today, they'll want to do the same in < 3 years.
For minimum maintenance issues, I'd pick PostgreSQL for db (data integrity FTW), Redis for caching, and do a server-rendered app.
Of the server-side frameworks I know, Elixir Phoenix is my favorite, but Rails is older and arguably changing less. Probably some Java framework is even more crufty / stable?
Of course, there are other concerns besides stability that might argue for different choices - eg, using Elixir Phoenix means that a lot of your caching would be unnecessary, because the dynamic parts of view rendering are "cached" in compiled functions and the responses are very quick without further caching. And you'd get richer experiences with an SPA, in exchange for a lot more work. But you highlighted maintenance.
Go with or without any framework (like Buffalo)
Vuejs with Vuex
More important than technologies is testing. If your whole app is well tested (with both unit and E2E tests), you won't be reluctant to upgrade some part of your system because it doesn't answer your needs. Moreover, it would bring a lot of reliability and would be self-documented. That's the biggest point IMO.
For nowadays mood, we are really glad using React, Node, GraphQL and PostGreSQL for all our apps.
Note: we wrote a blog post about one of our long running customer story, which may interest you: https://marmelab.com/blog/2018/03/28/1859.html
http://www.ukessaypapers.co.uk/
1. use something that has been around a long time and is well supported/known
2. you probably don't need stuff like cassandra or other "big data" data stores (just yet). PostgreSQL/MySQL is plenty fine to start with (and easier to manage/think about).
3. please use only a single cloud provider.
- DB Engine: MariaDB on RDS (multiple instances) + Redis for specific use cases. - API layer: Symfony (PHP) - Frontend : React - Caching: Redis - Ops: Rancher Orchestration on EC2, and a bunch of AWS features like S3, ALB, ...
We had some hiccups here and there, but I'd say we are happy with the stability of it.
DB: dynamodb
API: API Gateway and AWS Lambda
Frontend: angular 2+, or React
Cache: AWS Elasticache
Blob Storage: S3
At your peril. Sorry, but it's true.
Any time there is something even slightly amiss it's going to be "Uh oh - looks like Mongo is giving us trouble again."
No matter what the real problem is a slam-dunk scapegoat. And your name is going to be forever attached to it.
Also consider how you're going to deploy & maintain the stack itself. Do you have an ops team that's familiar with deploying war files into a servlet container? Well now on the critical path to this thing going live is them learning how to maintain a new app server, language and deployment process?
That said, at the end of the day - all the tools, frameworks, and languages you mentioned are just fine, with some slight nuance:
DB: MySQL is fine, Postgres has some nice additional features if you need them: GIS, great scripting support, etc
API/App: Stick with the big ones (Elixir? C'mon...). Slight edge to Node because JSON de/serialization is automagic. Slight ding to PHP because... PHP.
Frontend: Who cares? It's going to be outdated in 6 weeks anyway.
Caching: Which ever one you don't have to maintain or think about. Did you choose AWS? Great. Elasticache supports Redis and Memcached.
PaaS: Whichever one supports all the features that you need.