Ask HN: Is my software stack choice sound?
As a side-project I am developing a simple app to send short messages to subscribed users. The users will be able to respond to the messages either privately or publicly.
It is a simple app, but I am not web or mobile developer. So this will be a learning experience for me.
This is a mobile-first app. I am thinking of what software stack I should use. I have reduced the choices to the following:
frontend: Flutter, backend: REST + Wt + PostgreSQL
Or
frontend: Flutter, backend: GraphQL + Phoenix/Elixir + CouchDB
I am a C/C++ developer and familiar with Wt and little bit of REST and RDBMS. But I am not familiar with Flutter, PostgreSQL, GraphQL, Phoenix/Elixir and CouchDB.
What software stack should I use? I am leaning towards the first option as I am familiar with C/C++.
48 comments
[ 6.2 ms ] story [ 115 ms ] threadEither choice is fine, but pick one. Choose whether you want to learn or earn.
In this case, if you want to learn, pick elixir/graphql. From what you've said, that'll be quite the learning experience.
If, on the other hand, you want to earn (or at least get something shipped as soon as possible so you can test the market), pick the Wt option.
As for CouchDB vs Postgres: "When in doubt, Postgres." Has never failed me. I burned a lot of time on other DB systems[0], but in the end Postgres was always easier for the whole picture of what a DB should do from backups to persistent to extensions, everything really. Never tried CouchDB though.
Anyway, seems like a good project to try out some new tech since it should be pretty easy. Whatever you choose should be fine for your first 100k users.
[0] Riak, Mongo, hand rolled custom graph DB based on BerkleyDB, tried to make Neo4J work, even wrote a persistence layer for NetworkX at one point.
P.S. not arguing - genuinely would like to know, since it never clicked.
P.S.S. Definitely use Postgres - "When in doubt, Postgres" is a good axiom unless you need something very specific and know what that is.
P.S.S.S. Elixir isn't that widely used. If you want something simple use FastAPI with Python or Flask. For real projects, .NET Core API will work well, but it's a little slower to set up if you don't use a template. Your C/C++ WILL carry you through C# though. This choice will REALLY depend on what language you want to learn - most of them have good API frameworks.
Instead, they need only agree on the shape of the data available, and the frontend queries it however it needs to be queried while the backend resolves the nested calls with no additional effort* from those writing the backend.
Also, some clients come with additional affordances for hard problems, like pagination.
* okay, fine, you have to wrap your data resolution in Dataloaders, but that's not particularly challenging
But if a team does want graphql without the fuss of openapi, json schemas, and swagger docs, I recommend tools like Hasura that autogenerates graphql apis over my db. Auto generation of graphql resolvers are the way to go for productivity. Supabase even created a Postgres extension to perform graphql directly within postgres https://supabase.com/blog/2021/12/03/pg-graphql
Say you have a REST interface for Student and Class.
What do you do in rest if you want to get just a class? What do you do if you want to get a class and its students? What do you do if you want to get a Student and also its class?
For every iteration of the way you need to access the data you have to modify and extend your REST endpoint or do separate queries. Every time the frontend devs want new data related to existing data, they have to do another query or ask the backend to include it.
Or lets say Class has 1000 attributes. Does your REST interface return all 1000 every time? Or can the frontend specify which attributes to load? Things like computed attributes and functions that return data related to the object. GraphQL allows this.
In graphql doing these relationships is dead easy. You define a class and how a student is related to it. Then you could even ask a query like “give me the classes of a student. For each class give me all the students in those classes. For each of those students, give me their classes”. Without the backend having to add anything. All the backend has to do is define the relationship and permissions about how to access the data.
Most GraphQL libs provide helpers for avoiding n+1 queries also, so things get optimized in the above case by only making 4 queries to the DB. Obviously thats a contrived example and most nested things wont go so crazy. But it goes to show how a powerful the API can become from defining such a simple relationship.
As a backend dev you just have to work on transforming data, defining the relationships and checking permissions. You let the frontend determine what data they want to access.
Before working with REST almost every page seemed to need a specialized endpoint to be implemented for the frontend to work, but with GraphQL, the frontend can work much more independently without having to ask the backend to provide a specialized view for more data.
That is to say, what differentiates GraphQL from SQL?
I can write a Graphql endpoint in python (Using Django-Graphene). I can write python functions that return data to the graphql. I can write ORM queries that write data to grapqhl. I can specify access, permissions and certain queries based on the user.
Graphql is more of a relational FFI than a SQL replacement.
Also the frontend never sends SQL to get data because it is unsafe. This is what a GraphQL framework allows you to do, clean up and filter queries so they are safe and then translate them into efficient lookups in a RDBMS.
That's not even getting into the tremendous attack surface you've just opened leaving your application vulnerable to SQL injection attacks. SQL injection is a real problem and scrubbing all the inputs is a pain and you can never be sure you got everything. One mistake and congratulations! - you've made headline news!
These are the problems GraphQL solves.
CouchDB is a json document based noSql system that is exposed through traditional HTTP calls (e.g. GET to get a document, POST to create a new document, PUT to update a document, and DELETE to delete a document).
In the "expose data via REST" the stack is often "service doing SQL against a RDBMS" and that service is a either Java or Python or Ruby... that can often (not always) be done as "here is couch, with this data stored in it." It's memory and CPU footprint (even doing a 3x replication in our k8s environment) is a fraction of a Java container, much less a Java container and a database.
Grab a couchdb docker compose and spin it up and give it a try. It has a built in admin console. https://docs.couchdb.org/en/3.2.2/index.html
https://docs.couchdb.org/en/3.2.0/ddocs/ddocs.html (note the design document for the users database)
This isn't the right type of solution for all data - but there's a lot of situations where people have a bug lump of data that they're trying to expose... list of all current flights, or classes, or ongoing games.
It can be quite useful for a lot of use cases. One size never fits all.
If you're just making this because you're excited about new tech and want to see what it's like to work with, then by all means go with the more experimental stack but understand that it will take longer and will be riskier.
With that being said, I believe it might be quite a learning curve for you. Not because it will be difficult, but just because there might be many topics at once to learn. Like you said, GraphQL, Postgres, Elixir (and a little bit about the BEAM ecosystem), Phoenix and CouchDB. If you are doing anything for the Web, you will inevitably touch JavaScript as well.
Not sure if you are open for suggestions, but one thing you could consider would be using JavaScript or TypeScript with NodeJS. I'm not the biggest fan, but you would be able to use the same programming language in the frontend for our Web app and also the backend with NodeJS and a full-stack framework like Remix[1] or NextJS[2]
This could reduce the overhead to start your side-project and will give you fewer puzzle parts to learn and put together.
[1] Remix: https://remix.run/ [2] NextJS: https://nextjs.org/
From my perspective I wouldn’t choose CouchDB. For one, its a pretty obscure DB that doesn’t have a lot of support. For example almost all major hosting providers now allow you to launch an app with Postgres hosting out of the box.
Way way way way more tools and articles for building an app with postgres.
Also I would recommend Postgres to take full advantage of Phoenix and Ecto (the ORM). In Elixir GraphQL and Ecto are deeply integrated to avoid n+1 queries when you are writing nested lookups.
In my opinion having used Django, RoR, frameworks in Go and Rust, Elixir has the most robust and powerful ORM out of any language and it would be a shame to miss out on it by using CouchDB.
I prefer GraphQL over REST tremendously. For a large site, in REST we had this process of the Frontend would have to tell the backend what data they wanted, the backend would make the endpoint and then the frontend would consume it.
With Graphql the backend just defined all available data and the frontend picks and chooses what it needs. GrqphQL also prevents weird n+1 requests where you need to request the data of the children from a previous REST result.
GraphqL + Phoenix + Postgres is a game changer IMO
Flutter is a good choice if you know it, but it’s terse and barebones, and honestly you’ll grok JS a lot easier coming from a C background.
If I were you, I would go for REST/Wt/Postgres on the backend and JS with Cordova + some UI library (Ionic or Framework7?) for the frontend.
Its also not even much a paradigm shift in my experience. Queries are simply analogs to GETs and Mutations are analogs to POSTS. Thats about all you need to know to get started. To convert a REST app to a GraphQL app is ridiculously straight forward. What it does much better than REST is nested lookups, being typed, and being able to request more than one piece of data at a time.
Of course, in the long run, this is reversed.
REST might be easier to get up and running with too, because Phoenix does it out of the box. You could then move to GraphQL iteratively once you have your core logic in place and the Flutter app working.
Overall I think you find Elixir refreshing coming from a language like C++. The tools all work pretty well and are easy to learn. The REPL is nice. The package manager is world class. Testing is built into the language. The developer experience is just really great.
This post sold me on it https://blog.wesleyac.com/posts/consider-sqlite
Phoenix comes with its own database tool called Ecto[0] which is excellent, and it uses Postgres by default. If you're not intended to leverage CouchDB for offline support you should go Postgres without a second thought.
That said, I'm also curious about how to implement offline support with Phoenix in a nice and trivial way.
[0] https://github.com/elixir-ecto/ecto
Yeah, that is the intention. Have not checked if PouchDB can be used with Flutter.
If I over-spend my new stuff budget, I take on debt, much like tech debt, that I’ll have to pay down later. I’ll make slower progress because I’m trying to learn too many things at once, and I’ll have to go back and fix suboptimal tradeoffs where I didn’t have enough experience to make the right tradeoff at the time.
Conversely, if I don’t spend the new stuff budget at all, then I won’t learn anything new and might miss out on a tool that would have made the problem easier.
For side projects where the whole point is learning, sometimes I’ll spend my new stuff budget by e.g. learning a new frontend framework first, writing the backend in something I know. Then once I’m familiar with everything and have built back up the new stuff budget, I’ll go back and reimplement the backend in the new language I wanted to use, leaving the rest of the stack unchanged.
I'd do the backend with Django and Postgresql. You can't go wrong with Postgresql, it's an industrial strength database. Nothing wrong with MariaDB (or Mysql for that matter), either, I just prefer Postgres.
Writing your backend in Django will allow you to get the basic functionality working in an afternoon. Don't worry too much about getting all fancy with the user models at least to begin with, or any "clever" stuff. Don't worry about doing it at Google scale until you're half way towards Google traffic.
You can use generic views in Django to rough out a web UI for it, then make it as fancy as you like with Flutter.
I find in my personal projects using python backend (flask or django), with a postgres database almost always leads to the most useful work getting done quickly. There are a lot of ways to make python fast if you know C++ when you need it.
I'm not familiar with Flutter, no opinion there.
For quickly launching an API enabled backend, maybe look at encore.dev? It's not one of your listed languages (it's go), but at least it looks somewhat like C.
I recently had to launch an API for a hackathon, and this was as fast as Ruby on Rails for me. Comes out of the box with Postgres and PubSub (2 lines of code if you need it). Encore also does the infra provisioning and hosting for you (either your cloud, or for free at their cloud). Tip: add `sqlc` for the queries, it works well together with Encore.
GraphQL/REST: it looks like your app will have a limited set of calls that are easy to predefine. In that case, REST wins hands down. IMO GraphQL really shines if you have (a) a data model with many relations, (b) possibly from different underlying systems/services, and/or (c) different people working on the front-end/consumption side vs the data exposure side. But that's beyond 'a simple app'.
Other tip: if you're thinking of building an admin interface. Don't, just use something like Retool first while you're getting users.
How is this done? Any documentation, tutorial or example that you can point to?
> this allows you to deploy fixes and basic updates without going through the app store review process.
Does Apple and Google allow this?
In my experience, the one time I used CouchDB I found it so flakey that I recommended we refactor to use a normal SQL database. Of course, a lot has changed; but:
The big power of a relational database is that it's extremely easy to just tweak your indexes and get views of your data that you didn't anticipate when you created your schema. Document databases like CouchDB (and MongoDB) require that your data is organized in a way similar to how you're going to query it.
The other power of a relational database is transactional integrity. You can update multiple rows in different tables, and the update either happens or it doesn't. Updating multiple documents in CouchDB (or MongoDB) doesn't have the same guarantee, meaning that your data can be corrupted by an incomplete update.
Most of the techs you mention are immense, humongous beasts. I'd say drop them all.
If you really, really need an "app" instead of wrapping a simple website I'd go for React Native to skip the native stacks and something easy on the backend because it will not matter in the slightest what you use: Python/flask, Node, Go. Whatever floats you boat, but keep it simple. (I think you may like Go as a C programmer.)
Database? SQLite or PostgreSQL.