Ask HN: What is the future of back-end development?
Apparently, WebAssembly will revolutionize the client-side development. The web apps will be very much like desktop ones: high performing, can be written in any language. At the same time they are cross platform, can be easily distributed, and do not require installation.
Meanwhile, every "builder of web3.0" who's trying to "fix the Internet", or "make the web faster, safer, and more open" is doing it in such a way that back-end development is not required. I.e. they are working on decentralized (either p2p or federated) universal back-ends that they assume should be used by everyone.
Do you believe that in future the back-end development as we know it today will be obsolete as everybody will use some p2p BaaS platform and the web-development will be all about implementing client-side apps?
163 comments
[ 1544 ms ] story [ 3585 ms ] threadStartup cost on server side apps is big O of 0. The average JVM for most apps run from maybe 1 hour (continuous deployment shops) to 1 month (slower to release shops). 3 seconds to start the JVM? Does't matter in either of those, don't server traffic until it is done.
Dalvik's major advancement was faster class loading.
The JVM is an extremely fast system and if you run java on top of it that is fine. There is a reason so many of the world changing distributed systems (Cassandra, Kafka, ZooKeeper, etc) are built on top of it.
I believe what you are thinking of our java business systems which are usually bloated/slow and developed by similar developers who think backends can be automated.
There is trade-offs on both cases, if you can optimize for developer speed and sacrifice app performance that might be fine for your use-case. Software applies to pretty much every industry and not everyone needs scale but not everyone can live with poor performance either.
You're wrong.
(See, I can just drop off one line unjustified comments too, but I wont~)
WPF native applications are failing against electron applications; slack, atom, vscode. If you think that because it runs in a browser it'll be slow, try writing a native windows 10 application sometime. You'll be both astonished and disappointed by how terribly slow the 'GPU accelerated' application is.
All web assembly does is cement this desktop dominance from 'native' applications and allow people to start pushing realistically into the mobile space with react native, etc.
You don't think it'll revolutionize anything? Haaaa~
I write Windows 10 applications for a living. I've never had any performance issue...
And this is for a dead simple little setting.
Put bagel in toaster.
Update the db model.
Add a migration.
Update the viewmodel <==> db model translation in relevant requests.
Get bagel and butter it.
Commit.
Test the migration on a staging db if feeling paranoid, cause it's just adding a column.
Push to master.
Update production.
Take last bite of bagel.
If you aren't also doing the frontend, tell the guy who is to check out the updated swagger docs.
We're also not doing the common website/webapp thing, but rather on-premise enterprise software. So we have to update installers too, and be relatively paranoid about updates, because it is intensely painful when we have a bad update with one of our customers.
On our less important software that'd just be a direct push to production though.
This kind of mentality is why most applications are shit and broken. And why most hires are worthless.
And everyone is not working on a cookie cutter over-bloated rails website. Sorry but this kind of remark simplifying our job always pisses me off.
In a system of any complexity, changing your data model is an activity that requires due diligence to mitigate any downstream risk.
We also have throwaway apps like an internal time clock system which monitors when people show up based on their cell MAC.
Latter case: Adding a checkbox is easy.
Former case: Adding a checkbox is easy but we have to test it first.
That sounds miserable.
We started adding in other employees in multiple departments cause they liked it better than manually tracking.
When your tables have millions of records for enterprise customers, it's significantly more involved than you describe
Once your application reaches a certain scale there is a huge chance that generated migration could lock a table for minutes to hours which isn't acceptable.
We allow developers to run generated migrations in dev/testing environments but when its time to go to staging we evaluate the changes made and check query plans and figure out what type of locking is going to happen and how long it would take it prod. There is very few times the migrations Django or Rails generate are acceptable at our scale in production.
For our large scale applications we back up the database and have required down time from our customers while doing a major migration.
Everything isn't always simple. But adding a check box in 90% of cases usually is.
Now lets say you are doing 15,000 requests per second on that table.... is preventing those from executing acceptable in your system? Its not in mine which is why we don't run ORM generated migrations.
You monster. You fucking monster.
The reality is that ORMs are a huge headache and a chunk of dependency and magic that make some people uncomfortable. Even though I make more mistakes writing the glue code between SQL and my backend than I would with the ORM, I know how every bit of that code works, and I've reduced significantly the magic I have to deal with in order for my code to work (basically, just the database driver since I own everything else).
ORMs are a huge surface to secure, tune, and understand, and I think that it's unfair to tell someone their app sucks just because they refuse to use one.
But seriously. Pushing to master, AND updating production...where are your unit tests!?!?
Now that I'm getting deeper into the project, I need to add in features like PDF generation. No problem! I'll use microservices for that. Well, that means I need to build a Docker container, deploy it to AWS ECS, and configure AWS API Gateway to talk to it. None of these things are what I would consider "front-end" dev work.
Back-end is surely changing, and many of the needs can be shifted to the front-end/BaaS platforms, but there's still plenty of work left! I think we'll see the back-end role transition more toward building and maintaining these microservices and possibly the business logic portions of the front-end.
It's easy to do and you don't need to mess with anything more than two simple API.
- More functional principles and more resilient implementations.
- Complete decoupling from lower levels. Meaning that you don't care about servers anymore. Apps are Dockerized and resources are dedicated automatically.
- No more human designed APIs. Why write APIs if modern machine learning principles can replace the whole concept?
- More lambda / serverless concepts.
- APIs and integrations will be automated and auto discovered.
- More (micro)services, mixing of different languages and stacks.
- More streams, more events,...
If nothing else the increasing focus on data collection and processing will push the need for custom backend forward for many years. Any every application that needs to store or share data via an API, needs an backend that understand the data it receives so that it can validate, process and distribute that data correctly.
Backends (custom backends) aren't going away any time soon.
Your question reads as if you believe that "backends" equals desktop applications. Even in that case it's still hard for web application to compete with desktop applications. It's a matter of personal preference of cause, but something like Google Docs feels weirdly restricted, confined and cramped somehow, simply because it lives in the browser.
So much of backend work is redundant, login, user info management, allowing the creation of groups of users, user following, user data uploading or sharing. Not to mention all the DevOps, managing CloudFormation scripts, VPCs, proxies, load balancing, dns, ssl, cdn, database schema, the list goes on, and that's before there has been any original development at all.
With react native and electron (I haven't found the atom browser or vs code weirdly restricted) the web stack is moving into desktop territory and with serverless frameworks becoming more prevelent hopefully this will make writing original software easier with less redundant work.
As to your point about data processing needing a custom backend, I see the exact opposite. Generally all this data is just funneled into a generic hadoop cluster and can be manipulated any which way from there using spark or any number of data analysis tools.
How many times should this be repeated: web stack is easier to write only if you don't know shit of the native SDKs. Otherwise it does not even compare.
I can't comment on how they compare with native iOS apps.
Most people are already developing a webstack and if they have the ability to re-use code on their mobile/desktop apps then that is a win for everyone. If you are writing 100% mobile app then you probably aren't reaching for react-native.
It's a hell of a lot easier than native SDKs when you need to support multiple different client platforms. This is coming from someone who would rather not have to deal with GUIs at all.
True, but that's been the case for 10 or 15 years now. Most of the applications we develop are just CRUD applications... with a twist.
There seems to always be some detail that prevents you from having a generic backend. I mean look at something like SAP. It's an off the self application, it pretty much contains everything you'd imagine needing for running a large business. Yet every single installation is customized like you wouldn't believe.
Most of the stuff you mention isn't making the backend go away, it just makes it a little easier to implement and run. Just throwing EVERYTHING into a Hadoop cluster seems... careless. You'd still need to validate that whatever the client sent is actually any good. Otherwise you're potentially polluting your data. I don't believe in client side validation as something that provides any kind of usefulness, beyond giving helpful hints to the user.
The backend can't truly go away because it's always going to be managed by somebody, AWS, App Engine, Azure, etc. From the developers point of view though, running systems on these platforms could be made easier to implement and run to the point of being effortless.
I didn't mean to suggest tossing out server side validation. Though now that you mention it, if you wanted to, you could just toss every request into Hadoop, it's not polluting your data because the metadata associated with it indicates each is a request, and nothing more. When it gets to be too much, you could flush it, or do whatever you want with it.
On top of all that, the very concept of web apps (in my opinion) is fundamentally flawed in that they break the web (fundamentally flawed in a similar way that Flash as a interactive platform was flawed e.g user interaction along a timeline is an inherent conflict).
So personally I expect all the client-side web app stuff to collapse under its own weight at some point. As a example, I've been a web developer for 20 years, and for my own projects at home I don't even bother with web front ends; I use QML and QT and bang out a great UI in a fraction of the time it takes to produce a web UI. It's not worth the hassle anymore, but then I have the luxury of my own projects not having many (any?) users beside myself.
As for back-end, I've actually seen a shift more towards "systems" e.g a set of isolated sub-systems interacting via event busses or queues over HTTP/JSON either as micro-services or something close to that. Then a separate "web front-end" app pull data for output/display, almost as a client to that system. So a cleaner separation of The Real System from Web Application Back-end. And in this space, things seem to becoming much more standardised means of communication between parts, whilst at the same time there is a Cambrian explosion of new tools, languages etc which can be exploited. There is very much an idea that separate sub-systems can be written in languages which can best take advantage of for the problem space.
It's very interesting to see the differences in the way the front-end and back-end communities are approaching progress/innovation.
Anyway, thats my opinion/view of it all. Sorry for the wall of text
But perhaps less hackathon orientated?
I unfortunately don't have the specifics myself, but this would be the perfect point for someone else to chime in about DOM manipulation speed and the other various sources of browser overhead. I know they're nonzero myself.
WebAssembly won't turn the browser into a "perfect" runtime. Things will still be glitchy and slow and buggy and stuff, like they are today. Except now there will be multiple WA implementations, with low-level bugs 99% of webdevs won't be able to debug... :D
It may be helpful to see WA as a W3C-ratified JVM (Java VM) for your browser: like the JVM, WA is bytecode-based, and like the JVM, WA will be targetable from many languages.
You might even compare it to JVM + Swing - where WA is the JVM, and the DOM (and all other related bits) are like Swing. (Swing is notorious for being slow, although it's recently a lot faster.)
I think client-side will "break though" in the way you describe when there's something that bridges the (relative) ease-of-use of the DOM and the performance/accelerability of WebGL. I doubt that will happen soon though, considering the cementedness of HTML+CSS and the associated investment (eg full-stack CSS3 hardware accel).
-
I don't have a clear picture of the backend side of things at the moment, but I can offer these comments:
You've probably heard of WebTorrent. WebTorrent is not BitTorrent, because WebRTC P2P channels use SCTP on top of DTLS, which is effectively UDP on the wire. Besides being a headache for sysadmins who manage deep-packet-inspecting proxies, this essentially isolates the Web as its own "network" in practice - the only way you can talk to servers is via TCP+HTTP or UDP+DTLS+SCTP+<your protocol>. This makes things somewhat difficult. WebTorrent is actually a fully independent network that uses SCTP instead of TCP or UDP - the protocol is the same, it's just tacked on top of SCTP (& co). I expect (or at least hope!) that in a couple years next year most BitTorrent clients will have WebTorrent support.
There's also the fact that mobile networks are notorious at handling "unusual" data, because cellular data gets mangled by lots of pesky/fussy infrastructure as it bounces between your device and what might be considered the "traditional" Internet backbone. I've heard UDP is touch-and-go, for example, or even access to unusual ports. There's also the well-known fact that radio dropouts are still common and fundamentally hard to fix, even in 1st-world areas. This makes peer-to-peer networking incredibly hard.
A relay-as-a-service system to handle issues like these (SCTP->TCP gateway; connection persistence (very hard); etc) would make for an excellent DDoS generator, so at this point these types of things would need to be fixed at the application level, on a case-by-case basis. Unfortunately.
For another example consider Skype, which recently switched to an entirely client-server model, wherein the client talks solely to Microsoft servers; in the old days if the Skype client decided you had an awesome CPU/RAM and network and your NAT config was sane, it made you a supernode for clients who didn't have working NAT. Yup. Ref: "skype supernode", also http://www.zdnet.com/article/skype-ditched-peer-to-peer-supe...
There's also the fact that it's not (yet) possible to cleanly and reliably detect what kind of connection you're using on all OS platforms, so the old supernode system might decide your flaky home 3Mbps DSL connection isn't supernode material, but once you switched your laptop from Wi-Fi to your $5/MB 100M...
Thanks. Fixed!
What you can do with WebAssembly regarding front-end development is running your Qt/GTK+ type of programs on web.
But noone is interested in that. As a matter of fact, current trend seems to be ditching that sort of program on desktops and using HTML/JS/CSS stack for desktops as well (atom, slack, etc) because its much easier to create a top-notch UX using HTML stack.
HTML stack is very resilient. It evolves at a fast pace. Don't underestimate it.
WebAssembly will definitely have its own use case in the future (games, apps with specific performance requirements) but its not replacing frontend development.
Regarding backend, its not going away. Serverless doesn't mean backend-less. It means backend deployed in a very distributed fashion.
I personally think backends will be thinner and thinner as databases gain more responsibility an functionality.
No, because it's what the hordes of inexpensive web developers know how to use.
Ask most companies how much they are spending on their web-dev teams and they might disagree with you on that one
Software Engineer Salary Range: http://www.itcareerfinder.com/brain-food/it-salaries/compute...
I seem to be doing a lot more "mashup" work than ever before, joining up disparate back end systems in order to hide it all behind a simple modern front end. In the past if I was developing an order processing system I would work on the view/UI side of things as well, now I expose the functionality via an API and let the front end guys develop a nice interface for it all.
Also lets not forget you don't actually want any mission critical, confidential or security stuff happening on the front end no matter how WebAssembly gets.
There will always be a backend.
You'll write your code, complex or simple; you'll hand it off to some cloud system; you'll write a bit of configuration; you're done. Likely the configuration part will become less and less required.
You won't think about hardware, scaling, load balancing, etc, it will just happen for you. The data store being used will be abstracted away so that you don't know/care much about who provides it. Configuration problem.
Containers, AWS Lambda, these are just the first course of this meal.
Why? Because developers are expensive. Time spent doing anything other than the critical piece of writing business logic code is time wasted, and longer time to market.
That's how I see the future of the server side.
"Serverless" as in AWS Lambda has very specific use cases. The people who are serving websites on Lambda or something are not only doing it wrong, they are wasting money while locking themselves in to an architecture that doesn't fit their app.
I say this as a heavy Lambda user (https://hearthsim.info/blog/how-we-process-replays/). Lambda is super useful if you have to scale CPU-heavy tasks. I do think that model will play a big role in the future, but I definitely do not think it'll be the go-to model for backend. It's a complete paradigm change.
[1] https://news.ycombinator.com/item?id=11644042
S3 is hosting the site. S3 is awesome for static site hosting, in fact. I also use it.
If I bothered setting up my own blog, I certainly wouldn't spend a bunch of time setting up comments on Lambda just for my two corporate-firewalled readers when I can just use a hosted solution. That is, if I even cared about comments in the first place. There's just much more interesting things to work on, with and without Lambda.
Now take that idea to the absolute extreme. I write code, I push it into the cloudy-cloud machine, and it's serving clients immediately.
Can you clarify more on this.
When you scale, you need to make your infrastructure fit your app. Not the other way around.
Everyone leaves Heroku eventually. :D
Essentially, we can't keep teaching each and every developer database connections, SQL, et al. The cognitive load associated with these sort of programs actually slows down advancement, and the quicker you can setup cloud-powered services that get the job done, the better it will be. Although, I can't say what that will do the livelihood of developers. We'll have to wait and see.
Serverless has it's niche for some tasks such as responding to events, but to do any heavy lifting I'd take a handful of beefy VMs over unlimited lamda functions any day.
Everyone is ditching those beautiful and smart CS principles on FE by switching to hybrid apps, walking entire DOM on every update, etc.
There's no reason same wouldn't happen on backend. In fact it already has - interpreted languages, GC, micro services.
For better or worse progress in software always meant worse efficiency, because hardware was always forgiving that.
Considering that the end of Moore's Law is more or less here already, as pointed out in many HN articles, that's really no longer something we can take for granted.
1. Response time. A dedicated machine that stores state in-memory from request to request can respond orders of magnitude quicker than one that does not.
2. Resource Efficiency. 1-2 machines running optimized code will cost less than the 10-20 machines (or 100-200 machines) the cloud provider will have to allocate behind the scenes to run less efficient code.
Why is that a problem? You can design a system and implement it with AWS Lambda (or similar) and a few other things like their (or another) database, message queue, and data storage. Then, when you find that you've got processes that are bottlenecked by this, move them into dedicated servers or containers with faster routines. Just like we already do when pairing (possibly) slower but higher-level languages with C libraries.
As well, companies who have dedicated infrastructure teams are wasting money when we reach a world where that problem is purely automated. And that's where I think we're going.
If you think that infra teams is wasting money, then I'd say its the same case of 'serverless utopia' - when you're using 'purely automated serverless' platform you're paying way more money, because, hey, someone has to build that infrastructure and it isnt magic, its people, ops people. So what you're doing here is simply changing money buckets labels in your head and you have to now deal with a product that's tailored for everyone, instead of you specifically.
One team has 2 devs and an op guy
One team has 3 devs, one of which knows enough ops to ship code
One team has 1 dev, 1 MBA, and 1 QA
One team has 1 MBA, 1 QA, 1 Marketing guy
I pick the 3 dev team, and hope each of them can fill in some of the other roles on the side (ops, marketing, qa).
Not everyone would pick the dev team, but I would.
Ops is not hard. A good dev can do it. I am a developer, not an ops person. I have written 100s of ansible scripts. I have dealt with deployments of single servers up to 100 servers across all kinds of hosts.
Your argument is "OPS PERSON CAN CODE TOO"
My argument is "DEV PERSON CAN OPS TOO"
Really labels are stupid.
I guess following your analogy. If you only had a dentist and a salesperson available to do your heart surgery, who would you choose?
I'd go and look for actual surgeon. Or bite the bullet and try to live my life as it goes, without risking complete failure of both these guys messing with my heart. ;)
Hardware scaling and load balancing is infrastructure. Abstracting the data for the client, that's backend development, that's where your business logic lives.
I personally don't think any programming language fits the above description. And we have yet to invent it.
That compute will become like a common utility and will most certainly be used to power the APIs that consumer electronics call. The P2P adds unnecessary complexity when client/server, a simpler solution, works well to bootstrap an application from a trusted source without the need for homomorphic encryption or other exotic schemes.
Bandwidth, compute and storage will continue to increase faster in the center than our consumer electronics on the edge. The backbones, internet exhanges and datacenters are where density equals economy of scale.
And webframeworks will come and go without a revolution, just another leaf in the wind.
What is it that keeps backend developers in a job? We write code to give access to data in a way a consumer can use.
ie: we have input, we process the input and give the output (CS 101)
Every time we write something new one or more of those three elements has changed. Input, Processing or Output.
eg:
Input: The DB has a new table with data you need to expose on the website
Processing: The list of comments need to be ordered by upvotes rather than chronologically
Output: The existing JSON output also needs to be output as protobuf
Whenever any one of these elements is standardised it reduces our workload. Usually as a result of standardisation tools and frameworks are developed to take advantage of this.
If you want to know the future of any technology keep on top of their standards.
EDIT:
As well as keeping track of standards you should extrapolate what would happen if everybody adopted that standard. Who knows, maybe you could build a tool everyone uses and start a business around it.
oCaml/Reason can compile to JavaScript making it posible to write everything in one very powerfull language. In the near future we can probably also use jsx.
Seems like an improvement over using node and React.
FaaS will become big.
Back-end development with this will become so easy that a front-end dev can cobble together a pretty performant and secure back-end.
Only FaaS provider will hire back-end devs.
</joke>
I think since much is transitioning to realtime data (which FaaS can't do godo ATM), there is still a huge demand for back-end devs in the next years.