Ask HN: Why web frameworks are becoming so complex?
Since I started working as developer I can see that every year majority of leading frameworks (at least in PHP, Javascript, Ruby...) for web applications are becoming more and more complex. Instead of writing code in programing language we move into xml/yaml direction and hundreds of weird config files. Each frameworks use different syntax for config, often different templating languages, few layers of abstraction with a lot of magic behind the scene, Dependency Injections, ORM'S, CRUD generators with supposed to make our life easier actually are painful to use...
Even small website often requires hundreds of megabytes of dependencies. Sometimes one application can have 2 different build systems e.g. (ant, grunt) and few dependency managers...
Micro/nano services trend looks optimistic, but I am not sure if it's not double-edged sword, and instead of using complex frameworks developer will be forced to use complex communication system to orchestrate communication between them.
In few years instead of writing code are we going to write configs files?
43 comments
[ 3.4 ms ] story [ 85.4 ms ] threadYou don't need a license to release a framework, and a framework doesn't have to be sensical to be adopted.
If you learn the fundamentals of the things you're working with, particularly HTTP, web browsers, the web in general, and your chosen programming languages, things will become much more clear.
This might be because I'm not building anything that's going to blow the doors off of the internet, but I've found having at least a casual familiarity with a few frameworks (Django is the first that pops into my head, though you may be talking about .JS stuff) has made prototyping, at least, far simpler. Building a basic Python CRUD app from scratch, depending on what you're trying to do, could take, I'd guess, two weeks to get something* up and running. With Django, you could absolutely have your basic front-end and admin interface done in a day.
Once you get into the tricky bits (or just bumping against the edges of the Django database abstraction layer), yeah, I can see how it's annoying -- is that more what you're talking about with the "we can not just set breakpoints in yml file" bit? That setting up models by hand is more predictable than letting the framework do it?
I think complex web frameworks are developed by expert coders to make their lives easier, supposedly. With little concern for other less-expert developers. But it just turns out to be so complex because they are trying to DRY the shit out of everything.
In my experience, the widely-used ones tend to be fairly sane for the most part, and can often be used without having to delve too much into the complexity unless you need to - which I think is the important bit
But that said, the Webpack + React + Flux model is the simplest kind of programming I've done in a very long while.
They have done a phenomenal job of abstracting away complexity behind stable apis.
In practice, a lot of frameworks are made by people (who are just people, and don't 100% know what they're doing at the outset). I'd say that if you're spending too much time in configuration, you're not using the right frameworks.
No, in a few years those frameworks will be forgotten. From the terms that you've used, it sounds as though you're coming from the Java world, which has an inexplicable love of config files - if that's the case, it might be a bit longer than a few years, but they'll eventually figure out that config files aren't a good representation for options.
Though the layers of magic and abstraction will continue to increase and grow, at least until simple LOB usecases can be achieved in a couple of lines at most.
In fact, I think we're starting to see a trend in the other direction. Angular 1 was a massive mess of complexity, but Angular 2, React-based frameworks, and various other frameworks coming out these days tend to be much simpler.
Believe it or not, the web has been the platform where most of the innovation in UI development paradigms has been taking place in the past few years. And I don't just mean in terms of the sheer number of new things coming out.
The web community pioneered UI development paradigms involving functional programming techniques and immutable data, and have brought its benefits to other platforms (through React Native). I personally would never want to go back to the old way of building UIs. And I suspect I'll feel the same way about client-centric data fetching when I get to try out Relay/GraphQL, Falcor, Om Next, etc in production.
If constant change is the price we have to pay for innovation at such a relentless pace, I personally welcome it with open arms.
RE: the video. I'm almost certain you know this, but Elm is a part of the web community.
- https://www.youtube.com/watch?v=1XNATGjqM6U and https://www.youtube.com/watch?v=HPyKHxy7X0w and http://web.archive.org/web/20120714082000/http://www.reactiv...
(True Elm is part of the web-community, just saying React was not the first mover but I agree it will certainly be most likely the one that makes FRP cross the chasm)
However, I agree with above poster. I remember developing my first windows app by doing a main event loop. It's been that way for a long time. Only recently are we doing more asychronous callback / bound type mechanisms of reaction. And still not as strong as the FLP stuff mentioned in the video. Not as much functional stuff, just functional paradigms.
I personally really like the perfection of static input/output transformation graphs. But yeah..it's not been that way for a while and I agree with OP, the web being so event/input/signal based, has spurred the movement for better ways of reactive programming.
Blame this partially on the weird phenomenon of programmers refusing to touch languages they consider aesthetically ugly. Javascript? CSS? Ick. Much better to pile on a build system to compile those languages from another language that looks kind of Ruby. HTML? Ew - no significant whitespace. Let's generate HTML from something that looks and acts like Python.
But, snark aside, I think it's a good thing if a framework can be agnostic about the kind of configuration syntax it uses, although that in and of itself implies some minimal complexity. Whatever choices the developers make about this, of course, amount to personal preference. Heck, I prefer INI files for configuration.
>Even small website often requires hundreds of megabytes of dependencies
That depends on the language and framework you're using. And how small you consider a "small" website.
Unfortunately (in my opinion, with which I know others will disagree) frameworks tend to want to do too much, and as such tend to bloat into vast, overly complex attempts at reinventing the wheel, the chassis, the transmission, the bumpers and deciding what kind of air freshener you should want hanging from the mirror. I would prefer a framework not have a templating library or ORM or code generators baked in at all. Those should be implementation specific decisions. Rather, a framework should manage dependencies, routing, global config, responses, have some hooks to allow extensibility - the basics. If I want more, I can pick libraries for that.
The JS community is just now being introduced to compilers, and there is significant push back and lots of people crying for "simpler age". I think it will be a while (if ever) until languages like closurescript, scalajs, & typescript start seeing widespread adoption outside of their fiefdoms (google, fb, microsoft).
If you are interested in a typesafe universal js ( or isomorphic js, whatever we are calling it now ) proof of concept you can check out this project using scalajs and scala-js react.
http://ochrons.github.io/scalajs-spa-tutorial/index.html
But there is a learning curve and unlike Haskell where you just know have to know the language, you have to know both the JS quirks and the compiler's quirks too. It is a leaky abstraction. This is the problem I have with a lot of the compile-to-android frameworks vs. just putting with with Java.
Your gripe isn't about the tooling, but developers ability to learn.
You might want to give Ruby Sinatra a whirl. I just helped a friend build a starter site in it. The Hello World for it is 1 file with 4 lines of Ruby, no config files at all. Save that 1 file, run it, and you have a live website.
What I think really makes it cool is that there is a decent amount of sophistication to it, but you only see it when you need it. It doesn't force any particular way of doing things onto you. It's also kind of a cousin of Rails, so if your needs get to that level, you should be able to switch over pretty easily and make use of the enormous ecosystem of gems.
I think express.js on Node has kind of the same idea, might be worth a look also.
Might as well be
"Your app is spaghetti 1 year after you leave the company."
Sticking with Ruby for examples, if you want to put up a quickie site with 4 routes and no DB, then it seems kinda silly to pull in Rails and deal with all of the complexity and expectations. Throw up a Sinatra site and be done with it.
On the other hand, if you're getting more demand for it and now you want more logic and analytics and a DB after all and some user accounts etc, then you should know when it's time to move over to Rails.
If it is a small team of close knit developers who communicate well, and expect to work together for a long time. Roll your own setup, loads easier and much more flexible.
If you are a corporation with reasonable dev turnover and good management, you might be able to pull off your own solution too.
If you are using contractors, have high dev turnover, a distributed team, and especially all of the above. Use a framework, any framework, otherwise you will wind up with a disaster within 6 months.
There was a really great talk a couple years ago about Twitter's defunct FlightJS framework. The framework was meh, but something in the presentation stuck with me, "If the shit is in a box, no one can smell it.".
I took that to mean that if your application is structured and organized ( like what a framework imposes ), suboptimal parts of the application will have a smaller impact on the app as a whole, and refactoring those parts is often a matter of just "throwing out the box".
On the front-end React has done a fantastic job of providing lots of boxes for contractors to shit in.
Page loads are very large on some sites now, where they don't have to be. Though, advertising can be blamed for a lot of that. There's just a lot of extra cruft out there now, without much added to the user experience, in fact, it often detracts from it.
What happened to using only what is necessary? Why throw so much unused stuff behind a site, and especially, why load anything you're not using? Why complicate things?
Maybe I'm nostalgic for static pages or something, but I miss the days where things were simple in terms of page loads and efficient design, and I think a technologically-advanced but focused (avoiding excessiveness) approach is best going forward.
This philosophical approach is one reason I like the Hacker News site itself. It accomplishes its task well, and is almost solely text and minimal css. It's running on some stuff built underneath (as dynamic pages require), but it seems pretty minimal in general. It needs nothing extra.
I'm sure people said the same thing about the gutenburg press vs locking a bunch of monks in a room for 20 years.
That being said, if you don't understand the tech, or the tooling that implements it, don't use it. Keep doing things the hard way, I really don't care.
In a few years we will be writing config files for machine learning algorithms to stitch together a ui based on requirements doc, or an api specification. You can already see this within AWS API gateway and lambda functions, and a lot of the R&D being done around procedurally generated ui's.
The developers that fail to adapt will be left building sites using legacy tooling for shit clients who don't know better. Or move into management where they can impose their archaic views on their subordinates who do know better.
Either way, no one is forcing you to write code you don't understand, that is 100% your fault.
I look forward to the day that I can automate away the need for sub par developers polluting codebases with their versions of "simplicity". I am currently re-writing a 100k line js app for a fortune 500 company, the initial contractors were trying to keep things "simple" by using 10 year old tech. They all got fired 2 days after my proof of concept demo.
Your example, though it's buried under a bunch of stuff calling people dumbasses, makes an interesting point: You're right, there isn't anything simple about 100,000 lines of code in ANY language, even if it's 100,000 necessary lines. What were they trying to do that was so easy to replace? What kind of JS was knocking around in 2005 -- is it all display stuff and wonky jQuery?
Here are some ( sanitized ) examples from my current project that exemplify the danger of allowing this line of thinking among your contractors:
1. Completely unmanaged client dependencies on a per page basis ( negligible code, infinite heartache )
2. Template files that do not reuse code due to laziness or differences in markup style ( 20k+ extra lines of code )
3. CSS files crafted on a per page basis, again little code reuse due to developer style and competence ( 20k+ extra lines of code, huge download size, difficult cacheing requirements )
4. Confusing and undocumented deployment procedures ( I had to completely rebuild the infrastructure from the ground up )
5. Each and every page had a its own js file, with its own global exports, dependencies, and code style. Most were over 1000 lines of untested jQuery Stackoverflow answers. 65 unique pages, sharing almost no code.
6 My favorite, a grunt task that searched the code for 'some_var + other_var' and replaced it with a string literal "https://my-dynamically-generated-cdn-fingerprint.cdn.com/res...
7. Superflous build tasks and bugs due to inconsistent project structure. Some people like /dist others /builds etc.
The list goes on, I'm finding more stuff every day.
This is a well funded project, 1 year into development.
Just implementing react, gulp, and sass allowed me to remove 60k lines of code over 2 months. The app still works in a request / response method, haven't gotten to that yet. The backend is still a tangle of coffeescript.
This is the risk you run by not imposing structure on your developers via solid frameworks and tools. If you can't pick the best tool for the job, hire someone who can. Otherwise you'll end up hiring him anyways to fix your broken 20 million dollar "unicorn".
This raises an interest point that I think I was missing before -- "imposing structure on your developers via solid frameworks and tools." The idea that even if you don't need the functionality of a particular framework, a framework's prescriptive way of doing things might reduce enough of the variance in styles to make a big project more coherent.
Having a framework in place minimizes the impact of those "bad programmers" to just a single ui component, or api route, or database migration.
Not having a framework in place means that that intern or contractor from india can now introduce bugs that impact the entire system and are nearly impossible to track down.
I feel like it shouldn't be such a hard sell to get software engineers to admit that structure and consistency are good things.
When I said simple, maybe I meant "efficient" instead. I just see a lot of misuse of frameworks where all of the code is thrown in and then barely customized to make it work, and all the extra unused cruft is left in. Basically pages are loading a bunch of stuff that is never actually used...
So I think maybe we were speaking past each other a bit, because I generally agree with you.
The initial post at first seemed a bit hostile, but some of that I think we're inferring- because we're reading our own tone into the text. Also, as you said, it's something you constantly deal with, so I'm sure it's a big point of frustration for you.
When frameworks work to allow you to clean up a codebase or prevent it from becoming the mess you describe, then they definitely make sense. When they cause bloat for no good reason- that's when I have a problem with them.
I guess it all comes down to developer use/misuse of them, not the frameworks themselves, in most cases.
"I don't want to have to learn new stuff."
GTFO of this industry then.
Backend wise I've taken a liking to Express because it hides nothing. I prefer it over Rails or ASP.NET MVC, where there is so much magic. I would switch if I could but we have too much legacy code.
The way I see it, it's becoming complex the more you learn. For instance, the first time one learns a programming language, one learns the language and steers clear of frameworks. The deeper you go, you learn tricks and stuff "which this library does" or "this framework does". Eventually you get exposed to the stuff that does more stuff in less work, away from the programming and more on configuring.
> Even small website often requires hundreds of megabytes of dependencies.
You wouldn't want to write those from scratch.
> Sometimes one application can have 2 different build systems
One tends to use the build system in the language nearest to the platform. I still have yet to see a person write Grunt on a Java project or Maven on a JS project.
> Micro/nano services trend looks optimistic, but I am not sure if it's not double-edged sword
It is double edged. More control but you need to set configurations, set conventions. Do it wrong, and it's easily spaghetti.
> In few years instead of writing code are we going to write configs files?
For all that configuration, your are configuring something. That something is written by someone. So while you're configuring here, someone is programming somewhere.