Ask HN: Why Isn't the Construction of CRUD Web UIs 90% Automated?
It seems silly to write the same types of code over and over to build CRUD web user interfaces. There have of course been hundreds of UI builders. Why aren't they used? Styling should just use plain CSS. Behavior should be mostly auto generated config from a standardized format such as json schema or a db schema. Behavior customizability should come from tweaking the auto generated json config, and adding some HTML and JavaScript to higher order components. The build targets should use plugins, starting with angular and react. Components should use a shared plugin infrastructure. Designers should be able to prototype in an invision like tool with real CSS and HTML that can be checked into source control. Why isn't this the way things are done?
187 comments
[ 3.0 ms ] story [ 156 ms ] threadMy guess is because web coders have very, very short memory and couldn't be bothered with learning whatever programmers were doing in '90s and '00, both on server side and on desktop side.
> There have of course been hundreds of UI builders. Why aren't they used?
There was this Borland Delphi thing, and these different GUI toolkits like Motif, Tk, Qt, GTK+, wxWidgets, or Java Swing, but then desktop software became mostly irrelevant, replaced with web. Somehow very few people cared to port good ideas from desktops.
> Behavior should be mostly auto generated config from a standardized format such as json schema or a db schema.
There you go with learning history. There were attempts on desktops at doing such things, but they died off. They weren't that good of idea.
> There you go with learning history. There were attempts on desktops at doing such things, but they died off. They weren't that good of idea.
What are some examples of tools that did this and what were the issues?
It looks like it's morphed into Apache Isis http://isis.apache.org/
It worked perfectly. Even my brother, who wasn't a programmer at all, was able to write his business software with it. It was reasonably fast. Downsides: theming wasn't possible so all applications looked the same, and Btrieve wasn't as reliable as it needed to be.
But otherwise it was great. I never understood why such a program doesn't exist these days. Tempted to write one myself....
Edit: they still exist: magicsoftware.com.
Your clients could be $10 Raspberry Pi Zeros.
I want to see more lean, tiny, fast applications. Low TOC, low investment because they aren't complex. Low end hardware that can be made reliable, small, and maintainable.
I think it's hideous that most business applications I see (CRUD interfaces) are 90+% text, and yet they are built on layer and layers of GUI and web interfaces.
Your Raspberry Pi example would be lean and easy to maintain. No virus scanners, no 20GB OS footprint to maintain. No worries about your user visiting Facebook all day instead of working with your business application.
I want applications that are so incredibly dull that they will be taken for granted as much as office telephone.
The difference is that if you're a small shop, the licensing costs will kill you (although Magic was a $2000 product, which is around $3,700 adjusted for inflation. R:Base and Magic cost around the same.
That isn't supposed matter because most real-world customers only have simple needs, so you want to bang out the apps quickly. But in practice I have never seen a use case that was really as simple as it was supposed to be -- they all end up in the "too hard for the framework" bucket.
Because the way we abstract our data in the database is often very different to how we think about the date in a specific context.
Show me a single component on any platform which can be customized via properties to "do everything".
There have been countless attempts (http://thedailywtf.com/articles/The-Mythical-Business-Layer , http://thedailywtf.com/articles/the-inner-json-effect , http://thedailywtf.com/articles/The_Enterprise_Rules_Engine to name but a few) at trying to reimplement a fully-fledged programming language in XML / XSLT, JSON or whatnot. Ultimately, all of these attempts seem to end up being more complex and unwieldy than the tools they were intended to replace.
The OP calls for a tool that allows you to automate most of the cumbersome work involved in building web UIs. Perhaps programming languages just are the closest abstraction that still allows for a sufficiently large degree of freedom when it comes to accommodating specific requirements.
After all modern programming languages and frameworks already automate most of that really arduous work. They allow us to define data-driven applications in concise terms that for the most part already resemble a specification written in natural language.
More accessible languages that more readily provide clear feedback (think of Bret Victor's work for example) could be a way forward.
Real-world programming is absolutely choc full of things that look like simple applications of a well known solutions, but which turn out to need lots of special-snowflake code to actually implement. Part of this is unforced errors, but usually I can't look at other people's work and say in detail how I would have done it substantially better.
So I think there is something fundamental going on. One guess is that although we can write tools that automate the perfectly common case, we are not good at writing things that smooth the inevitable app-specific corner case.
And we may be butting up against the fact that most objects in mathematical space do not have effecient ways of representing them. I'm thinking of kolmorogrov complexity and the pigeon hole principle. Most outputs have no short programs to produce them.
I've never actually managed it - the edge cases mean I end up chasing my tail around instead of "just coding the app".
Now I use small libraries for individual parts (like menus and forms) but even then 50% of the app becomes special edge case forms anyway. Accepting that the job is to just implement the spec and not invent a rube goldberg machine is an important lesson to absorb.
I wonder how often it's actually non-standard requirements and how often it's a mix of processes and poor communication/changing requirements.
I think in many projects, in hindsight it's perfectly obvious it is just a standard CRUD app - but it might need a few iterations to get the understanding of all stakeholders to that point...
Also, seems to me that such tools clash with the organisational process of how webapps are created today: It almost seems to be required that a web app has some kind of custom-written "business layer" in between, even if that business layer doesn't do anything than pass requests around.
But you are likely onto something that some things are just standard apps even if you can't see that at the time. And my mind might not be well tuned to seeing it even in hindsight.
Which makes me think that banging out simple apps fast is a very specific skill, because you need to see how to meet business needs within the slot provided to your by framework.
Apps created with ui builders (or even with just a constrained set of ui components like bootstrap) tend not to have good ux since every workflow has to be shoehorned into an existing abstraction. It's convenient for the programmer (and the budget), but not for the user.
Almost every CRUD app builder includes the ability to validate fields and properties, and this would appear to be sufficient for nearly all scenarios except it is not.
I can not name a single CRUD app that I have built that didn't require multi-field validation... i.e. start date prior to end date, time must be completed if date is filled in, if this part of the form is filled in then that part must be blank.
Without this capability to really express the application business logic, the technical simplification fails short.
Thinking about scenarios, example "Sign up" is not really just "Create a user row in db" but probably assign them a group also, and more.
Similarly, we rarely would use a "R" in CRUD (except administrators). Read is a basic operation, more frequently we need transformations/filters/aggregates on this Read, hence streams.
I have lately been trying to formulate the 90% of the 10% remaining. Checkout my other comment if interested.
What's really going on here is that you have a domain object with some nontrivial invariant. It's a mistake for the UI or anything that is aware of the UI library to try to model these domain invariants. Instead a call should go the model layer which then returns (or throws) an error providing enough info for the UI to gracefully help the user.
The UI framework can play an important role in this: but that role is to help programmers provide a UI for the validator that lives in a model layer. It is not to do the validation itself. (Although frameworks should provoide an arsenal of simple, single field validators, of the "this field has data-type phone-number" kind).
But for the current state of CRUD application frameworks that do not do this, it implies some enforcement at the data layer and probably some hint at this at the ORM level. This really does not exist in any sufficient way.
It reminds me of the idea that a graph/diagram generator that used some generic data source could be a common analytics tool, and that with a few well-defined defaults no-one would need to run lots of different analytics tools, or they could be generated automatically and would be good enough.
In reality they never are good enough. There is no one-size-fits-all analytics system because the KPIs differ from company to company, project to project, and these differences go all the way down.
Sometimes we see something being sold as a "this is all you need", i.e. a SAP solution or some generic tooling, even Stripe, etc. But the level of customisation typically required to make it sufficient for a mature organisation to use it is typically quite high.
A strategy to resolve this usually is: Change the org to fit the tech, rather than the tech to fit the org. But I've seen this fail consistently as typically the tech is too simplistic.
Then there's the "customise it to fit your needs", but going back to custom multi-field validation, or analytics... the number of permutations of logic is immense and the customisation tools themselves only allow simple expression of that customisation.
The end conclusion is that we need extreme levels of customisation, but when provided through tooling the effort to perform and manage the customisation (with auditing of who changed what, when, etc) is such that actually it is simpler and cheaper to write the customisation in code.
I think the reason is that building something with these builders may be easy at first, but when you want to do something a little different than foreseen, the cost goes up enormously.
It's this "cost gap" that developers are so afraid of, also because it is very unpredictable.
Since the views can be composed with a master view, the actual code generated is just a table element with a loop to add the rows/cells.
Although the view model is usually a plain C# class object, those objects can be generated from looking at the DB schema, meaning you only have a two-step process.
You could generalize it to make a table for any C# object using reflection, but that's kind of abusive IMO.
What I don't want to care about is storage implementation to implement those models in a given channel/client. If that starts leaking through the model abstraction layer into implementation, my business rules aren't testable, I can't change backends to scale without rewriting all my implementations, all of which are going to be bug-ridden and spaghettified.
The fact that we would even describe storage as a transformation (as if it was just a mapping with some consistency rules), should flag up how brittle such an approach is going to be. No wonder it is so painful for businesses to make any changes to their systems.
Now having said that there are many simple cases that do map well to a table. Those are already managed directly by businesses without the assistance of developers - in spreadsheets.
It's actually a pretty complex problem to solve securely, leading to scripts that try to support every configuration including off-the-shelf PHP hosting.
An ideal would be something like S3 where invariants like size, type, file format could be verified at time of upload and then exposed in on-demand manner to the rest of the application.
Also, does the generation output single page apps for popular frameworks such as react and angular?
Web2py was created because the author found Django too difficult to teach to his students at DePaul University. He rebuilt Django while fixing many issues in how it was designed, and simplified the API significantly in the process.
Instead of Django's admin interface, web2py gives you a Grid component which allows you to create an Admin interface for any model/relationships that you like within any page you want. So your Admin pages are just another route/controller which use the grids as necessary.
Web2py's ORM is super nice to use. It's very similar to using plain SQL, and incredibly powerful. It holds your hands a bit less than an ActiveRecord style ORM but I find it also traps you in fewer corners. Especially when it comes to reports/aggregating data.
Likewise, it's a pain to set up email addresses for the user's username in Django, or to add custom fields to the User model. In web2py, it's dead simple.
Web2py has an easier to use authorization layer than Django's, and it handles migrations more smoothly.
Web2py's HTML templating language is much nicer than Django's. It's just python, nothing extra. Super easy to pick up.
Web2py forms support file upload fields, and image upload fields out of the box. It also has a module for automatically resizing image uploads as necessary. It includes a datepicker for date fields, and it also has the "LOAD" function which makes is super easy to Ajax up your site.
The other big thing that I love about web2py is that all the form/grid helpers output Bootstrap compatible HTML out of the box, so you can throw on a theme from Wrapbootstrap.com and just like that you have a beautiful site.
It's super cool :)
https://reactstudio.com
It's maybe 80-90% of the way there, depending on what you need. Closing the gap feels demoralizingly hard with no resources (I'm spending time on it unpaid).
The design environment is quite complete, although it doesn't even try to expose all of CSS -- instead it's more like a "walled garden" of the parts that make sense for mobile-first designs.
I'm most proud of the exported code's quality (no extra junk, no giant library hanging around) and also the component and plugin architecture. You can either design React components visually or import them from code (e.g. an npm package), and both will work the same within a design.
The fundamental problem is that the technological foundation for web apps is so unsuitable, like building a house on quicksand using decayed planks. CSS is a cruel joke with its thousands of properties, their decades of cruft and their unpredictable interactions. HTML was designed for documents, then was semi-accidentally turned into a view layer, but it never adopted any of the learnings from the successful '90s GUI desktop frameworks. Some of that missing structure has been added back by efforts like Angular and React (and the latter is fine to my taste), but it's just really frustrating that web front-end development is still stuck endlessly patching over a rat's nest of <div>s into something that resembles actual UIs. But what can you do? It's not like browsers are going to add a real app framework. So here we are, still trying to decide whether buttons should be <a> or maybe <input> elements. Sigh.
1) For the end user, it's too hard to download and run an app. 2) For the developer, it's hard to create an acceptable multiplatform app.
The platforms can also change and break applications in unpredictable ways. For example, iOS 11 won't run 32-bit applications.
<button>, obviously =)
https://github.com/hofstadter-io/geb
I've actually written something (not open source) that largely solves this problem for myself and my company, but, its a very hard problem to solve in general. The largest benefit that I find from it is that it allows you to make some things that would done directly (EG: UPDATE statements) be by a web UI. Figure if you had a particular table that only got updated once or twice a year, is it worth it to write a bunch of CRUD code for that? No. Is it worth it to write 20-ish lines of JSON to describe the table and avoid a bunch of problems that direct DB updates cause (accidentally hit enter before I typed where on that update, no tracking on who makes what change when)? Yes.
Has a lot of benefits. I understand its a hard problem, I don't understand why no one (aside from perhaps django-admin which I find too obtuse) has tackled.
I may rewrite existing code to golang (PHP now) and open source it at some point in the future.
Often the way the data is represented in the DB is very different from how a user thinks about the data in a given context. And another user, in a different context, might again think very differently about it.
So you need a lot more configuration for the input data than you might think at a first glance.
Lastly, there are good frameworks already that do a big part of the work.
Django for Python is one that I'm very familiar with. If you combine it with vue.js and bootstrap on the front end, it's very quick to create a working MVP.
It is, several times over.
Using SQL or some other domain-specific query language against structured records? That's the first 90%. HTTP and hypertext? That's the next 90%. CSS and modern HTML authoring? Another 90%. Web app framework with an ORM and MVC / data binding / reactive / flavor of the month? Another 90%.
Your mistake is assuming that automating 90% of it reduces the work by 90%, rather than letting you spend 10x as much time on making the bespoke parts fancier.
For example, instead of a one-step translation of "invision mockup" to "final site" it would make sense to translate the mockup into an automation conveying that specific design language.
HTML and CSS deceive us by apparently being those generic tools to solve everything. But because they're generic and composable in a high level sense they actually get in the way of the intention - layout, content, and interactions are all shoved into non-overlapping functionality boxes. And then we add JS on top of that, making it possible to break out of the model badly.
The web is not unified like that, the foundation is way too shaky. Visual UI builders have failed idea because they cannot encode complex business requirements.
But those are almost never mentioned in hacker news, which may say something about our community more than it says something about the possibility of building such tools.
The main downside was the programming language - for an if/else statement, condition and both blocks all had to be on a single line. Variables and functions could be referenced in double-quoted strings meaning the syntax highlighting was always wrong. And there was no chance of loops or user defined functions. This meant code editing was done with repetitive clicking in the IDE.
The other downside was inability to diff or branch. It kept a history of deployed versions and that was it.
If you squint a little, you could also regard the Wordpress ecosystem as getting close - I've seen things done with custom fields that are remarkable.
http://rubyonrails.org
I think if you push your question to the extreme, we can ask why Literate Programming never took hold. Wouldn't it be vastly more cost effective for businesses to completely commoditize the entire IT department with just a specific kind of technical writer who could pass along these specs to a service that would compile the application as written? Competitive technology simply isn't that predictable and circumscribed in capability.
The issue is you think you can drag and drop data fields and be all set. But to deliver a halfway decent UI, you need custom bits and pieces here and there. From layout, to extra JS, to new "picker" types for some fields, and on and on.
When I've looked at it, the issue is either the forms toolkit gets stupidly complex or doesn't have a good customizability story, and in frustration you throw away the toolkit and just do it by hand.
There's room to do something better, so long the customization is really easy and trivial to add one-offs.
Exactly!