Ask HN: Do you use automated tools to create APIs, or do you code them manually?
Do you use some sort of framework/tool for creating the APIs needed for your product/service/application/ecc, for example (https://loopback.io/), or do you code by hand?
72 comments
[ 4.1 ms ] story [ 171 ms ] threadI think early stage and MVP projects are almost always written by hand.
We've had to drop-in-replace, or add a validation or access layer service for something, and using protobuf has made this super easy. Anything interacting with that service is none the wiser.
I've been playing the FoundationDB Record Layer for a personal project of mine, and with this setup I can generate not only the API implementation, but also the models used by the persistence layer:
Protobuf (Messages) -> gRPC -> Scala/Monix -> Protobuf (Models) -> FoundationDB
Sounds really cool! Is this something that comes out of the box or generated by your own plugins?
Basically you create your tables and run PostGrest. Bam! You have an http interface / api for your database. We would then create light wrappers around those that took on specific responsibilities - security, audit etc. The wrapped apis is what we exposed publicly.
This may not sound all that helpful but it made the bit we implemented unbelievably tiny. As a plus, we found that a Java application that exposes an endpoint and calls an endpoint is fast to start / stop because it doesn't mess around with DB connection pools.
[1] http://postgrest.org/en/v5.2/
1. the interfaces must remain stable to the outside world that relies on them
2. They select what underlying resources and functionality is accessible by outside users, and what is hidden. A lot of your internal implementation is either a mess, “temporary”, insecure or intentionally internal.
3. They control access to the internal application through authentication, authorization, security, and translating data in both directions.
4. When the internal representation changes, they map the new implementation to the old interface to ensure the system remains reliable to API consumers.
5. They offer migration paths when change is necessary
That being said...
Auto API generators are really useful for internal systems where you control the underlying system, the API, and all systems relying on the API.
They are also useful to build an initial API that you plan to fork.
I like to design systems in that order :
1. my data model ( database schema ), because it gives you good questions to ask regarding the business side of your problem and let you go very deep just by asking « for each a, how many bs can we get »
2. my external api. Because it requires you to « dumb down » your problem, and see which part of your model you want to expose and how.
3. I actually start coding my business process, and then bind them to the model on one end and api on the other
If i were to use code generation tool, it would need to generate both the db and the api stubs, together with the correct information exposure. I’m not aware of any tool that would let you do that.
For OpenAPI, Connexion by Zalando is one of the best implementations I have used. You just need to write the logic and provide the API spec.
This helps elevate changes to the API itself; you can easily write automated systems which detect changes to the specific SDL files. Or, the way companies like Namely [1] do it, keep those SDLs inside a separate repo, then publish the adapter libraries on private npm/etc to be consumed by your implementation.
[1] https://medium.com/namely-labs/how-we-build-grpc-services-at...
NFS is based on this, as with other services. Conceptually it’s exactly the same, with some underlying differences.
TLS certs are encoded in ASN.1 DER, for instance, and LDAP messages are encoded in ASN.1 BER.
Thanks for the Namely case study as well. It was timely reading. :)
For professional stuff... it really depends. I like GRPC but codegen needs team buy-in... It can quickly make a fast development loop hurt if done poorly. Doubly so if IDEs are involved for some users and the IDE is constantly updating it's caches of types and interfaces. I've just seen it turn into a hot frustrating mess very quickly.
I might think differently if confronted with a huge API surface area to build off the bat, but I haven't run into that yet.
I typically have a repository generated by Spring Data, a small service layer with business logic on top of those and then an MVC controller that only talks to the service layer, never the repositories.
Each controller also has its own DTO class(es) for request bodies and responses and a small converter between DTO and entity. Kotlin extension methods make it easy to add the toDto() method onto the entity so a typical controller will fetch the entity from the service and return entity.toDto().
Kotlin, Spring Boot and Spring Data are amazingly well suited for this.
Also, you could use projections in place of DTO’s.
You don't really need DTOs because you can use projections and set a default projection to be used when that entity type is returned in a collection. Any entity fields that should never be exposed can be annotated with @JsonIgnore. And then if you need endpoints that aren't CRUD, you can build those the usual way.
If I was to start a new API today I'd use Hasura. It automatically creates a GraphQL schema/API from a Postgres database. It's an amazing tool.
https://hasura.io/
Looks really interesting to easily layer a graphQL API on top of a Rails app with a few serverless functions...
I'll expand a bit on my previous comment.
So the idea is that Hasura is a stateless layer on top of Postgres that generates all the necessary GraphQL schema/queries/mutations/real time subscriptions for doing CRUD based on the Postgres schema. If you change the tables (either via the Hasura admin or some migration system) it all adapts automatically as you'd expect. It can use a remote Postgres DB, you don't need to run the API and DB in the same machine.
Performance is fantastic. Hasura is very efficient in terms of speed and memory consumption. Even with a free Heroku dyno you should get thousands of reqs/s.
On top of direct data from tables you can also read Postgres views. Essentially you can read a custom SQL query from GraphQL.
Hasura can also integrate external GraphQL schemas via a mechanism it calls "stitching". The idea is that you can point remote GraphQL schemas to Hasura (on top of the current one from Postgres) and it will serve as a gateway of sorts between all your GraphQL clients and servers.
Hasura does not include authentication, but it's very easy to integrate with your current system or with services like Auth0 via JWT.
Hasura also includes a powerful fine grained role-based authorization system.
Whenever anything happens you can configure Hasura to call a URL (webhook) to do something. Maybe a REST endpoint or a cloud function. This is usually the way to integrate server side logic.
The only problem we've found is integrating Hasura with our current authorization system. Our users have multiple roles and we have no way of deciding which is the current role. Hasura requires a single role to be passed to its authorization system on the request headers. This is something that is being worked on AFAIK.
Their youtube channel has lots of little videos showcasing all the functionality.
https://www.youtube.com/channel/UCZo1ciR8pZvdD3Wxp9aSNhQ/vid...
We reduced memory usage by 80% over Node. We never had a performance bottleneck with Node either but it feels nice to be running on the smallest Heroku dyno and knowing you won't need much more for at least a couple of years.
As for the developer experience we vastly prefer Go over JavaScript. It's more tedious at times but there is no more ambiguity. We love that we barely need any dependencies. Moving from JS to Go was extremely easy as all devs in our team are polyglots and Go is pretty simple. I don't know how easy it would be for a JS only dev, but I imagine it wouldn't be too hard.
When using NPM/Node/JavaScript it seems there are always hidden dangers, probably more in the front end than when doing backend Node. With Go there are no surprises, everything feels solid and predictable.
After about 2 years with Go we are still happy with the decision.
Having used loopback before, it's a quick way to get an api up and running, I personally struggle with injecting logic into endpoints/writing custom endpoints.
If the code's "all there", I know where to look. If I have to intercept hooks it adds an extra layer when searching.
Summary, loopback has been great for creating APIs where all I care about is crud, but for larger projects I stick with snippets/generators so I can extend easier later.
It doesn't make sense to send SOAP messages to browsers but I cringe every time I find myself with a vaguely documented Rest API when integrating systems.
I think the key is to pick a documentation tool that the team will actually use.
Often I have a simple problem where I can write a simple clean API quickly by hand. Generation is a negative, generated APIs tend to be complex and hard for the user to read.
Sometimes my requirements need something that a tool does better. For example protobuf gives me an efficient over the wire API that can be used in multiple languages: I'll let protobuf generate those APIs as I can't do better by hand (though we can debate which tool is better for ages).
Sometimes I have a complex situation where I'll write my own generator. For example I once made a unit system generator for C++: it was able to multiply light-years by seconds and convert to miles/fortnight - no way would a handwritten API support all the code needed for that but with generation it was automatic (why you would want to do the above is an exercise for the reader). The API was easier to understand than boost's unit system (APIs are about compromises so I won't claim mine is better)
What's important is that you have rigorous testing around your API.
APIs are essentially external contracts people build against. You don't want to break this contract.
make sure it
- never changes unless you know about it
- updates the documentation whenever it changes
That being said, I just finished a school project where we (our class) were divided into small teams and we had to implement small RESTful web apps. My team chose to kick it off by grabbing two people from the front- and backend team and writing an API specification by hand. It was a breeze and we were done in a few hours. After that front- and backend (almost) never had to interact with each other again until the end of the project where we had to stick the two things together.
This probably isn't applicable to real-world cases where the requirements are ever-changing and everyone's a full-stack dev (or you don't have a team at all) but I found this sort of separation quite useful for this project. (It kept team sizes managable, different kinds of devs were in seperate teams, we didn't have to wrestle with any tooling that would halt the whole project.)
I see no problem with generating client/server boilerplate from spec though (like Swagger does, I think).
1) Client code in various langs. 2) Server code in golang of python or nodejs. 3) Swagger. 4) Rest interface if you want to. 5) Gorm definition of you use golang gorm.
(i) TCP/IP
(ii) HTTP
(iii) ASN.1
(iv) SQL
(v) The key-value session state store I wrote for my Web site (cheap, simple, quick, dirty version of Redis).
Etc.
Now, how can the design and programming of such APIs be "automated"????
anyone still using CORBA? or implementing new projects with CORBA?