For sure, there are a few tools out there like Requestly to change API behaviour, but it's a frustrating experience. In terms of the direction, planning to keep this simple so I've no plans for additional features.
Nice this made me go back and check up on the Gorilla LLM project [1] to see whats they are doing with API and if they have applied their fine tuning to any of the newer foundation models but looks like things have slowed down since they launched (?) or maybe development is happening elsewhere on some invisible discord channel but I hope the intersection of API calling and LLM as a logic processing function keep getting focus it's an important direction for interop across the web.
afaik, the langchain solution loads entire openAPI spec which consumes a lot of token and won't work for many large API. For efficient token usage, api2ai divides the task into two steps: api planning and params parsing. First step takes a summarization of all the endpoints. Once the endpoint is known, we parse params using the schema of the selected endpoint.
What would be particularly useful is if it saved token values and then (through search) joined them on the response of the auth call to get the initial token.
That way you could easily determine what auth call was needed to get you a token to use the endpoint.
Sometimes I click on a path parameter and it doesn't "create" it, even though there are several other examples in the list. Not sure if it's a bug, or something I'm doing wrong.
Overall, this is an absolutely wonderful tool and I've wanted something like this for a long time. Incredibly useful, thank you!!
Do you mean the devtool protocol[1]? I didn't follow the space so have no knowledge on it. On the other hand there seem to be a polyfilled API on chrome.devtools.network.Request which OP's extension uses extensively https://github.com/DefinitelyTyped/DefinitelyTyped/blob/mast...
As a follow up, the algorithm that powers this makes use of the chrome.devtools.network API. Specifically it passes the Request object that is in the HAR 1.2 archive format.
So if you can pass the equivalent of that in Firefox/other browsers to the insert method and switch things up a bit, it should be relatively straightforward. I will think about pulling out the core logic into its own lib.
It could do as it works with the HAR 1.2 format. There is another library that can do this. It isn't suitable for the FE as it uses QuickType & pulls in a ton of dependencies, but it is far more configurable.
The documentation states 'automatically populate based on JSON requests that fire as you browse the web' so does this mean that gRPC protobuf are not captured?
I saw your sibling comment about "keeping it simple," however that is a bit counter to "generates OpenAPI specifications" since those for sure are not limited to just application/json request/response bodies
The latter is likely problematic, but the former is in wide use still, including, strangely enough, the AWS API, although some of their newer services do have an application/json protocol
I know that's a lot of words, but the tl;dr would be that if you want your extension to be application/json only, then changing the description to say "OpenAPI specifications for application/json handshakes" would help the consumer be on the same page with your goals
The description doesn't explain exactly what this extension does.
I assume it monitors all XHR requests as you browse a website, and if the request/response matches [some criteria (e.g. is JSON?)] it will assume it's an API request and log it?
Is that correct?
If so, it will only work on websites where the frontend is implemented like a PWA, with lots of AJAX calls to fetch data, etc. For sites whose pages are all generated server-side, the extension won't generate any API schema, right?
Edit: Also how does it differentiate "API requests" with regular AJAX content fetching? If a website fetches some arbitrary content via an AJAX request (e.g. some lazy-loaded HTML), that's not an API request. That's just part of a website's layout.
How would it, there isn't any API in the first place with classic websites. Your could maybe consider the urlencoded post requests an API, but then the reply is another html website so how do you formally specify the reply format? "The queried data is somewhere right below the third <h3> except when there's a new message for the logged in user, then it's the fourth one"
Yeah but my question remains: by what criteria is a request classed as an "API request"? Websites make tons of XHR requests and not all of them are API requests.
I want to know what this extension does that's different than me looking at the browser's Dev Tools > Network tab.
Worse, for something like SvelteKit load functions, this will think there's a "real API" where what's actually there is an internal detail and will change often.
this is very cool!
I just tried using it, unfortunately, my NextJS app dir project makes most requests from the server side, so it was only capturing "posts" made from the client. Is there a way to run it from the server?
I'll second/third the feature request for auto-including auth headers/calls (as many of the sites I'm trying to understand/use APIs from use persistent keys, and scraping these separately is just unnecessary extra time).
On that same note, I'd greatly appreciate keeping the initial request as a "sample request" within the spec.
I'd also greatly appreciate an option to attempt to automatically scrape for required fields (e.g. try removing each query variable one at a time, look for errors, document them).
The problem with this type of tools is that they only produce specs based on infos they can get.
The spec produced will be incomplete (missing paths, methods, response variants, statuses). For that you should use a framework like Fastify, NestJS, tsoa, FastAPI, which have built-in OpenAPI support.
Can be very valuable for reverse-engineering though :)
> Akita makes monitoring and observing system behavior accessible for every developer. Quickly discover all your endpoints, see which are slowest, and learn which have errors
Translation: Install a Docker extension that intercepts and inspects your network requests to infer the shape of your API.
I feel like when you're targeting developers, you should quickly explain what it is you actually do.
Companies in general should do this, not just ones targeting developers! Instead they have a bunch of vague marketing copy that means nothing. It's a pet peeve.
My favorite is when they think they're keeping it short and to the point, with no bull. So, they'll have a hero section with copy like "Sharpen capacity. Scale across segments. Nuff said." No, not enough said, say more!
> Companies in general should do this, not just ones targeting developers! Instead they have a bunch of vague marketing copy that means nothing. It's a pet peeve.
This seems to appeal to purchasing teams. When you write what the app actually does suddenly it’s technical and the team doesn’t understand what is written any more.
https://www.useoptic.com/ is another one, which is a little more tailored to building & updating OpenAPI specs. Works well on live traffic and/or tests.
Very nice! Auto generating type information from looking at permutations of values is hard though. Q: Does this handle optional values? Also, being able to mark string field as "enums" and then collecting the possible values instead of just typing it as "string" would be mega handy.
It doesn't have any way of determining which values are optional, so it doesn't make that distinction. Hear you on the enums, I'll take another look at what's possible without adding overhead.
108 comments
[ 4.6 ms ] story [ 165 ms ] threadI mean, and I'm asking as a backend dev, if you have to integrate with some API, you use the provided docs/swagger ui.
Why/when would you care to rely on an API integration when it's interface is not publicly shared?
It was always my step 1 towards Xxx. Keen to know what directions you were thinking?
I’d love to see more remixing on top of API’s websites typically only expose for their own use.
[1] https://github.com/ShishirPatil/gorilla
Let me know if you have any questions or feature request
https://python.langchain.com/docs/use_cases/apis
What would be particularly useful is if it saved token values and then (through search) joined them on the response of the auth call to get the initial token.
That way you could easily determine what auth call was needed to get you a token to use the endpoint.
Sometimes I click on a path parameter and it doesn't "create" it, even though there are several other examples in the list. Not sure if it's a bug, or something I'm doing wrong.
Overall, this is an absolutely wonderful tool and I've wanted something like this for a long time. Incredibly useful, thank you!!
I used https://vite-plugin-web-extension.aklinker1.io/guide/ before to have cross browser extension support. If you don't mind I could take a look to add firefox support (no guarantee)
[1] https://chromedevtools.github.io/devtools-protocol/
So if you can pass the equivalent of that in Firefox/other browsers to the insert method and switch things up a bit, it should be relatively straightforward. I will think about pulling out the core logic into its own lib.
https://developer.chrome.com/docs/extensions/reference/devto...
https://developer.chrome.com/docs/extensions/reference/devto...
https://github.com/AndrewWalsh/openapi-devtools/blob/main/sr...
Their type definition for HAR request isn't exported https://github.com/DefinitelyTyped/DefinitelyTyped/blob/mast...
So I can't drop in replace the type on https://github.com/AndrewWalsh/openapi-devtools/blob/main/sr...
It's amazing to see a tool that simplifies the process of generating OpenAPI spec. this is the best showHN this year.
https://github.com/jonluca/har-to-openapi
I wanted to draw your attention to "normal" POST application/x-www-form-urlencoded <https://github.com/OAI/OpenAPI-Specification/blob/3.1.0/vers...> and its multipart/form-data friend <https://github.com/OAI/OpenAPI-Specification/blob/3.1.0/vers...>
The latter is likely problematic, but the former is in wide use still, including, strangely enough, the AWS API, although some of their newer services do have an application/json protocol
I know that's a lot of words, but the tl;dr would be that if you want your extension to be application/json only, then changing the description to say "OpenAPI specifications for application/json handshakes" would help the consumer be on the same page with your goals
https://github.com/alufers/mitmproxy2swagger
However, having the capability delivered in a browser extension is extremely handy!
Thanks for sharing this, I suspect this is going to be super useful for my work
I assume it monitors all XHR requests as you browse a website, and if the request/response matches [some criteria (e.g. is JSON?)] it will assume it's an API request and log it?
Is that correct?
If so, it will only work on websites where the frontend is implemented like a PWA, with lots of AJAX calls to fetch data, etc. For sites whose pages are all generated server-side, the extension won't generate any API schema, right?
Edit: Also how does it differentiate "API requests" with regular AJAX content fetching? If a website fetches some arbitrary content via an AJAX request (e.g. some lazy-loaded HTML), that's not an API request. That's just part of a website's layout.
> Instantly generate an OpenAPI 3.1 specification for any website or application just by using it
I want to know what this extension does that's different than me looking at the browser's Dev Tools > Network tab.
https://github.com/AndrewWalsh/openapi-devtools/blob/main/sr...
"Using" could mean navigating between pages, submitting data via forms, etc.
https://kit.svelte.dev/docs/load
(You can also use it to generate automated tests)
If you're interested: mish@stepci.com
I'll second/third the feature request for auto-including auth headers/calls (as many of the sites I'm trying to understand/use APIs from use persistent keys, and scraping these separately is just unnecessary extra time).
On that same note, I'd greatly appreciate keeping the initial request as a "sample request" within the spec.
I'd also greatly appreciate an option to attempt to automatically scrape for required fields (e.g. try removing each query variable one at a time, look for errors, document them).
Thanks for this :)
The spec produced will be incomplete (missing paths, methods, response variants, statuses). For that you should use a framework like Fastify, NestJS, tsoa, FastAPI, which have built-in OpenAPI support.
Can be very valuable for reverse-engineering though :)
https://www.akitasoftware.com/
> Akita makes monitoring and observing system behavior accessible for every developer. Quickly discover all your endpoints, see which are slowest, and learn which have errors
Translation: Install a Docker extension that intercepts and inspects your network requests to infer the shape of your API.
I feel like when you're targeting developers, you should quickly explain what it is you actually do.
My favorite is when they think they're keeping it short and to the point, with no bull. So, they'll have a hero section with copy like "Sharpen capacity. Scale across segments. Nuff said." No, not enough said, say more!
This seems to appeal to purchasing teams. When you write what the app actually does suddenly it’s technical and the team doesn’t understand what is written any more.
so I can be typing in the URL bar for any website I have landed on in the past and tab through all the available routes?
e.g.
- news.ycombinator.com_
- news.ycombinator.com/new
- news.ycombinator.com/submit
- news.ycombinator.com/show
etc.