Launch HN: Human Layer (YC F24) – Human-in-the-Loop API for AI Systems
What's really exciting is that we're enabling teams to deploy AI systems that would otherwise be too risky. We let you focus on building powerful agents while knowing that critical steps will always get a human-in-the-loop. It's been dope seeing people start to think bigger when they consider dynamic human oversight as a key ingredient in production AI systems.
This started when we were building AI agents for data teams. We wanted to automate tedious tasks like dropping unused tables, but customers were (rightfully!) opposed to giving AI agents direct access to production systems.
Getting AI to "production grade" reliability is a function of "how risky is this task the AI is performing". We didn't have the 3+ months it would have taken to sink into evals, fine tuning, and prompt engineering to get to a point where the agent had 99.9+% reliability—and even then, getting decision makers comfortable with flipping the switch on was a challenge. So instead we built some basic approval flows, like "ask in Slack before dropping tables".
But this communication itself needed guardrails—what if the agent contacted the wrong person? How would the head of data look if a tool he bought sent a nagging Slack message to the CEO? Our buyers wanted the agent to ask stakeholders for approval, but first they wanted to approve the "ask for approval" action itself. And then I started thinking about it... as a product builder + owner, I wanted to approve the "ask for approval to ask for approval" action!
I hacked together a human-AI interaction that would handle each of these cases across both my and my customers' Slack instances. By this time, I was convinced that any team building AI agents would need this kind of infrastructure and decided to build it as a standalone product. I presented the MVP at an AI meetup in SF and had a ton of incredible conversations, and went all in on building HumanLayer.
When you integrate the HumanLayer SDK, your AI agent can request human approval at any point in its execution. We handle all the complexity of routing these requests to the right people through their preferred channels (Slack or email, SMS and Teams coming soon), managing state while waiting for responses, and providing a complete audit trail. In addition to "ask for approval", we also support a more generic "human as tool" function that can be exposed to an LLM or agent framework, and will handle collecting a human response to a generic question like "I'm stuck on $PROBLEM, I've tried $THINGS, please advise" (I get messages like this sometimes from in-house agents we rolled out for back-office automations).
Because it's at the tool-calling layer, HumanLayer's SDK works with any AI framework like CrewAI, LangChain, etc, and any language model that supports tool calling. If you're rolling your own agentic/tools loop, you can use lower level SDK primitives to manage approvals however you want. We're even exploring use cases where HumanLayer is used for human-to-human approval, not just AI-to-human.
We're already seeing HumanLayer used in some cool ways. One customer built an AI SDR that drafts personalized sales emails but asks for human approval in Slack before sending anything to prospects. Another uses it to power an AI newsletter where subscribers can have email conversations with the c...
205 comments
[ 3.7 ms ] story [ 243 ms ] thread1. fire the async request, 2. store the current context window somewhere, 3. catch a webhook, 4. map it back to the original agent/context, 5. append the webhook response to the context window, 6. resume execution with the updated context window.
I have some ideas but I'll save that one for another post :) Thanks again for reading!
The cost of this is that you have to be careful in creating new versions of the workflow that are backwards compatible, and it's hard to understand backcompat requirements and easy to mess up. And, there's also additional infra you need, to run the Temporal server. Temporal Cloud isn't cheap at scale but does reduce that burden.
but we're all probably better off not investing that wheel
do you think something like langgraph state is sufficient?
I've also been working on a solution to this problem via long-polling tools.
[1] https://github.com/modelcontextprotocol
Again, genuinely looking to learn - where does MCP fit in for async/headless/ambient agents, beyond a solid protocol for remote tool calls?
or is there a cleaner way to do that?
i do think that AI-calling-tools is insufficient to provide bidirectional communication rails for user input/review though...not disagreeing just maybe thinking out loud a little here
A few things come to mind, divide the problem into chunks that can be solved in parallel by many people. Crowd source your platform so there are always people available with a very high SLA, just like cloud servers are today.
1. Run the whole process in a workflow function. Give the run a unique ID, which can be used to automatically deduplicate runs, and will be used to look up the workflow later. This fires off the request in a "step" function, and calls `recv` to wait for a response. The request should include a key that can be used to calculate the workflow ID. 2. DBOS automatically "stores the context somewhere" because of the `recv` 3. A separate HTTP handler in DBOS catches the webhook, and uses the key in the response to calculate the ID of the workflow from #1. 4. The HTTP handler calls `send` with that ID, so that the workflow can pick up whatever response is sent 5. The original workflow resumes with the response from `send` 6. The original workflow can do whatever it wants with the response
DBOS provides reliability around the whole thing (restarts any workflows in the case of any server restarts), and provides some tracing for the process out of the box, so it was quite simple to get started, and have it hosted in DBOS cloud , which also provides a public IP so that the external service can send a webhook response.
concurrency abstractions keep changing (still transitioning / straddling sync+threads vs. asyncio) - this makes performance eng really hard
package management somehow less mature than JS - pip been around way longer than npm but JS got yarn/lockfiles before python got poetry
the types are fake (also true of typescript, I think this one is a wash)
the types are fake and newer. typing+pydantic is kinda bulky vs. TS having really strong native language support (even if only at compile time)
virtual environments!?! cmon how have we not solved this yet
wtf is a miniconda
VSCode has incredible TS support out of the box, python is via a community plugin, and not as many language server features
ALSO - something I think about a lot - if a all/most of the HumanLayer SaaS backend was open source, would that change your thinking?
edit: I want to add here that while ycomb companies like yourself may have VC backing, a lot of us don't and do consider 500+/mo. base price on a service that is operations-limited to be a lot. You need to decide who your target audience is, I may not be in that audience for your SAAS pricing. This seems like a service that a lot of people need, but it also stands out to me as a service that will be copied at an extravagantly lower price. We have truly entered software as a commodity when I, a non-AI engineer, can whip up something like this in a week using serverless infra and $0.0001/1k tokens with gpt-o mini.
I think in building this some of the things that folks decided they don't want to deal with is like, the state machine for escalations/routing/timeouts, and infrastructure to catch inbound emails and turn them into webhooks, or stitch a single agent's context window with multiple open slack threads, but you're right, that can all be solved by a software engineer with enough time and interest in solving the problem.
I will need to clear up the pricing page as it sounds like I didn't do a good job (great feedback thank you!) - it's basically $20/200 credits, and you can pay-as-you-go, and re-up for more whenever you want. We are early and delivering value is more important to me than extracting every dollar, especially out of a fellow founder who's early. If you geniunely find this useful, I would definitely chat and collaborate/partner to figure out something you think is fair, where you're getting value and you get to focus on your core competency. feel free to email me dexter at humanlayer dot dev
If there’s something that truly has an incremental cost to you, like providing priority support, that goes into the “enterprise pricing” section and you need to figure out how to quote that separately from the service. My guess is most people don’t want to pay extra for that, or perhaps they’d pay for some upfront integration support but ongoing support is not too important to them. Idk, that’s just my guess here.
Coding agents can help more with tasks and not quite big entire massive platforms on their own. Humans may be able to scale much further and bigger with their skills.
My feedback: what’s there looks inviting. Email interaction is handy, other ways would be too.
If there was a low code way to arrange the humanlayer primitives for folks at the edge of using it, I think human tasks could meet something like this even broader. Happy to chat offline.
Onto your comment: The coding for coding agents is still kinda prototype. It feels like some folks quietly have setup a very productive workflow themselves for quite sometime.
Still, there no doubt you could ship production code in some cases - except ai needs to handle all the things development explicitly and implicitly checks before doing so.
Getting to build some things that became more than few orders of magnitude larger than planned, one can learn a lot from the deep experiences of others… and I’m not sure where that is in AI. Speaking to someone with experience and insight can provide some profound insight, clarification and simplification.
Still, an axiom for me remains: clever architecture still tends to beat clever coding.
The best code often is the code that’s not written and not maintained and hopefully the functionality can be achieved through interacting with the architecture.
This approach is only one way, but it can take both domain knowledge and data knowledge, to put in enough a domain and data driven design relative to how well the developer may know the required and anticipated needs.
The high end of software development is many leagues beyond even what I just described. There’s a lot of talk about 10x engineers, I’d say there can be developers who definitely can be 10x as effective or reach 10x more of the solution, than average.
If a lot of the code AI is modelled on is based on the body of code in repos, most on a wide scale may be average to above average at most, perfectly serviceable and iteratively updated.
Sometimes we see those super elegant designs of fewer tables and code that does near everything, because it’s developments 5th or 6th version creating major overhauls. It could be refactored, or if the schema is not brittle, maybe a rewrite in full or part of the exact same team is present to do it.
Today’s AI could help shed a light in some of those directions, relative to the human using it. This again says in the hands of an expert developer AI can do a ton more for them, but the line to automation might be something else.
There is agentic ai and human in the loop to still figure itself out, as well as how to improve the existing processes. 2025 looks to be interesting.
I think K8s ecosystem did this well but it required big cross-enterprise working groups that produced things like CSI, SMI, OCI, and before that could happen, there was like 5+ years of storming and forming that had to happen before the dust settled enough for enteprises to step forward and embrace k8s/containers as the new compute paradigm
maybe i'm overthinking things.
onto the coding things -
> clever architecture still tends to beat clever coding
love this
> best code often is the code that’s not written and not maintained
it's too bad not-writing-code isn't as satisfying as deleting code
> it’s developments 5th or 6th version creating major overhauls
yeah the best agent orchestration architecture I'm aware is on interation 4 going on 5. I told him to open source it but he said "its .NET so if I open source it, nobody will care" XD
> There is agentic ai and human in the loop to still figure itself out, as well as how to improve the existing processes. 2025 looks to be interesting.
i'm stoked for it
There are lots of other process managers that could fit well with agents as well, python and not that could be reasonable bolt-ons or extensions.
I'd be very happy to check out the best agent orchestration architecture you're referring to, email is in my profile if easier :). 1
In terms of open sourcing, I don't think anyone would have an issue with .NET, I wouldn't. Many businesses have exposure to Microsoft 365 already and using Azure, kind of a massive built in audience.
I wasn’t sure if this would be relevant to you or useful at all, but it’s a quick solution I built for HITL workflows. Happy to hear your thoughts or if you think it’s applicable!
We're a medical device company, so we need to do ISO13485 quality assurance processes on changes to software and hardware.
I had already been thinking of using an LLM to help ensure we are surfacing all potential concerns and ensure they are addressed. Partly relying on the LLM, but really as a method to manage the workflow and confirm that our processes are being followed.
Any thoughts on if this might be a good solution? Or other suggestions by other HN users.
Hey, if you're specifically looking for providing deterministic guardrails around agent calls, I'm solving that particular problem.
We're sort of an "RPC layer for tools with reasoning built in", and we integrate with human layer at the tool level as well.
We're operating a bit under the radar until we open-source our offering, but I'm happy to chat.
From what I've seen, this will bring the implementation needs for this kind of functionality down from "engineering team" to a single programmer.
I was wondering: have you thought about automation bias or automation complacency [0]? Sticking with the drop-tables example: if you have an agent that works quite well, the human in the loop will nearly always approve the task. The human will then learn over time that the agent "can be trusted", and will stop reviewing the pings carefully. Hitting the "approve" button will become somewhat automated by the human, and the risky tasks won't be caught by the human anymore.
[0]: https://en.wikipedia.org/wiki/Automation_bias
I think we have this problem with all AI systems, e.g. I have let cursor write wrong code from time to time and don't review it at the level I should...we need to solve that for every area of AI. Not a new problem but definitely about to get way more serious
Intriguingly, it's rather similar to what we see with LLMs - you want to really activate the person's attention rather than have them go off on autopilot; in this case, probably have them type something quite distinct in order to confirm it, to turn their brain on. Of course, you likely want to figure out some mechanism/heuristics, perhaps by determining the cost of a mistake, and using that to set the proper level of approval scrutiny: light (just click), heavy (have to double confirm via some attention-activating user action).
Finally, a third approach would be to make the action undoable - like in many applications (Uber Eats, Gmail, etc.), you can do something but it defers doing it, giving you a chance to undo it. However, I think that causes people more stress, so it’s rather better to just not do that than to confirm and then have the option to undo. It’s better to be very deliberate about what’s a soft confirm and what’s a hard confirm, optimizing for the human in this case by providing them the right balance of high certainty and low stress.
I also like that idea of:
not just a button but like 'I'm $PERSON and I approve this action' or type out 'Signed-off by' style semantics
If the table name is SuperImportantTable, you might gloss over that, but if you have to type that out to confirm you're more likely to think about it.
I think the "meat space" equivalent of this is pointing and calling: https://en.m.wikipedia.org/wiki/Pointing_and_calling (famously used by Japanese train operators)
By just doing something manually 10-100 times, and collecting feedback, both understanding of the problem, possible solutions/specifications can evolve orders of magnitude better.
I think uncle bob or martin fowler said "don't buy a JIRA until you've done it with post-its for 3 months and you know exactly what workflow is best for your team"
Coding with English (prompting) is often most useful where existing ways of coding (an excel formula) can’t touch.
Using llms to evaluate things like an excel formulas instead of using excel doesn’t feel in the spirit of using this ai’s power.
What I don't understand from quickly skimming your description and homepage: Do you source/provide the humans in the loop? That's a good value add, but how do I automatically / manually vet how you do the routing?
- providing the humans can be super valuable, especially for low-context tasks like basic labeling
- depending on the task, using internal SMEs might yield better results (e.g. tuning/phrasing a drafted sales email)
I wonder if you can achieve this workflow by just using prompt and the new Model Context Protocol connected to email / slack.
https://www.anthropic.com/news/model-context-protocol
If you want to tell a model to send a message to slack, sure, give it a slack tool and let it go wild. do you see a way how MCP applies for outer-loop or "headless" agents in a way that's any different from another tool-calling agent like langchain or crewai? IT seems like just another protocol for tool calling over the stdio wire (WHICH, TO BE CLEAR, I FIND SUPER DOPE)
Yeah it still takes a bit of work, but it's more extendable (can swap slack to email) and more versatile (human can message agent to interrupt or ask for clarification).
I haven't used langchain (beyond RAG) or crewai, not sure what was capable before.