Ask HN: How do you manage skills files?
How do you find skills, keep them organized, and make sure they actually work? Do you keep improving them over time?
I believe skills will eventually be eating by model capabilities, but until then I'm just looking for a better way to manage things.
321 comments
[ 0.22 ms ] story [ 17.1 ms ] threada model capability is never going to fill in an unknowable blank that a custom skill (or whatever equivalent your paradigm supports) can.
a model might have the cleverness to whoami and look through the .ssh folder for keys and evidence of past connections when asked to connect to bob, but a skills file can just easily say "We connect to bob using key Z and user X." so that the operation gets done without all this nonsense needless inference as far into the future as the information is valid for.
a concise information dense skill is going to always dominate on tokens-burnt for any given task that requires insider knowledge. it simply gets rid of the entire investigative phase of work.
This of course is from my own experience writing code, where agents are already good at software engineering conventions. This probably doesn't hold as well for other tasks, say writing marketing copy with a unique voice
For now, I keep skills pretty minimal - single sentence prompts I send all the time, like "Remove all the slam poetry from the docs in this repo."
I also tend to share often. All skills go into a repo my team can access. No pressure, use them, riff on them, add your own - sharing and engaging on how we do the work is more important than making everyone do the work the same way to me.
But, for custom use skills, ofc no model will be able to replace them and it's not efficient to try to do that as well. For this type of skills I create and maintain them by myself, my question was about "general use" skills, they are everywhere on the internet, how do you manage them?
I know everyone's down on MCP, but custom-built client side MCP tools are what I find useful instead. But that's me.
Can you explain what this means?
Eventually you arrive at building custom software that does a lot in the traditional way, but delegates certain tasks to the model where it makes sense or it's non-trivial/impossible to express via code.
it has all the skills/docs my particular application needs
i treat it as ADRs as it helps the AI understand the parts of the system it is working on
I try to keep my collection of community skills short, usually a few established names (mattpocock, mcollina, trailsofbit). And then I check new releases (or when mattpocock published a youtube video for instance :D)
> keep them organized
For skills I wrote myself, I have my own private github repo. I use skills like /commands most of the time, so I can tell if they work straight away.
For community skills, a package manager really helps. vercel-labs/skills and withastro/rosie are good options. I also built one myself: https://github.com/osrim/ski. It has some cool features like an update command and a security scan.
> Do you keep improving them over time?
In my global AGENTS.md I have a note to agents to explain any frustrations they had doing a task, and to suggest any skill/tool/AGENTS.md improvements. I am trying to keep AGENTS.md files small but still finding the balance.
We have a bootstrap script to deploy company-managed skills to each developer's "personal" skills. Hooks for codex and claude code try to refresh the skills on each startup.
I tried to control the execution of tasks performed by each project using claude.md within the project, but claude.md is only read at the beginning of each session, so it felt like the instructions weren’t being properly reflected.
So I revised the strategy to manage frequently used features in skill units. In doing so, instead of organizing skills by project, it was structured to be integrated into the general skills of the individual repo.
When skills are spread out across multiple projects and the number increases, it becomes impossible to keep track of which skills are available, so they end up not being used.
I also think that eventually, once Claude(model) advances, it will be able to replace most of the skills, so I believe registering and managing countless skills actually degrades performance.
For general tasks, the model seems perfectly capable of figuring out things itself, for project or environment specific tasks, I just put that information in the readme or agents.md file.
Caching certain scripts so it's not reinvented each time with risk of error/need reviewing.
Today Fable had to fetch a zip file from a web page with a eula prompt, then get at a file in a disk image in the zip.
This is something that will need to happen a lot as part of this project.
I asked Fable for a skill/script combo suitable for Haiku to accomplish the task, and now that task happens at minimal cost during an analysis run.
I will often make a skill out of the docs for any of the frameworks or libraries that we're using but with which I'm unfamiliar. When I'm creating that skill, I focus on idiomatic implementation and usage. It's not enough for the code to work—I want it to work "with the grain" and "through the front door", as it were.
By default, these models are just all too willing to reinvent the wheel and monkeypatch as they go.
Skills are more for things you do often. I run mutation tests, type check,linting,etc. I _could_ just prompt and copy/paste the same prompt each time I need to, or I can just run /tests.
I also have skills for specialized tasks I need every once in a while, like a ux skill, a text skill optimized for xyz, etc.
This documentation is its own git repo, and the agents.md file has an explicit instruction to update the docs when it has learned something general that can be useful in future sessions. I then occasionally review and prune those docs.
The description in the front-matter (at the top of the skill markdown file) is the only thing in the context and used by the agent to determine when to read in the rest of the skill file.
Skills itself may be lengthy so...
I'm not sure if this differs greatly from skills. Maybe my wording makes these "skills" less likely to be read at the correct times, but I haven't seen an issue.
skills are just an agents.md broken up into chunks so you can manage and share them separately. unfortunately there's no real good workflow for managing or sharing them separately, so most people end up treating them exactly the same way they do agents.md.
I create/edit/delete at least one skill per day. I can't imagine working effectively without those files.
The most common case: if I see something took AI too much time and tokens and it is done, I ask my Cursor immedietly after to save it as skill. So next time I do the same I just refer to skill. I don't need to remember the name of the skill, I just mention something like "do {explaining briefly the task}, you have done something similar in the past and it is saved as skill"
[0] https://github.com/ankitpokhrel/jira-cli
An example skill I have is SessionMiner, which is installed via post session hooks in Claude and Kiro, and analyzes the session, what was accomplished, and whether or not it should be turned into a skill, then when it summarizes it, the decisions it came to and either fires off a message to me for followup if it decides a new skill or tool should be built, or it catalogues the approach so that future analysis can identify trends in how I use the tools.
Over time it has built me a fairly decent stable of repeatable skills and tools, and highlighted process deficiencies and nominated process changes that I have pursued.
Another skill is a communications analysis skill; I started using it summer last year I think, and it scans my communications across a broad cross-section of my activity online. It tracks the commitments I make, ensures that I follow up with people that I might miss, ranks and scores my communication against my own personal targets that I set to make sure that I am communicating effectively. As a person who has had a decently successful career despite autism spectrum and unmedicated ADHD (I was medicated, but unfortunately each medication I tried had adverse side effects), it has made me much more effective in tracking work and following through, especially on the "boring" stuff that is actually critical to being a dependable team member, and effective partner for the teams I support.
Just a couple of examples.
I can see what the goals are there, and they do make sense I suppose, but I'm not confident that what you're handing off there can be handed off to that degree.
But maybe that is not the point and the point instead is to see what the LLM thinks would be correct, and then think about that and collect learnings about the world from it. It might not be right, but it still tells you how normal people think. So that's useful.
Just a very roundabout way to achieve that, but that's fine, I guess.
It is interesting how people delineate what is a "skill". A 50 word prompt can be called a skill. This process you are describing sounds like it is a highly authed and polling or hooked into multiple apps (slack?, text messages?, email?, forums, etc.) and then piping output to an LLM and to generate reports that it pushes to you based on output. You might need some data store to hold all the different communications locally as well.
That is almost a full on app/service but it is still just called a "skill".
Giving the agent an external tool to consider the sequencing of anything with dependencies led to some creative construction and offloading sequencing not unlike offloading calculation. (I also made a `bc` skill).
https://github.com/sj4nes/clanker-tools is where I've been riffing on this. My plan is to collect not "just skills" so much but "capsules" of reliable knowledge that agents can pull without confabulation. I'm already hitting the organizational stumbles, so this HN thread is right-on-time.
In the current version of my setup, I've decided to accept that tradeoff.
But it would also be interesting to check whether agent behavior can be controlled well enough by a skill-management skill telling them to synchronously commit any changes with their signature; that would get the best of both worlds.
Because it's obviously written by AI.
https://agent-plugins.org/
And sometimes it doesn't follow the instructions well. I have a skill for that too: it tells the agent, given what it knows about attention and LLM:s in general, to evaluate the instructions and the mistake the LLM made, try to diagnose why it didn't follow the instructions as expected, and come up with an improvement of the skill based on that diagnosis.
Making sure they actually work? Trial and error, mostly. I know some folks have tried auto-researcher approaches, but I haven't found that to be the best use of time in my work.
All skills, MCPs, CLIs, etc. live inside of it. I have it symlinked to all my dev machines so that it doesn't have to be an MCP.
`capsule` is then progressive to dozens of skills/tools thru `capsule` -- ex. `$capsule plannotator [args]`.
In some harnesses, I make it human-invoke only, and call it directly. In others, I let the model invoke it, and it has a top-level description that hints at what's inside.
Maximal context/session start control and capability extension.
Another thing i discovered is less is more (in case of skills as well)., don’t add lots of skills., keep them very handful - I’ve got 9 skills so far (many people have 100s installed from marketplaces and plugins)
This is probably less relevant for code that exists a ton in the LLM training data already as an llm is probably competent to some degree in that anyway.
A big caveat here is though that now you need to treat your skills repo very carefully as mistakes in there can easily spread to all of the new code you write using a coding agent.
Source at navikt/copilot
Git is of course required for this but is not enough so I built a tool to do just that: https://github.com/genged/capshelf
Using capshelf I manage my skills across projects. When I start a new project I can just: $ capshelf add security-review
From the skill repo.
And if I create a new skill I can promote it to the repo so everyone can install it: $ capshelf promote security-review
It pins the skill content hash so there are no unexpected edits that can break your flow.
It also supports MCP configs and agent configs.
- Keep them organised in software repos that you install with symlinks for all coding harnesses that you have. Progressive disclosure based on the frontmatter does the rest.
- I make sure they work with AI evals. Think of them like integration tests to prove behaviour. They're useful to optimize your flows. I try to make my skills be mostly a translation between natural language and good small fast tools that they call.
- I change them as a new problem arises. Not just because.
Skills can't be eaten by model capabilities if skills represent a workflow that is custom to my team or my person.
I wrote about a good mental model in the past:
https://alexhans.github.io/posts/series/evals/building-agent...
Like, ok, I have a debugging skill, now how do I make evals except for the most trivial things?
This is also my biggest gripe with AI. I.e. for specifications, no matter what hype machine I tried, it never fulfilled my criterias, which are: easily verifiable, concise, small specs. Hence I built https://github.com/RicardoMonteiroSimoes/Yamlet initially for claude code, but then decided to use extend it for pi.dev. I now have a dedicated docker image for pi.dev, that only contains Yamlet plugin, and whenever I work on spec I spin it up.
The end result is a .yaml file that easily works in git + git diff, so that I can then proceed with the technical specs-
Skills are for packaging instructions for how to interact with your organizations homebrew process and tools. By definition skills shouldn’t be useful outside of your org because they’re just docs and third party tools already have them for humans.
To be fair it's the only skill i have/use but I got real tired of explaining the same 12 things over and over. Having it document every incident means I have a dense library of every problem we've run into over the last six months which helps identify recurring problems for RCA
The idea would be that if you already know what you want from an autonomous system, you don't need to verify manually every time and instead just run these tests to see if there's any regression of any kind. Generally I recommend structure output and evals that are just a plain assertion, if possible. Cheaper, faster, deterministic assertions.
Does that make more sense?
Though most of the time my skills are just things I found useful and could avoid repeating myself by having as a skill.
That I also use it to route model used with https://github.com/flurdy/pi-skill-model-router is also a reason
Think ~/.codex/skills/<symlink-to-myskill-a/
Same for ~/.Claude or any other tool that supports skills.