HN Explorer

Show HN: Lowfat – pluggable CLI filter that saved 91.8% of my LLM tokens (github.com)

156 points by zdkaster ↗ HN
Hi HN, not sure if anyone would be interested, but just wanted to share that I've been maintaining my small tool called 'lowfat' that helps me filters some of my verbose CLI output. It's a single binary, works as an agent hook or a shell wrapper. It has a plugin system to customize filters per command.

The idea is pretty simple: agents don't need the full kubectl get -o yaml or any 10k-line dump to make decisions. So that lowfat sits in between, strips the noise, and passes through what matters. Here's my real report after 2 months of personal use:

  lowfat history --all

  lowfat plugin candidates
  ─────────────────────────────────────────────────────────

    #  command                    runs   avg raw      cost   savings  source    status  
    1  kubectl get                101x     14.4K      1.5M     93.9%  plugin    good    
    2  grep                       103x     13.5K      1.4M     96.2%  plugin    good    
    3  git diff                    81x       995     80.6K     57.9%  built-in  good    
    4  kubectl                     90x       485     43.6K     33.6%  plugin    good    
    5  docker                     127x      5.5K    693.6K     96.1%  built-in  good    
    6  ls                         489x       117     57.3K     56.2%  built-in  good    
    7  find                        30x     16.5K    495.0K     95.5%  plugin    good    
    8  git show                    63x       490     30.9K     38.0%  built-in  good    
    9  git                        177x       368     65.2K     76.1%  built-in  good    
   10  git log                     86x       556     47.8K     78.5%  built-in  good    
   11  kubectl logs                 5x      3.6K     17.8K     43.0%  plugin    good    
   12  git status                  86x       152     13.1K     58.0%  built-in  good    
   13  docker ps                   20x       467      9.3K     52.8%  plugin    good    
   14  kubectl describe             6x       656      3.9K      1.2%  plugin    weak    
   15  docker images                9x       940      8.5K     61.8%  built-in  good    
   16  k get                        2x      2.1K      4.2K     35.9%  plugin    good    
   17  terraform                   10x       395      3.9K     32.1%  plugin    good    
   18  git commit                  32x        77      2.5K      0.0%  built-in  weak    
   19  docker build                 8x       487      3.9K     37.6%  built-in  good    
   20  docker compose              22x       979     21.5K     89.4%  built-in  good    

  total: 4.4M raw → 4.1M saved (91.8%)
My toolset above is kind limited, but it works pretty well for my usecase without any interruption Kinda help me not reaching the token limit for my company Bedrock limit usage and keep optimizing the saving on the go for later usage.

But, why not alternatives (https://github.com/zdk/lowfat#alternatives) ? The answers are: - My goal is to make the core lightweight but extensible via plugins i.e. not trying to bundle every command in the installed binary so that people own their output filters. - Customizable per usecase via plugin or filter pipelines as I am using my own toolset. - Customizable for non-public CLI tools, for example, some enterprise might have their interal CLI tools that public won't have access. - People should own their data. So the design is local-first, No telemetry forever. - I kinda love UNIX-style composible pipes, so lowfat-filter has implemented this style. - Be able to adjust aggressiveness of the filter, so we can control that we won't strip something the agent needed.

GitHub: https://github.com/zdk/lowfat

Anyway, if anyone is interested, feedbacks and ques...

40 comments

[ 3.5 ms ] story [ 73.1 ms ] thread
devdoc83
How do you handle the risk of stripping out the exact stack trace the agent needed? That seems like the hard tradeoff here.
itsthecourier
gonna ask the same... do far it's has been manually choosing what's useful in each command for the agents?
ramon156
In a perfect world the LLM needs to be very explicit on what it wants to read
zdkaster
It has the strip aggressiveness level suport. You can tune up 3 levels for each template output of your stacktrace using lowfat-filter dsl, shellscript or python.
alex7o
I would like to have deeper comparison with alternatives like rtk, which are already fast and written in rust, also the previous comments mentioned something that has been a know problem with rtk that it sometimes strips the thing that the llm needs (or expects, causing more work to need to happan not less)
zdkaster
I have just put the comparison in the repo in case you want to checkout.
threecheese
The docs are missing any examples of what this does, instead showing _how_ it works - and only for the codebase itself, rather than the behavior of the app.

What would be useful:

  - examples of text that can be filtered, and why that would be valuable
  - a data flow diagram of runtime behavior, showing how filtering removes unnecessary context
fcanesin
I am thinking that a small tool that simply refuses to pass large CLI output to the LLM and warns it to filter the results before reading would achieve this better as the LLM would be forced into thinking and writting the filter itself.
itsdesmond
Have terms been established to describe these types of tools? How do I refer to small utilities to perform specific transformations to LLM behavior? CLI filter seems pretty good to describe this tool conversationally but not so much when searching, they some low cardinality keywords.
jemmyw
I've tried rtx and lean-ctx and these tools seem to end up confusing the agent more than helping. Any saving is irrelevant if the agent decides to work around the tool and makes even more calls than it would otherwise.

I don't know about cost saving, but if it's keeping the context size down I've had a lot better results using subagents to keep a higher order conversation clean for longer.

pradeep1177
Would this have any impact on the response quality from the agent?
wood_spirit
I have my own llm wrapping harness, which does this and has a few more tricks. For example, it doesn’t have a lot of mcp but it does have search_mcp and load_mcp tools (and search_skills) so the llm can find what it needs when it needs it without bloating the normal baseline context. The LLMs have proved really good at using them. There is also a waypoint tool they can use to record their thinking in the context without it being the final output. Am thinking about a search_expert to find colleagues it can bring into conversations too. And a lot of other stuff.

Pro tip they worked well for me with response truncation: in the truncated output, say that the full text is available in /tmp/whereever.txt - that way, the llm will be able to query and read more using built in tools without reissuing the big tool call.

unphased
great approach. I did that with my opencode based setup as well, it's neat and fun to tune skills and mcp loaders and stuff. Then i got fed up with opencode's design limitations. And then, my own harness work is on hold in favor of a harness-puppeteer paradigm, but that one has also been on hold! I'm mostly currently pulling on the thread of making it easier just to review the voluminous conversation turns!
tegiddrone
Still learning myself, but I've seen MCP tools just lightly wrap upstream json-body REST APIs. Works. But not only is the json structure more tokens but often the model just needs a small subset of fields in the payload.
cityofdelusion
This is a nice little project but I’m weary of sensationally inaccurate titles for stuff like this and the infamous caveman mode. It doesn’t save 91% of tokens: it reduced in one user case 91% of output tokens on the raw CLI output. I am being pedantic about this because these sorts of claims go viral and are inaccurate.

A proper benchmark will compare a large sample of identical prompting with and without the tool, against a specific harness. Once you apply Amdahl’s law, there is no way this saves 91% of tokens holistically, which the title implies.

I work in a non-tech company and these sorts of things keep going viral, with no understanding and with no comprehension of what is actually going on. Engineering is gone and cargo cult magical incantations are in.

[deleted]
(comment deleted)
tuo-lei
the bigger problem is agents defaulting to the broadest command possible. kubectl get -o yaml when a jsonpath query would give 1/50th the tokens. filtering after the fact works, but you're still paying for the round trip. better to teach the agent to ask narrow questions in the first place.
[dead] urax
[flagged]
avocadoking
Do you have any insight if LLMs sometimes get confused by your filters?
rahulyc
Great idea. I'm thinking if it could make sense to send the output to a cheap / local model to filter out only the bits that "matter" and pass that through - for the cost some extra time, but maybe it's worth it for saving tokens in the larger model.
sakuraiben
Would be interested to see what kind of eval results you get from this
clutter55561
Tools that remove the fat seem like a good idea, but I’m highly suspicious of their effect on the LLM’s reasoning.

LLMs were trained in the typical full-fat output found everywhere on the internet, and all of sudden they get a slightly different response that may look like nothing they have seen before.

Does that really save tokens in the long run?

davidetroiani
Add a comparison table between your repo and alternatives like rtk. I’m interested.
KuhlMensch
I'm glad this class of tool exists.

But it'll be a case of measuring first, then perhaps a staged integration of a tool like this.