Which specific model? and are tools available? I would say that there is lots of RL with tool use, but that the reasoning traces are not always perfect. In general, models think about things that might be wrong, but will recover from this poor line of thinking.
This was using llama-serve on my local which by default gave it current time & unhelpfully "get_info" which is specific to the browser
The full request was
```
{"messages":[{"content":"What animal lays the biggest egg?","role":"user"}],"return_progress":true,"sse_ping_interval":1,"stream":true,"tools":[{"function":{"description":"Returns the current local date and time in ISO 8601 format, with the IANA time zone name","name":"get_datetime","parameters":{"properties":{},"required":[],"type":"object"}},"type":"function"},{"function":{"description":"Get runtime info (OS name), may call when user asks about local files or shell commands","name":"get_info","parameters":{"properties":{},"required":[],"type":"object"}},"type":"function"}],"reasoning_format":"auto","reasoning_control":true,"backend_sampling":false,"timings_per_token":true}
```
Disabling tools removes the bit about mentioning instructions.
Just an interesting detail I noticed.
Do y'all have any penalty/regularization w.r.t. reasoning traces or is it pure SFT -> RL?
interesting, I didn't see it on the playground or on my local llama-server.
SFT -> RL. In our fine-tuning notebooks, we typically say that rollouts that go past the max context length (too much reasoning) get 0 reward, so there is slight pressure for shorter reasoning, but I don't think we have strong regularization. I think the agreed-upon recipe is to first make the model smarter (longer reasoning), then teach it to reason faster/denser.
Yes! LFM3-VL-2B is a very strong OCR model for its size, especially given how general a model it is. You can also fine-tune it with a specific output format and it will dramatically improve. We have a Google Colab here: https://colab.research.google.com/github/Liquid4All/cookbook...
Working on the app with on-device transcription. Is interesting how the live mode is restarting the text every 30 sec, wonder how it can impact a thing that may be agreed over several minutes.
Say I would like to get action items out of an hour long recording, am I supposed to go straight from the audio with LFM3-Audio-2B? Or is it transcribe first and give the text to LFM3-1.7B?
What if I want to fine-tune the action items routine. How many labelled examples do you estimate such task may need?
It all depends on your hardware and throughput requirements. Suppose
1. I were building for my work laptop (M4 Max, 36GB RAM)
2. I had a batched use cases (transcribing + making action items for all meetings in a day)
I would use the LFM3-Audio-2B for ASR (audio -> text transcript) and then LFM3-1.7B for extracting particular action items and adding them to my tool of choice (Linear, Jira, Notes app, etc).
If you can fine-tune (100+ labeled samples, and a GPU), you likely can fine-tune LFM3-Audio-2B for audio -> action items in a single step.
Thank you for an update. Single step does sound exciting, look forward to check it out once I have the full system and data in place.
Would LFM3-Audio-2B provide timestamps and speaker labels for each action item? Did you guys train it for any of those?
unfortunately it's not a strong multi-speaker/diarized output audio model. It's a future goal but it doesn't have a timeline. If you have the hardware, I think fine-tuning might be possible.
unfortunately the models are still stronger without extra modalities. So LFM3-1.7B is better at pure text tasks than all other models. the LFM3-VL-2B is better at text/vision tasks than the Omni, the Audio model is a stronger ASR/TTS model than the Omni, etc.
It's a goal of ours to continue unifying training across modalities so that there is truly one model, instead of just this shared pre-trained backbone.
Did y'all do any particular work that makes these models great for moderation? One of the issues between cloud models vs local is that often people are trying to push the limit of what their hardware can do and it doesn't leave too much headroom for moderation to make the models a little more constrained for kids or schools to use locally, or just for family in general who don't want accidental unhinged results.
In the past I've noticed that some LFM were way better at moderation than much larger models. Not necessarily prejudging what is acceptable, but allowing you to interrogate content with a series of questions to customize the moderation. Other models I tested were far worse.
Decision models are like the next optimization in that direction, but none of the benchmarks for the popular models ever show how good they are at tasks like content moderation. The smaller and more efficient we can get that workload, the more that it can simply be on by default in a lot of use cases without feeling like you're giving up a lot of performance, creative freedom or accuracy. You can't have a high number of false positives or people would rather take control back.
We need local content moderation to be cool, efficient, accurate and configurable so that people can get the higher quality results which unsafe models tend to produce, while filtering out the unsafe results fast enough that it can even be done in an agentic way and fed back into the model to steer its results.
Can you describe more in terms of what moderation you want? I can say with confidence that we don't train for the typical adult-only content you can see on some "de-safe'd" models, but it's not clear to me exactly what you're asking about. We can also discuss over email if you prefer.
on moderation, we run an open guardrail benchmark at raxIT Labs and the hosted d1 models do well. d1.1 is #1 of 30 at checking model output for harmful content (94.4 balanced accuracy), d1-ultra is #1 on harmful user requests (88.2). we haven't run the new 2B open weights yet so no number for those. results and method: https://dm-as-guardrails.raxitlabs.com
broadly, Liquid focuses on picking architectures that have really high throughput/low latency that are close to full attention (slow) in terms of expressivity, and then we do the data work to close any remaining gap. so the big goal is to make it very very fast to encode data without any loss in quality.
One thing I have been using LFM 2.6 for is annotating emails, text messages, and other metadata. I keep it all ingesting into one giant database and annotate it all as it comes in on an RTX A4500. The LFM annotations are basically triage, and then slowly annotations trickle into my halo model based on lowest confidence (right now that's DeepSeek 4.1 Flash 1017) for an upgraded annotation. (Agents can then search this database for context).
Excited to roll out LFM 3 to replace it, looks like a big step up. Right now LFM 2.6 gets trustworthy annotations on just over 40% of data points. Just waiting for these to get into LocalAI.
I can highly recommend gliner2.5, been using it for tagging news articles, fine tuned on my home 5090 in a few minutes on 500 articles tagged by gpt 6.1, works fantastically. Deployed on hetzner running on a cpu .
gliner-style models are absolutely crazy, I learned about them a couple weeks ago when a co-worker fine-tuned an LFM encoder for promptable classification. as far as I know, you don't need to fine-tune; you can just prompt for classification. My co-worker's demo is here: https://huggingface.co/spaces/LiquidAI/prompt-routing
You're doing something similar to what I've been prototyping for the automatic metadata enrichment part. So far I've been using deterministic older techniques + manual for my personal data stream, but I'm trying to learn what the state of the art is for efficiently annotating data with LLMs and newer ML.
This seems interesting, but if anyone else is also doing a similar thing, I'd appreciate tips on how to approach it. So far, I think I have some small RAG model in a demo or the very small qwen models. I think fine tuning seems interesting too. Some of the decision models applied in layers (like a binary search on a pre-generated corpus) could be fast enough to help too?
idk too many options, i'm sure this is a solved problem lol
It's definitely not a solved problem, and it has been kicking my ass. I don't just want metadata tagging/categorization, I want something that identifies the networks between the information, graduates topics to actual objects, and routinely consolidates/synthesizes narratives that can be easily looked up.
That's very cool. I'm curious if you need a GPU for your use case (email and text triaging). There are some open source alternatives[0] that run/train on CPU, so you can self-host or run locally (and train on your own data for better accuracy), would something like that work for you?
You might also consider the decision models, d1-2B is very strong and good for calibrated classification. Would be happy to share links/resources if you're interested.
I initially tried annotating everything three times, then judging the annotation with the same model. That failed predictably, so yeah just logprobs right now, judged on a hand-annotated dataset. Mostly correlated - ~80%.
Agreed not, but I think we are also moving towards a place where harnesses have code mode built in and where data swizelling and tool use is something the LLMs have to be competent at. I think that's present enough here!
And to beat Qwen3.5-4B with such a capable model is amazing, on top of so many win/draws against 9b in the more general arena.
I can definitely see there being some roles for local agents that offload shell scripting & doing some scut work. I don't know what speed to expect but like, if someone has a decent NPU or GPU and can just get a bunch of basic systems work done with a small local model that could really speed up a lot of workflows that I see day after day after day.
I’ve been using 2.6B as my main home assistant model and it’s been good. I switched to Ling 3.0 Tiny since it’s faster with A1B. Really hoping 1.7B is good enough (2.5 version was too dumb for HA) so I can run everything on one machine.
As mentioned by another commenter, I mean Home Assistant. Essentially the main orchestrator for all my smart home things but it doesn’t limit to just smart home. You can add calendar and location sharing to HA.
Just tried both the Omni model and the text only model, both as mlx versions, in lm studio omlx and rapid mlx, none of them could actually fire the model up, kept saying safetensors cant be read (yes I realize Claude should hear this not this thread).
Surprisingly (or maybe not?), LFM3-VL-2B gave me better results than LFM3-1.7B on text-only prompts (general knowledge, chatting) through liquid playground.
As an example, when asked “what does a cow say?”, LFM3-1.7B said that a cow can say “Oink”, and when I asked the same question to LFM3-VL-3B, it (correctly) said “moo”. I assume this is lower quant problem, though: Q8 versions I’ve played with through llama both give the correct “moo” answer.
First class hardware support makes this really exciting and different. On device processing is cheaper and faster, critical for real time applications of AI. Really excited for the rise of local, smart intelligence!
I'm testing this out as a dumb scout model alongside Qwen 3.8 Flash Next (IQ3_S, strata). Currently have it labeled as a "dumb but fast scout, findings should be verified" since some initial testing shows useful-but-sometimes-inaccurate.
on a 7900XT (20GB VRAM, ~800GB/s IO) w/ full context:
54 comments
[ 4.3 ms ] story [ 87.9 ms ] thread> However, the instructions say I must use available tools when answering questions.
On "What animal lays the biggest egg?" with no system prompt
I assume this is something baked into the model through lots of RL with tool use?
The full request was
``` {"messages":[{"content":"What animal lays the biggest egg?","role":"user"}],"return_progress":true,"sse_ping_interval":1,"stream":true,"tools":[{"function":{"description":"Returns the current local date and time in ISO 8601 format, with the IANA time zone name","name":"get_datetime","parameters":{"properties":{},"required":[],"type":"object"}},"type":"function"},{"function":{"description":"Get runtime info (OS name), may call when user asks about local files or shell commands","name":"get_info","parameters":{"properties":{},"required":[],"type":"object"}},"type":"function"}],"reasoning_format":"auto","reasoning_control":true,"backend_sampling":false,"timings_per_token":true} ```
Disabling tools removes the bit about mentioning instructions.
Just an interesting detail I noticed.
Do y'all have any penalty/regularization w.r.t. reasoning traces or is it pure SFT -> RL?
SFT -> RL. In our fine-tuning notebooks, we typically say that rollouts that go past the max context length (too much reasoning) get 0 reward, so there is slight pressure for shorter reasoning, but I don't think we have strong regularization. I think the agreed-upon recipe is to first make the model smarter (longer reasoning), then teach it to reason faster/denser.
I'm curious if this model would be useful for OCR or at least text documents parsing (PDF, DOCX, etc.), what do you think?
1. I were building for my work laptop (M4 Max, 36GB RAM)
2. I had a batched use cases (transcribing + making action items for all meetings in a day)
I would use the LFM3-Audio-2B for ASR (audio -> text transcript) and then LFM3-1.7B for extracting particular action items and adding them to my tool of choice (Linear, Jira, Notes app, etc).
If you can fine-tune (100+ labeled samples, and a GPU), you likely can fine-tune LFM3-Audio-2B for audio -> action items in a single step.
In other words: Should I select the model that matches my desired capability set or just use Omni?
It's a goal of ours to continue unifying training across modalities so that there is truly one model, instead of just this shared pre-trained backbone.
In the past I've noticed that some LFM were way better at moderation than much larger models. Not necessarily prejudging what is acceptable, but allowing you to interrogate content with a series of questions to customize the moderation. Other models I tested were far worse.
Decision models are like the next optimization in that direction, but none of the benchmarks for the popular models ever show how good they are at tasks like content moderation. The smaller and more efficient we can get that workload, the more that it can simply be on by default in a lot of use cases without feeling like you're giving up a lot of performance, creative freedom or accuracy. You can't have a high number of false positives or people would rather take control back.
We need local content moderation to be cool, efficient, accurate and configurable so that people can get the higher quality results which unsafe models tend to produce, while filtering out the unsafe results fast enough that it can even be done in an agentic way and fed back into the model to steer its results.
broadly, Liquid focuses on picking architectures that have really high throughput/low latency that are close to full attention (slow) in terms of expressivity, and then we do the data work to close any remaining gap. so the big goal is to make it very very fast to encode data without any loss in quality.
Excited to roll out LFM 3 to replace it, looks like a big step up. Right now LFM 2.6 gets trustworthy annotations on just over 40% of data points. Just waiting for these to get into LocalAI.
This seems interesting, but if anyone else is also doing a similar thing, I'd appreciate tips on how to approach it. So far, I think I have some small RAG model in a demo or the very small qwen models. I think fine tuning seems interesting too. Some of the decision models applied in layers (like a binary search on a pre-generated corpus) could be fast enough to help too?
idk too many options, i'm sure this is a solved problem lol
0: http://jeffyclassify.com/
Liquid has also been very good about keeping speed in mind, and the writeup here makes me expect similar. Really good stuff.
And to beat Qwen3.5-4B with such a capable model is amazing, on top of so many win/draws against 9b in the more general arena.
I can definitely see there being some roles for local agents that offload shell scripting & doing some scut work. I don't know what speed to expect but like, if someone has a decent NPU or GPU and can just get a bunch of basic systems work done with a small local model that could really speed up a lot of workflows that I see day after day after day.
It's mediatek NPU claims to handle ONNX. That plus the 12GB ram makes the wonder if I can get this phone to be a mini-agent.
on a 7900XT (20GB VRAM, ~800GB/s IO) w/ full context: