I am thinking to fine-tune it to recognize better my handwriting. It already works quite well by default, but my writing is just horrible, so it got trouble sometimes.
Awesome guide, shame how a couple of the Qwen leads got kicked out and replaced with more “business” minded leadership. Hopefully this doesn’t mean the end of the open source era from Qwen.
Fine tuning is a story that is nice to tell but that with modern LLMs makes less and less sense. Modern LLMs are so powerful that they are able to few shot learn complicated things, so a strong prompt and augmenting the generation (given the massive context window of Qwen3.5, too) is usually the best option available. There are models for which fine tuning is great, like image models: there with LoRa you can get good results in many ways. And LLMs of the past, too: it made sense for certain use cases. But now, why? LLMs are already released after seeing (after pre-training) massive amount of datasets for SFT and then RL. Removing the censorship is much more efficiently done with other techniques. So I have a strong feeling that fine tuning will be every day less relevant, and already is quite irrelevant. This, again, in the specific case of LLMs. For other foundational models fine tuning still makes sense and is useful (images, text to speech, ...).
Qwen filtered out a lot of porn during data curation, and a finetuned model can perform much better than context engineering. Abliteration can only remove censorship, not add something non-existent in the training data.
The problem with this is context. Whatever examples you provide compete with whatever content you want actually analyzed. If the problem is sufficiently complex, you quickly will run out of context space. You must also describe your response, in what you want. For many applications, it's better to fine-tune.
Remember how the tab-next-action model from Cursor was all the rage ~2 years ago when they launched it? That was a fine-tune of a ~70b model (they kinda alluded to this in a podcast).
> where latency matters more than raw accuracy – think industrial inspection
Huh? Why would industrial inspection, in particular, benefit from lower latency in exchange for accuracy? Sounds a bit backwards, but maybe I'm missing something obvious.
This reply is entirely AI generated. You guys are trying to find reason in a hallucination. It's unfortunately impossible to put into words what the "LLM smell" is at this point, but I trust someone else who spends a lot of time reading LLM output can back me up on this.
I've seen these agent-written fake anecdotes on Twitter, Reddit, and now here, all with the exact same formatting. They pretend to be real people with real anecdotes, but they're all completely made up.
The two day old account is an obvious hint but I got to be honest, the content didn't look suspicious on first read. I know you touched on it above, but what do you think triggered your AI generated thought ?
Unfortunately, this looks to only cover the larger MoE models. I imagine the smaller models are what most people would target. 9B just dropped two days ago, so not surprised it’s not explicitly documented, but does use a hybrid mamba architecture that I expect needs some special consideration.
25 comments
[ 3.5 ms ] story [ 41.3 ms ] threadhttps://www.atredis.com/blog/2024/6/3/how-to-train-your-larg...
We are far from that still, for reliability in most applications you need fine tuning.
For any new modality you need fine tuning
For voice, image and video models you need fine tuning
For continual learning you (often) need fine tuning.
For any domain that is somewhat OOD you need fine tuning.
To fully ground a model you need fine tuning
Qwen filtered out a lot of porn during data curation, and a finetuned model can perform much better than context engineering. Abliteration can only remove censorship, not add something non-existent in the training data.
This guy did some great work in the age of Qwen 3.0: https://huggingface.co/chenrm/qwen3-235b-a22b-h-corpus-lora
Huh? Why would industrial inspection, in particular, benefit from lower latency in exchange for accuracy? Sounds a bit backwards, but maybe I'm missing something obvious.
7B on 15W could be any of the Orin (TOPS): Nano (40), NX (100), AGX (275)
Curious if you've experimented with a larger model on the Thor (2070)
I've seen these agent-written fake anecdotes on Twitter, Reddit, and now here, all with the exact same formatting. They pretend to be real people with real anecdotes, but they're all completely made up.
That’s wild. And scary.