It is definitely an interesting problem, because Portugal is a small enough country that the actual total corpus of available texts in (non-Brazilian) Portuguese is potentially problematic.
I'm not sure the direction should be to finetune a small local model for each country or language. These models are already not particularly great at information retrieval, so I doubt anyone would use them for questions like the author suggests (ie who was the president between X and Y). Similarly, they are a little too lightweight to be used for translations too.
If the budget is indeed so modest (5.5 million euros!), I would focus completely on preparing datasets and making sure all open cultural artifacts that we can find are well documented in them. That way every model, private or open, that gets trained in the future could better represent the culture and language of your country.
this is the type of question that should never ever be asked to an llm running on some A100 on the other side of the world, local llms are already more than capable to answer these
That idea is different than what most are talking here in other comments.
The grammar and vocabularies don't match, but I think the worst are the expressions. Both sides have *a lot* of expressions that vary per context and location.
There is no public website to use it, be it free or paid, the dataset is not public, the code is not public (The github URL in the article returns 404 ), the claimed model intelligence is so low that is pretty much useless at 32K context and massively inferior to GPT‑4o.
As per tradition in Portugal, some people managed to get 5.5 Million to produce nothing and no one is asking questions.
You want a better idea? Just fine tune the open source Kimi 2.6 with an open source Portuguese dataset, the cost would be under a million and we would be getting something useful.
It would be really nice to know what happened to 5.5 Millions whilst not being able to even provide a functional website to use the model.
I’ve noticed that ChatGPT is noticeably dumber in languages other than English. It even will confidently repeat common but wrong superstitions from the target language as if they were fact.
Amazing. Every country should have a foothold on AI, as it will have impact every area of a citizen's life
Quite cheap compared with most public spendings
Europe countries already produce very little. Let's not let the wave pass and end up in a future where Europe is continuously reliant on US and Chinese tech as usual. And their definitions of what truth is
20 comments
[ 3.3 ms ] story [ 44.7 ms ] threadTrying to force a LLM into a specific language makes you missed out on most of the world knowledge.
If the budget is indeed so modest (5.5 million euros!), I would focus completely on preparing datasets and making sure all open cultural artifacts that we can find are well documented in them. That way every model, private or open, that gets trained in the future could better represent the culture and language of your country.
this is the type of question that should never ever be asked to an llm running on some A100 on the other side of the world, local llms are already more than capable to answer these
The grammar and vocabularies don't match, but I think the worst are the expressions. Both sides have *a lot* of expressions that vary per context and location.
https://simianwords.bearblog.dev/why-domain-specific-llms-wo...
There is no public website to use it, be it free or paid, the dataset is not public, the code is not public (The github URL in the article returns 404 ), the claimed model intelligence is so low that is pretty much useless at 32K context and massively inferior to GPT‑4o.
As per tradition in Portugal, some people managed to get 5.5 Million to produce nothing and no one is asking questions.
You want a better idea? Just fine tune the open source Kimi 2.6 with an open source Portuguese dataset, the cost would be under a million and we would be getting something useful.
It would be really nice to know what happened to 5.5 Millions whilst not being able to even provide a functional website to use the model.
Quite cheap compared with most public spendings
Europe countries already produce very little. Let's not let the wave pass and end up in a future where Europe is continuously reliant on US and Chinese tech as usual. And their definitions of what truth is