As an Arab person myself (and not getting into the Palestine occupation debate). It's sad that a non-Arab speaking nation is creating LLMs to better understand Arabic but I am not aware of any Arab nation doing the same effort.
I would say that it's sad that LLMs are being used for mass surveillance, and such applications are things no government should be proud of deploying.
And it's things like this that are ripe to come to other countries. An interesting book that goes into this concept is "The Palestine Laboratory." Sad times indeed.
Why would LLM training allow arabic native speakers to "better understand Arabic"?
This isn't a tool to better understand arabic in some innocent sense, it's a compensation for israelis refusing to learn arabic and having trouble gathering intelligence from linguistic sources. When you're actively involved in the genocide of a people you'll also have problems finding informants among them, it's much easier under a less intense apartheid regime.
The West Bank population is the perfect development platform for this tech. They can't complain or revolt - they're already under occupation! Who else but Israel has such a resource at their disposal?
And unlike other nations that might build these tools for internal use only, it's basically certain that this tech will be sold to e.g. the US.
See: NSO Group, Cellebrite, Paragon, Oosto/Anyvision, Corsight. They've all had or have contracts with US domestic agencies that needed surveillance tech.
I think China is both a far better environment for this tech ... and we just don't know how bad it's become. Every metric says China's youth is doing horrible. Xinjang is doing terrible. Mining provinces are doing terrible. And by terrible we mean "economic growth for China as a whole is probably around -3%" (guesstimates from US banks, not Chinese government figures), so there isn't a single place that is doing well, it's just there's disaster provinces and there's catastrophe provinces.
Chinese (esp. youth) are locked up, there are a lot of stories about migration being denied on a large scale for Han Chinese, including within China itself (not allowed to move cities, like you saw long ago in the Soviet Union)
And yet we hear nothing. Meanwhile there's "social credit score", mass unemployment in the capital city, and expert LLM teams (that have put the best US and EU teams to shame 2 or 3 times now) ...
I think we'll suddenly find out China far ahead of absolutely anyone else in this field.
8 comments
[ 4.4 ms ] story [ 26.1 ms ] threadAnd it's things like this that are ripe to come to other countries. An interesting book that goes into this concept is "The Palestine Laboratory." Sad times indeed.
This isn't a tool to better understand arabic in some innocent sense, it's a compensation for israelis refusing to learn arabic and having trouble gathering intelligence from linguistic sources. When you're actively involved in the genocide of a people you'll also have problems finding informants among them, it's much easier under a less intense apartheid regime.
And unlike other nations that might build these tools for internal use only, it's basically certain that this tech will be sold to e.g. the US.
See: NSO Group, Cellebrite, Paragon, Oosto/Anyvision, Corsight. They've all had or have contracts with US domestic agencies that needed surveillance tech.
Chinese (esp. youth) are locked up, there are a lot of stories about migration being denied on a large scale for Han Chinese, including within China itself (not allowed to move cities, like you saw long ago in the Soviet Union)
And yet we hear nothing. Meanwhile there's "social credit score", mass unemployment in the capital city, and expert LLM teams (that have put the best US and EU teams to shame 2 or 3 times now) ...
I think we'll suddenly find out China far ahead of absolutely anyone else in this field.
So not some journalistic investigative scoop. Its PR !
> building the LLM “focused only on the dialects that hate us”.
Well duh..
> “We had no clue how to train a foundation model,
Immediately followed by typical MIC over spending instead of fine-tuning
> training data eventually consisted of approximately 100bn words
So closer to gpt3 than any recent sota models
> existing .. Arabic models using standard written Arabic.. rather than spoken Arabic
I think this is the only ah-ha item in the paper.