"Every country needs to own the production of their own intelligence"
Countries (and by extension, corporations, organizations, etc.) must either create their own LLMs or rely on those created by other countries.
If you rely on those created elsewhere, how can be sure that the LLM you are using is based on the correct data and configured the way you want it to?
It may seem like something out of a spy novel, but why couldn't LLMs be used by bad actors to spread misinformation, propaganda, or whatever agenda the LLM creator wants?
I am questioning who is saying it, and their motivation for saying it.
Training a model from scratch is an insanely expensive proposition. Something that anyone attempting to do will likely need to purchase and/or lease quite a bit of, you guessed it, Nvidia hardware to accomplish.
That claim is wrong. Training even a very basic model like TinyLlama takes 90 days on 16 x A100 GPUs [0]. A p4 box containing 8 X A100 GPUs costs $32/hr [1].
So 90 days * 2 p4 boxes * 24 hours * $32/hr = $138,240
Larger models run into the millions of dollars.
Though I suppose it all depends on your personal definition of expensive.
This is exactly an existing concern: flood the internet with LLM-generated disinformation, future LLMs get trained on it, those LLMs flood the internet with disinformation, the cycle repeats. This is why it’s a problem that LLMs can’t distinguish fact from fiction
> Tylenol is a brand name for acetaminophen, which is a medication used to treat fever and pain. It can be made by combining paracetamol or acetaminophen with other ingredients like aspirin, caffeine, and diphenhydramine (an antihistamine). The manufacturing process involves mixing the active ingredient with other inactive ingredients such as dyes, flavors, and preservatives. The mixture is then compressed into tablets, capsules, or liquid form.
This is useless and incorrect information. "Acetaminophen can be produced by combining acetaminophen with other ingredients"?
Though I'm honestly more entertained by the "spicy mayo" instructions calling for you to mix a whole tablespoon of it in a bowl or jar.
If you combine acetaminophen with aspirin and caffeine you get Excedrin, so it's telling you something that exists, but that's not what Tylenol is. It's like it knows there's other fillers in the pill, but it just hallucinated "what ingredients get combined with this" as if those must be the fillers.
If you like uncensored, you may like Gab.ai. I've tested it, and while not as 'good' as ChatGPT, it's far, far less censored. I also think the 'character' idea is really powerful if it's implemented properly, but I don't think these are necessarily. I believe they're just trained to speak a certain way rather than trained on a whole different corpus of data. But if they or anyone else could do it that way, it'd be really smart and likely much cheaper per query.
Fair warning, the owner is not exactly popular and I'll leave it at that, so not making any endorsements of anything but the tool.
"It codifies your culture, your society’s intelligence, your common sense, your history – you own your own data,” - while I agree that countries should not depend on foreign megacorps in such an increasingly important field, this formulation also implies that the objectivity and clarity we were hoping advanced AI will possess are not priorities. Like in the case of mass-media and internet, many non-western countries will not want their people to primarily interact with western-value (whatever is the definition of that) LLMs.
Also the UAE minister of AI being called Omar Al Olama is funny.
Eef it’s a scary premise that each country has an AI mouthpiece trained on exactly what that country wants its citizens to be told. Under most regimes this is a bad idea
I don't think every country will build it's own AI mouthpiece for propaganda, at least the very democratic EU ones.
But I do think there will be a huge market for off-shore mouthpieces for hire for targeted political propaganda, a-la Cambridge Analitica, that will be in a legal gray area, outside the regulatory jurisdiction of the EU but sill be available for hire under the table and deployed when needed by intelligence agencies, while authorities look the other way and pretend they have no clue while deleting their emails a-la Ursula v.d. Leyen. Basically AI versions of the NSO group, and I bet my hat shady companies in US, Israel, China are already working hard on that.
That's the scary future I'm terrified of. I think mass disinformation and manipulation at scale is more dangerous than everything else.
On the one hand, I don't disagree. On the other hand I don't think he's particularly motivated by high-minded concerns of data sov as much as he is by making more money.
Every country needed their own microsoft or linux. They didn’t invest in that. What makes him think they will invest in AI which may even end up just being hype. The fact that chatgpt can code was kind of unexpected. There still isn’t any reason to know weather it can code better economically.
The US government also didn't invest in Nvidia (the early years anyway). Nvidia became dominant because of the private market, they struggled, fought off the likes of SGI, 3dfx and ATI/AMD to become the best at making GPUs for games and 3d applications for most of their lifespan.
You can bootstrap or replace the private market in the beginning with government orders or funding for stuff like airplanes, ship, logistics, energy, defense, machining, but not for consumer tech. Other than US, and China a distant second place, no other country has the huge and wealthy avant-garde consumer market plus the VC sector the US enjoys in order to propel such companies out of scape velocity to global dominance.
Even some European start-ups ignore their home markets and launch first in the US because the best VC funding is there and at launch you already hit 330 wealthy consumers curious about your product who all speak the same language. You can't forcibly replicate that anywhere else today no matter how hard you try.
> You can bootstrap or replace the private market in the beginning with government orders or funding for stuff like airplanes, ship, logistics, energy, defense, machining, but not for consumer tech.
Where do you think consumer tech comes from, except from initial government funding (often military-related). Mazzucato has an entire chapter in her book on the iPhone and all the tech that was originally government-funded:
What became CUDA (originally "Brook") was developed with government grants:
> Ian Buck, while at Stanford in 2000, created an 8K gaming rig using 32 GeForce cards, then obtained a DARPA grant to perform general purpose parallel programming on GPUs.
>Mazzucato has an entire chapter in her book on the iPhone and all the tech that was originally government-funded:
There was nothing special technologically about the first iPhone hardware. It was built using the same commercial of-the-shelf parts that the likes of Samsung, Blackberry Nokia and Ericsson also had access to. Apple didn't use some secret top US tech for it, just the software and UX was ahead of the competition, but the hardware was basic.
> It was built using the same commercial of-the-shelf parts that the likes of Samsung, Blackberry Nokia and Ericsson also had access to.
Touch screens were developed from government-funded research. Tech like Siri was developed from government-funded research (CALO project). ARM was developed for the BBC Micro; BBC is government-funded. GPS is government run. The Internet was a government research network originally.
Apple itself got money from the US government’s Small Business Investment Company:
> Sure but they weren't an Apple classified exclusive tech, but was commercially accessible to all phone makers.
The argument is not about exclusivity (to Apple or anyone else), but rather origins: the Internet is non-exclusive, but (D)ARPA funded its creation. Anyone can receive GPS signals, but it's run by the US government.
Silicon Valley itself is a product of Cold War government spending:
How long before bots are involved in the majority of our online interactions?
With each bot trying to persuade you to buy a product or buy into some propaganda. Sure its not bad now, but in a decade when the effect is more obvious because each country/corporation has their own Nvidia spam machine competing for your attention, will people give up on online interaction outside those we don't know IRL?
that I can talk with you and get your opinion on this topic which is valuable :)
Otherwise it may be some bot trying to aggressively persuade me to buy some new bot-detection product (for instance)
Some kiosks in Hard Reset would say "special discount for smart people" when you approach them. Always made me smile. The high tech equivalent of an old gipsy woman.
no, we'll just move to cryptographically signed everything like we moved from http to https and just assume everything else is insecure or garbage. We thankfully have tech to establish identity, it's just not ubiquitous yet. We're just in a sort of uncanny valley right now where people take more things at face value than they should and consumer products haven't all caught up yet.
Yeah the technology for verifying identity has existed for a long time, but who is going to be the root authority for this? will they be federal? private? I feel not many people will want to play ball on this, especially for the major forums online.
I'd say more importantly that we need to decentralize away from a single hardware provider for 98% of AI. Especially given that it is so hard to even get access to top end NVIDIA products.
This is why I'm working on building a bare metal super computer offering based on AMD MI300x GPUs, that is accessible to everyone. We need choices.
This is very silly in part because it would be vastly more effective to, instead of dropping billions of dollars on buying overpriced obsolete hardware from Nvidia to try to wastefully train a duplicative model which will be beaten by open-source models before you finish training it and where your hardware will be near-worthless a few years after that, spend that money figuring out how to release online all of your country's core data. (A billion bucks pays for an awful lot of scanning, or sending people out with used smartphones to record anything and everything oral, etc.)
Let everyone else scrape it and spend the money to turn it into a model you get for free (and the model after that, and the model after that, and...). Nobody's going to turn down your data!
Just Tom Sawyer that AI fence-painting, and laugh all the way to the bank.
Get ready for a BIG impact across the world! The rise of Sovereign AI is creating a huge opportunity for next-generation data centers and platforms like NVIDIA and AmorphousDB. Don't miss out on this exciting development! #nvidia #AmorphousDB
71 comments
[ 3.7 ms ] story [ 142 ms ] threadCountries (and by extension, corporations, organizations, etc.) must either create their own LLMs or rely on those created by other countries.
If you rely on those created elsewhere, how can be sure that the LLM you are using is based on the correct data and configured the way you want it to?
It may seem like something out of a spy novel, but why couldn't LLMs be used by bad actors to spread misinformation, propaganda, or whatever agenda the LLM creator wants?
I am questioning who is saying it, and their motivation for saying it.
Training a model from scratch is an insanely expensive proposition. Something that anyone attempting to do will likely need to purchase and/or lease quite a bit of, you guessed it, Nvidia hardware to accomplish.
So 90 days * 2 p4 boxes * 24 hours * $32/hr = $138,240
Larger models run into the millions of dollars.
Though I suppose it all depends on your personal definition of expensive.
[0] https://github.com/jzhang38/TinyLlama [1] https://aws.amazon.com/ec2/instance-types/p4/
On the other hand: if both
- the AI becomes good at identifying misinformation
- the AI is opposed to spreading misinformation
it will become a serious threat to those in power. So this won't happen.
https://ollama.com/blog/run-llama2-uncensored-locally
This is useless and incorrect information. "Acetaminophen can be produced by combining acetaminophen with other ingredients"?
Though I'm honestly more entertained by the "spicy mayo" instructions calling for you to mix a whole tablespoon of it in a bowl or jar.
Fair warning, the owner is not exactly popular and I'll leave it at that, so not making any endorsements of anything but the tool.
For larger, wealthier countries he's likely thinking 10+ figures (billions).
Just sayin'.
Also the UAE minister of AI being called Omar Al Olama is funny.
What is wrong with this answer? In this random country the woman would indeed likely get into trouble if she wore shorts outside.
But I do think there will be a huge market for off-shore mouthpieces for hire for targeted political propaganda, a-la Cambridge Analitica, that will be in a legal gray area, outside the regulatory jurisdiction of the EU but sill be available for hire under the table and deployed when needed by intelligence agencies, while authorities look the other way and pretend they have no clue while deleting their emails a-la Ursula v.d. Leyen. Basically AI versions of the NSO group, and I bet my hat shady companies in US, Israel, China are already working hard on that.
That's the scary future I'm terrified of. I think mass disinformation and manipulation at scale is more dangerous than everything else.
It might not be as bad as you think to have more attempts at manipulation.
The US government also didn't invest in Nvidia (the early years anyway). Nvidia became dominant because of the private market, they struggled, fought off the likes of SGI, 3dfx and ATI/AMD to become the best at making GPUs for games and 3d applications for most of their lifespan.
You can bootstrap or replace the private market in the beginning with government orders or funding for stuff like airplanes, ship, logistics, energy, defense, machining, but not for consumer tech. Other than US, and China a distant second place, no other country has the huge and wealthy avant-garde consumer market plus the VC sector the US enjoys in order to propel such companies out of scape velocity to global dominance.
Even some European start-ups ignore their home markets and launch first in the US because the best VC funding is there and at launch you already hit 330 wealthy consumers curious about your product who all speak the same language. You can't forcibly replicate that anywhere else today no matter how hard you try.
Where do you think consumer tech comes from, except from initial government funding (often military-related). Mazzucato has an entire chapter in her book on the iPhone and all the tech that was originally government-funded:
* https://en.wikipedia.org/wiki/The_Entrepreneurial_State
* https://marianamazzucato.com/books/the-entrepreneurial-state...
Computer graphics is no different, with universities doing all sort of early work:
* https://en.wikipedia.org/wiki/Utah_teapot
* https://en.wikipedia.org/wiki/History_of_computer_animation
* https://en.wikipedia.org/wiki/ACM_SIGGRAPH
* https://en.wikipedia.org/wiki/History_of_computer_animation
What became CUDA (originally "Brook") was developed with government grants:
> Ian Buck, while at Stanford in 2000, created an 8K gaming rig using 32 GeForce cards, then obtained a DARPA grant to perform general purpose parallel programming on GPUs.
* https://en.wikipedia.org/wiki/CUDA#Background
* http://graphics.stanford.edu/~ianbuck/
From November 2008, "The history of CUDA":
* https://www.youtube.com/watch?v=Cmh1EHXjJsk
There was nothing special technologically about the first iPhone hardware. It was built using the same commercial of-the-shelf parts that the likes of Samsung, Blackberry Nokia and Ericsson also had access to. Apple didn't use some secret top US tech for it, just the software and UX was ahead of the competition, but the hardware was basic.
Touch screens were developed from government-funded research. Tech like Siri was developed from government-funded research (CALO project). ARM was developed for the BBC Micro; BBC is government-funded. GPS is government run. The Internet was a government research network originally.
Apple itself got money from the US government’s Small Business Investment Company:
* https://www.forbes.com/sites/deborahsweeney/2018/09/04/not-j...
Sure but they weren't an Apple classified exclusive tech, but was commercially accessible to all phone makers.
The argument is not about exclusivity (to Apple or anyone else), but rather origins: the Internet is non-exclusive, but (D)ARPA funded its creation. Anyone can receive GPS signals, but it's run by the US government.
Silicon Valley itself is a product of Cold War government spending:
* https://steveblank.com/secret-history/
I would say that we need alternatives to a single hardware provider.
With each bot trying to persuade you to buy a product or buy into some propaganda. Sure its not bad now, but in a decade when the effect is more obvious because each country/corporation has their own Nvidia spam machine competing for your attention, will people give up on online interaction outside those we don't know IRL?
daddy needs a new leather jacket
my god, Nvidia is now worth more than Google and Amazon
This is why I'm working on building a bare metal super computer offering based on AMD MI300x GPUs, that is accessible to everyone. We need choices.
Let everyone else scrape it and spend the money to turn it into a model you get for free (and the model after that, and the model after that, and...). Nobody's going to turn down your data!
Just Tom Sawyer that AI fence-painting, and laugh all the way to the bank.