You are proving my point: if any unreliable sub-system were put in control of critical, apically dangerous systems such as the military, it would mean that somebody with control over (say) the military could make such decision.
Which would mean that the system was compromised before the secondary event.
I guess the issue here is the idea of empowering an artificial moron which could turn them back off again.
Incidentally, this is already happening owing to half backed natural intelligences, which made controllers that extend their reach out of the explicit intention of the user (e.g. "if user turns lights on, then turn on sounds").
> For industrial control, developing high-performance controllers with few samples and low technical debt is appealing.
They then proceed to develop a nine-stage pipeline of three interacting processes relying on a separate database in order to perform prompt generation.
As an aside while I remember its genesis I now have no idea what technical debt means in popular use. These authors use it to mean investment of effort.
> I now have no idea what technical debt means in popular use.
They define it as "the effort to model the problem, developing algorithms, collecting samples, and inquiring expert knowledge" (end of page 4 https://arxiv.org/pdf/2308.03028.pdf)
I, and I believe everyone I've worked with, believes it to be the cost of future work required to fix/change a decision your making now. Literally the opposite of what they say: the upfront cost of developing a system.
Deterministic controls have had a lot of staying power for reasons that are pretty obvious. I don’t see what benefit we gain by leaving human comfort up to a LLM when a thermostat works just fine.
Just an aside, but I think venture capital will fall out of love with AI when LLMs start making decisions for customers and we are forced to have human support staff clean up the messes it leaves behind.
I’m not sure if your are joking or not. I would not trust a system that made a mistake in the first place to fix the problem I am having with it or it’s decision. The “smarter” or more vertically integrated this system is, the harder it will be to convince it to do what I want.
You clearly are not thinking like a level 4 manager who has a vague grasp of how the system works beyond "This costs $X per year, how can we make it $X-1 next year?"
I can't wait for the secondary problems cause by the limitations of "AI" understanding.
We have had some consulting company of people managing these systems in some buildings, and they would basically turn off climate controls in unoccupied spaces.
Then they started having moisture issues in the important spaces because the data spaces are cold, next to an unconditioned empty space, which got hot, and causes condensation on the floors and ceilings.
Why on gods green earth would I need to use GPT for my HVAC.
Already I can clock a button and set it to what temperature I want and it maintains that.
If I were someone whom got excited at the prospect of my AC being part of a botnet I could even connect it to the wifi and control it from an app on my phone or through a home assistant.
Who wants this? Why is this a thing? Good God you spent so much time wondering if you could you didn't stop to think if you should.
Edit: God this is even worse than I thought "For industrial control, developing high-performance controllers with few samples and low technical debt is appealing." I think that using GPT to run our industrial control systems is like asking dolphins to be in charge of our nuclear reactors. Whoever proposed this should never be allowed to make decisions regarding technology ever.
The same reason even people who should know better implement all sorts of insane things in Excel. It's the tool available to them that they can find tutorials on using. With GPT it even seems like it's doing work for you.
Obviously anyone who knows anything about control systems immediately has five or six reasons why this is terrible, but in the meantime My First GPT is running lawn mowers from the sketchy discount shop because none of those people even knew how to ask.
Excel is at least capable of basic floating-point arithmetic. From the paper:
As mentioned in Section 3, we round all real numbers to their nearest integer values, based on the assumption that GPT-4 might have difficulty handling real numbers directly. In this section, we perform an ablation study to validate this assumption. The results are presented in Table 5, which demonstrates that using rounded real numbers can indeed enhance the performance of GPT-4, thereby confirming our hypothesis.
It should be reminded that for decades, interested in the study of non-strictly-quantitative control of systems for engineering advances, the Japanese have pioneered Fuzzy Logic.
The comparison seems strongly telling. One strong point could be that Fuzzy Logic remains a logic.
Am I paranoid or is this a clear evidence that we cannot control nor contain AI as people will try to misuse it as much as possible for convenience and profit?
You are probably naïve not having collected enough «evidence» that there is a global cultural problem that will cause «misuse», so what is getting harder to «control []or contain» are people.
The fact that you are using a term '«AI»' there where we are lamenting, here and elsewhere, that no intelligence ("AGI"; or automation of, "AI") seems to be involved, and/or the lack thereof is dramatic, should contribute in raising the issue.
The note about «convenience and profit» is important and differently noted in the page with the related idea of "not intending to perform crucial steps towards Quality" that svnt noted and foobiekr called «crossing the Rubicon».
This is really funny - it's bordering on truly absurd, almost incomprehensible madness to consider doing this seriously. I can't think of a single property you'd desire in a control system (state observability, auditability, guarantees on out-of-band input behaviour, stability under shocks, etc, etc) that would be present in an LLM control model.
I don't want to be disrespectful to the authors, and it's (vaguely) interesting to see how far they've been able to go with this, but this idea is still an abomination.
41 comments
[ 4.0 ms ] story [ 102 ms ] threadWhich would mean that the system was compromised before the secondary event.
smarthome: you're late.
me: smarthome, sorry, please turn on the lights.
smarthome: why don't you sit in the dark for a while and think about being late.
Is there anything else I could help you with? O.O
I guess the issue here is the idea of empowering an artificial moron which could turn them back off again.
Incidentally, this is already happening owing to half backed natural intelligences, which made controllers that extend their reach out of the explicit intention of the user (e.g. "if user turns lights on, then turn on sounds").
They then proceed to develop a nine-stage pipeline of three interacting processes relying on a separate database in order to perform prompt generation.
As an aside while I remember its genesis I now have no idea what technical debt means in popular use. These authors use it to mean investment of effort.
Ergophobia, "rejection of effort", proposed as a principle. (Crossed with "quality blindness"... "Poiotetagnosia"?)
'ergonomy' means "how to work".
And we counter the idea that "delegating to big [unlean] dumb servants that will somehow appear to do the trick as a cheap solution" fits.
They define it as "the effort to model the problem, developing algorithms, collecting samples, and inquiring expert knowledge" (end of page 4 https://arxiv.org/pdf/2308.03028.pdf)
I, and I believe everyone I've worked with, believes it to be the cost of future work required to fix/change a decision your making now. Literally the opposite of what they say: the upfront cost of developing a system.
Just an aside, but I think venture capital will fall out of love with AI when LLMs start making decisions for customers and we are forced to have human support staff clean up the messes it leaves behind.
They will start worrying when revenue is continuing to decline even after burning through the third CEO.
We have had some consulting company of people managing these systems in some buildings, and they would basically turn off climate controls in unoccupied spaces.
Then they started having moisture issues in the important spaces because the data spaces are cold, next to an unconditioned empty space, which got hot, and causes condensation on the floors and ceilings.
Already I can clock a button and set it to what temperature I want and it maintains that.
If I were someone whom got excited at the prospect of my AC being part of a botnet I could even connect it to the wifi and control it from an app on my phone or through a home assistant.
Who wants this? Why is this a thing? Good God you spent so much time wondering if you could you didn't stop to think if you should.
Edit: God this is even worse than I thought "For industrial control, developing high-performance controllers with few samples and low technical debt is appealing." I think that using GPT to run our industrial control systems is like asking dolphins to be in charge of our nuclear reactors. Whoever proposed this should never be allowed to make decisions regarding technology ever.
But GPT or LLMs... No. They are not for that.
###
AI: automation of intelligence.
AGI: implementation of intelligence.
Automation of intelligence (AI): taking a task that required an intelligent entity for performance, we devise algorithms that can provide.
Implementation of intelligence (AGI): we take the process itself of intelligence and replicate it algorithmically.
###
An AI solves a problem reliably. An AGI has judgement. The context is not about any of this.
Obviously anyone who knows anything about control systems immediately has five or six reasons why this is terrible, but in the meantime My First GPT is running lawn mowers from the sketchy discount shop because none of those people even knew how to ask.
As mentioned in Section 3, we round all real numbers to their nearest integer values, based on the assumption that GPT-4 might have difficulty handling real numbers directly. In this section, we perform an ablation study to validate this assumption. The results are presented in Table 5, which demonstrates that using rounded real numbers can indeed enhance the performance of GPT-4, thereby confirming our hypothesis.
The comparison seems strongly telling. One strong point could be that Fuzzy Logic remains a logic.
You are probably naïve not having collected enough «evidence» that there is a global cultural problem that will cause «misuse», so what is getting harder to «control []or contain» are people.
The fact that you are using a term '«AI»' there where we are lamenting, here and elsewhere, that no intelligence ("AGI"; or automation of, "AI") seems to be involved, and/or the lack thereof is dramatic, should contribute in raising the issue.
The note about «convenience and profit» is important and differently noted in the page with the related idea of "not intending to perform crucial steps towards Quality" that svnt noted and foobiekr called «crossing the Rubicon».
I don't want to be disrespectful to the authors, and it's (vaguely) interesting to see how far they've been able to go with this, but this idea is still an abomination.