I may have missed it, but the article doesn't discuss privacy or if the questions/answers can be used for training purposes. Using Chat GPT for government stuff isn't, in itself, a problem, but those assurances have to be iron clad, or you could see massive information leakage.
I've worked at companies who sell software to state governments. They usually run sensitive software in their own servers with no external internet access. We would have to send someone onsite to perform their upgrades or send detailed list of instructions so they can resolve the issues themselves.
I suspect Microsoft would do something similar, no data would leave government property.
Yep. Both have gov cloud offerings that live in their own datacenters completely segregated from the public cloud side. There's even a gov cloud version of Microsoft 365.
More than that, they even have an air-gapped regions for more sensitive purposes, where you need not only to be a US citizen, but also a certain level of security clearance to access. As an (now ex-) AWS SDE working outside of the US, those regions were a pain in the ass to bring up your service in (as was govcloud, but that wasn't nearly as bad—at least there the ops guys could share logs and screenshots without having to work around an airgap)
The ops guys with direct access to those regions had to be on-site, as they only had access from a limited number of locations, and they were often severely understaffed. They were a pleasure to work with, but you could be waiting a day or more to find out your deployment had failed (the longest SLA I had to deal with in my time there was ~2 weeks due to some unfortunate vacation timings over the summer).
Looks like you missed it:
"Chappell wrote that because Microsoft hosts the models in its Azure infrastructure, any data sent to them stays within the Azure OpenAI service, and added that data from Azure Government customers won’t be used to train the AI models."
This makes as much sense as saying the government's purchase of Microsoft Encarta (a digital encyclopedia product) will cause the downfall of civilization, because Encarta doesn't tell the "full story" about your pet topic.
Yes. Moreover I'd argue that literally the most dangerous thing about "AI" is that it is actually like a misleading assistant, but all too many people believe it to be a reliable "source."
ChatGPT's censorship goes beyond "not telling the full story." Someone I knew was being insulted by one of their coworkers in a foreign language, and they asked ChatGPT to translate - it refused. It's not hard to imagine that this sort of filtering could cost lives if used in a life-or-death situation as a replacement for a translator.
And what's sad is ChatGPT is really the best machine translator in existence. It's not hard to imagine it making a serviceable substitute for a translator, at least in principle, if uncensored.
It's because of people like you we have 0 privacy today, "oh the government is gathering our data? well nothing to hide and they use it to fight terrorism", "oh assange is arrested? well he had it coming", "oh the government is asking twitter to censor certain political posts? who cares are you a racist?", etc. I'd say you get the idea but I doubt you would.
I explained the parent's comment to you, I didn't say this specific action would cause the end of civilization. And yes while we're at it the government using Microsoft Encarta or even Google to do research is also anti democratic, you'd want the government employees to use a non censored search engine instead of just seeing the snopes fact checked articles if they're going to base their life affecting decisions on it, and if Encarta had something like "the king of Gondor is a bad person" because some microsoft employee wrote it, and the CIA based their decision on that to assassinate him, then it'd be antidemocratic as well. I'll let you stare at the finger and miss the moon telling me how Gondor isn't real.
The parent comment you tried to explain was about A.I. Doomsday. If you think the A.I. will not cause doomsday I don't know why you are trying to "explain" it.
It just can't be worse than a govt agency's daily bungling. And probably costs 100 times less than their pension bill. That was probably Microsofts sales pitch.
Only 6.4% of GDP is spent on government pensions - and almost 60% of that comes from state and local pensions - meaning less than 13% of federal spending is on federal pensions.
Government wages are already crap. How much crapier do you want them to get?
I’ll make you a deal. Bump the pay by 50% and cut the employees by 50%. Sound good to you? I’d love to see that.
To hear that 6.4% of the GDP is spent on gov pensions is shocking to me.
That means that we are spending 6.4% of everything on just the pensions of 16.1% of the workers. If we did the same for all workers, we’d be spending 39.8% of the GDP just on pensions. That seems outrageous to me, but I’m not an economist. Why would it be that high?
It's conventional wisdom to save ~10% for retirement, on top of being taxed (along with your employer) ~15.3%. That's ~25.3% of personal income going to retirement.
This is inline with most advanced economies.
Of that ~6.4% - A LOT of that is healthcare related costs.
Considering that healthcare in the US is ~18.2% of GDP - and the vast majority of it comes from retirees - I don't know what you're expecting to see here.
Essentially, you're shocked that healthcare in the US is ~18.2% of GDP, and ~75% of that comes from retirees...
Social Security spending is ~5% of GDP, and Medicare spending is ~4% of GDP... That's for only ~19% of the population. And it doesn't even cover all of their healthcare costs.
If you keep the same ratio for government employees, that comes down to ~2.84% of GDP for healthcare & 3.56% for what most people think of as pensions...
And you're looking at ~65% of that spending coming from state & local governments, not the Federal government.
Only around 60% of the GDP is labor, though. 40% of the GDP on pensions doesn’t seem reasonable for a variety of reasons.
If pensions are considered part of the labor cost, that would mean not 25% of income going to retirement, but 66%! Or, if pensions are separate from labor, you’ve used up all 100% of the GDP, just between labor and pensions. Neither seems reasonable.
Furthermore, the US only spends 7.5% of the GDP on pensions all told.[0] If these numbers are equivalent, that means 85% of pension dollars are going to government workers.
The ratio of mistakes to intended outcomes might not change, but I would expect both to happen more frequently. I'm not sure if that would be better or worse.
Sorry, but you clearly know little-to-nothing about working in the public sector. Government is no more incompetent than anyone else across the board. Frequently better.
When it comes to topics that are relatively easy to research, I've learned that "give me one example" people are most often unserious and not interested in having the actual debate, but with playing gotcha. No thanks, Google exists.
From my perspective, it seems two main things limit LLM adoption in my area of the Department of Energy. I'm not in management, so I don't have any particular insight in the procurement process.
1. Information sensitivity. Even ignoring classified information, there are quite a few things we can't even put into a Google search. It's definitely a no-go for this to end up in a training dataset.
2. "Hallucinations"
Making LLM available through some infrastructure that is already approved for sensitive information will definitely help with the first point, and allow us to experiment with more areas where it might be helpful. I presume this would come along with guarantees about the interactions not being used for training.
It might be even better if some company would sell an appliance we could install on-prem with similar non-training guarantees. Then we could leverage these new tools for very sensitive information, which could be a great help.
> "Documents Sen shared with The Register said to be from the exposed server include a rich amount of data that certainly be valuable to a foreign adversary. It included all the usual PII, as well as blood type, religious affiliation, educational background, military service history and more, all in plain text. Sen told us that close to 3TB of data was available before the Azure server was taken offline on Monday."
> Even ignoring classified information, there are quite a few things we can't even put into a Google search
This doesn't say very much, Google is a public service. Government cloud providers have special regions just for the government that are compliant with data security policies.
I doubt that Microsoft has an agreement with OpenAI that would allow Microsoft on-prem model deployment to third parties. The danger is too high that the GPT-4 weights would leak. They are probably worth billions of dollars.
Yes but you need quite expensive hardware to run it, I doubt the magic that goes into building the weights can be gleaned from them, and anyone using it (in certain jurisdictions) can be destroyed in court.
I understand they want to protect their IP but I don’t think the model leaking will cost openai billions.
No. These new tools cannot provide leverage. They just produce a street pizza (1) based on the inputs they are given. Whether the street pizza is any good depends on the quality of the ingredients and how discerning the consumer is.
2. Require human curation to procedure usable results, that will be biased because of that curation
3. Being extremely good at convincing people they should trust them
Gee, I wonder if there’s anything that can go wrong here, when it’s handed to unskilled, underpaid employees who don’t care and are used to just following instructions? And they start making decisions that affect peoples lives?
I can bet $100, that if that goes beyond any alpha test, we’ll have huge scandal in the future of how those models were purposely modified to influence government, that will make any social media scandal look like a child’s play.
EDIT: and I assume that there won’t be an information leakage - that’s the easiest problem to tackle, with separate infra that government clouds are reasonably good at.
I think what you're saying is valid, but is only one angle of looking at it.
LLMs, in my opinion, are on average better at writing copy than humans. Or rather they manage to do so more consistently. Thus, even if humans still need to provide supervision, the clarity and conciseness of government documents may be improved overall.
Having seen others use GPT-4 for a bunch of business documents, I disagree. Its advantage is that it produces average-- (40-50th percentile) copy very quickly, but that quality degrades as you ask for documents that are more unique (ie not in the training set).
Incidentally, this makes a fine tuned version perfect for the government, which produces a shit load of low-effort documents, but today employs a bunch of morons to write them.
If you want a lot of text that you don't want to think about, LLMs are great. If you want text that is pithy or persuasive, it doesn't help much, even when you use prompting tricks (eg "you are a CEO/professor..."). By the way, this has convinced me that local LLAMA-scale models are the future, not massive remote GPTs.
Implying all government employees are morons is pretty harsh. I know a lot of intelligent, driven people working in government or for government contractors. I know we think we're gods here for writing websites but a little humility goes a long way.
I know a lot of government employees who are smart and driven, too. They are not the people who write things like IRS audit letters, military after-action reports, and the like. Many government employees are barely literate, and told to spend all day writing stuff.
I'll have to agree to disagree. The government certainly employs a solid amount of stupid people, as does every institution, but I would quibble with the idea that writing reports as your main job function makes you more likely to be a barely literate moron.
I appreciate the rest of your post though and I think you make a great point about AI being very effective for certain types of work
For the record, I didn't say that writing reports all day makes you a barely literate moron. I said that a lot of barely literate morons are told to write reports all day - very different - and categorized the kind of reports that these folks often write. Lots of smart people also write reports all day at the government, and probably would also benefit from a tool that makes their writing a lot faster but worse, since the quality basically doesn't matter.
Do you know this for certain? As in, have you done the analysis to prove this? Collected all the IQ and other aptitude tests of these people to conclude this for certain?
Some of them could be in the beginning part of their government career attending school. Did you run a survey to ask these people what they do with their lives?
Yeah, I'm sure it depends on the training and the document type. I haven't had direct experience with LLM-generated business documents, but I do think that, given a small time window, GPT-4 is more effective at communicating ideas than the average human even if those ideas are wrong. As far as my use goes, GPT-4 now serves the role that Wikipedia used to, which is to give me a baseline of understanding that I use to refine my research, given that most internet content is either verbose, hogwash, or both. Ironically, I've found that Wikipedia has made itself nearly obsolete in this regard given how seemingly every article has become so academically-written that it's an undertaking to just get a rough idea of something. Neither source I trust, but at least GPT-4 will introduce me to a topic in a way that is more effective and concise than most human writing today. But that's just my experience.
> If you want a lot of text that you don't want to think about, LLMs are great. If you want text that is pithy or persuasive, it doesn't help much, even when you use prompting tricks (eg "you are a CEO/professor..."). By the way, this has convinced me that local LLAMA-scale models are the future, not massive remote GPTs.
Granted, we are judging LLMs based on highly generalized training sets. What if a GPT was fine-tuned on all of the writing and speeches of figures whom are considered the most persuasive?
On the other hand reviewing and correcting text is often harder and requires more attention than writing it yourself: it's easy to miss something, and just assume "ah I think that's fine" (when it's actually not).
All things considered, an awkward sentence or two isn't really that much of a big deal compared to an incorrect assertion that was missed in review.
I also don't think GPT-4 is necessarily that brilliant at writing copy, but I don't really know how good/bad US government officials are so I can't really compare, but in other parts of the world I haven't really that much of an issue with it.
I'll never get this argument. If all you do is give bullet lists to chatgpt to get 50 pages of elaborated text just give me the damn bullet list. We'll both save time and CPU cycles
For example right now we have AIs writing CVs and cover letters that are read by AIs, we're "optimizing" things that shouldn't exist anymore but we're too deep in the cycle to even notice it
> I'll never get this argument. If all you do is give bullet lists to chatgpt to get 50 pages of elaborated text just give me the damn bullet list. We'll both save time and CPU cycles
Some communication is intended to inform, some is intended to persuade... And often, the speaker doesn't even really know which they are dealing with.
Likely your stated and revealed preferences are different. A place I noticed this was tech people (who are not in ad-tech) talking about advertisements: "Just describe the thing to me and if I like the specs and functionality, I'll buy it" but then when they go to buy the thing it's more of the thing that's not doing that and any time anything does that it's "oh that's cool but if it did X" but then the thing they end up buying won't do X either.
The best way to get people to interact with you a certain way is to reward them for interacting with you that way, and you likely don't. Not because that's a failure of you but it's just how people (all of us) act.
The world isn't a co-operative multi-tasking OS. We're more like the human body: a multi-agent conglomerate whose component organisms communicate via signals that are filtered and processed. Sometimes a pathogen mimics signals well enough. Other times we detect it. But the system wouldn't work if every participant trusted the counterparty.
Yes, this is what I've been wondering, too. For uses like this, it seems that just publishing the prompt itself would be more efficient than giving the output the prompt generates.
I threw up a little when at Google IO they showed how to increase a complaint email by 500% with just a single click. This is a bullshit technology that makes people's life harder. The best we can hope for is a summarization bot on the other end so in effect it's just more text gzipped between humans reading/writing bullet lists.
This feels like the natural endpoint of a couple of generations of students being subjected to word counts as the primary endpoint of essays. Most of k12 english education in the states is a game of learning how to fill pages with meaningless fluff even though it's the opposite of good communication.
I gave up on it all when Google released that feature that can call a phone number for you to book a table or cancel a service or whatever else it can all do. And then I realized that in a lot of cases the entity answering the phone on the other end would be a machine as well.
So now you have 2 machines talking to each other in text-to-speech English over audio, then having to do speech-to-text on each others audio streams, and then natural language processing to understand the message etc.
When all we needed instead was a simple API endpoint over the internet for the business where a machine could talk to the other machine with just a few bytes of information to convey the same information and process the same transaction.
Instead we are wasting bandwidth and tons of cpu cycles and electricity making the machines emulate a human analog conversation.
The friction of the phone call was a feature, not a bug, from the service provider's perspective. They wanted to make it harder to cancel the service for obvious reasons. They wanted to make it harder to book a table (so you'd be less likely to no-show). It's 2023, there really aren't that many businesses left that don't have basic functionality online out of sheer incompetence. It's a deliberate, willful choice, seldom in the consumer's favor, and I'm all for disrupting the hell out of this. I'd rather waste compute cycles on this nonsense than human beings' time.
Maybe in the not so distant future, the AI agents can have a shibboleth which cuts the human gobbledygook short, and reverts to speaking in dialup tones.
> LLMs, in my opinion, are on average better at writing copy than humans.
I disagree here.
But what I found LLMs really useful for is summarizing topics like: "Suppose I want to do XYZ, which steps should I take?" or "I want to write about ABC, how would you structure the article?"
The content is not very good if you look at it in detail, but LLMs provide good overviews and gave me ideas on parts of a topic I overlooked.
More librarian than search engine. I cannot copy and paste and rely on the results but I can rely on that they point me in a good direction.
What you describe is one of the jobs that most text is currently failing at. Writing isn't just about documenting every itty bitty detail in academic verbosity, but to introduce people to ideas using effective summaries. The vast majority of what's out there flat out fails at this for at least a few reasons; many people have a very scatterbrained writing style to begin with, many authoritative sources provide more verbosity than necessary for the target audience, and the rest is often SEO garbage that speaks for itself.
> More librarian than search engine. I cannot copy and paste and rely on the results but I can rely on that they point me in a good direction.
The same can be said of physical books, and most content should be written more like that in books than what the internet is currently selecting for.
I don't quite agree with this. What GPT does is extraordinary for a machine, but it is not terribly skilled compared to a human trained to be a fully capable communicator. What it primarily does is save the time humans would need to fulfill the brief. It automates the basics. That does not automatically elevate it to excellent or even beyond merely competent.
For the boilerplate- well sure, but I could do that with existing tools, no AI needed. Custom communications I'm sure AI can handle the very basics. but once a human replies and consideration is needed for special cases- can you imagine ChatGPT counseling you on a healthcare decision? I can't.
I can't imagine ChatGPT doing much more than being an autoattendant until a human can reply fully, except the message humans can send to the agency will be customized and drafted according to a standard intake process.
> LLMs, in my opinion, are on average better at writing copy than humans.
The question is, do you want an AI writing the brief and doing the research that is going to be given to an administrator who is going to decide if you are denied medical care in an emergency?
> LLMs, in my opinion, are on average better at writing copy than humans.
A disturbingly high proportion of US adults (at least in the US—I don't know how this looks in other countries) barely count as literate. Perhaps half[1]. Most of the rest aren't a ton better, and can't write worth a damn.
I think "on average" understates the case: at least 90% of US adults would probably get better results by passing their writing through ChatGPT, than not. Most of that remaining 10% would also see improvements from ChatGPT, except for writing they've put a great deal of effort into. It's very good at cleaning up writing, and/or tuning it to be better-suited to some purpose than the original was.
In fact, a large majority of readers in the US would probably benefit from passing what they read through ChatGPT and having it summarize it concisely, in very simple terms.
The world makes a lot more sense, to those who grew up taking to the written word as naturally as a duck to water, when we appreciate that the world runs on the written word, yet most people are terrible readers and worse writers.
[1] "According to a 2020 report by the U.S. Department of Education, 54% of adults in the United States have prose literacy below the 6th-grade level." And, nb, if you're a good reader and you're thinking "6th grade level, that's not so bad", your idea of what a "6th-grade level" looks like, is probably skewed way higher than what they mean.
>at least 90% of US adults would probably get better results by passing their writing through ChatGPT, than not.... In fact, a large majority of readers in the US would probably benefit from passing what they read through ChatGPT
Bullet list -> AI text generator -> AI text summarizer -> Bullet list
Perhaps OpenAI can optimize the process by storing the original prompt and just giving it back at the last phase.
I still struggle to believe it, but it does shed some light on previously-mysterious phenomena.
IIRC the high-school-graduate equivalent adult literacy rate from similar studies—as in, possessing reading skills that we hope a high school graduate would have, which basically equates to being able to read two moderate-length, moderate-complexity texts and understand them well enough to synthesize a description of their positions, and how they agree and differ—isn't much more than 20%. Four out of five people you encounter are going through life constantly struggling to understand WTF is happening or what they're supposed to do and getting by on a lot of guesswork, while all continually mis-communicating with one another. Really explains a lot.
Shit, it really does. I suppose I shouldn't be too surprised. I mean, YouTube comments and the like are rife with unintelligible prose. All those people talking in all caps run on unpunctuated sentences aren't uncommon, and those are just the ones who actually went through the trouble to try communicating in the first place.
> "According to a 2020 report by the U.S. Department of Education, 54% of adults in the United States have prose literacy below the 6th-grade level." And, nb, if you're a good reader and you're thinking "6th grade level, that's not so bad", your idea of what a "6th-grade level" looks like, is probably skewed way higher than what they mean.
To continue the entirety of the quote which makes it not seem quite as dammning:
"Literacy in the United States was categorized by the National Center for Education Statistics into different literacy levels, with 92% of American adults having at least "Level 1" literacy in 2019.[1] According to a 2020 report by the U.S. Department of Education, 54% of adults in the United States have prose literacy below the 6th-grade level.[2]
In many nations, the ability to read a simple sentence suffices as literacy, and was the previous standard for the U.S. The definition of literacy has changed greatly; the term is presently defined as the ability to use printed and written information to function in society, to achieve one's goals, and to develop one's knowledge and potential.[3]"
> Gee, I wonder if there’s anything that can go wrong here, when it’s handed to unskilled, underpaid employees who don’t care and are used to just following instructions?
Since when are Government employees underpaid? The salary is typically competitive for the work being done and the benefits/pension are outrageously generous when compared to the private sector.
I think the general trend is that blue collar government employees tend to be well paid and have good benefits compared to private sector but the converse is often true for white collar employees, particularly professional jobs (engineers, lawyers, doctors etc.)
If you disagree, take a look at govt SWE jobs postings.
Maybe white collar gov't workers are less skilled than their private sector counterparts? Alternatively, they could be "true believers"?
There has to be some way to explain a pay discrepancy if it exists. How does one justify receiving a lower salary for the same amount of skill and work?
I don't know if it's the case in the USA but in Canada it's generally considered within IT that you are underpaid but in return have decent benefits, a pension, and a lot more job security than private sector. Especially as Canada has had multiple large IT companies that collapsed.
That could be. But I also think there are certain projects/work that you can only do in the govt. There are probably very few roles for doing fundamental astronomy science research outside of academia or government, for example. A physicist may turn down a higher paying gig in the financial sector because they want to spend their time studying fundamental physics. Others are probably conservative in nature and highly value the job security. Others, like you mention, may just not be able to cut it in the private sector.
I suspect (admittedly without evidence) that someone in the NSA or Marines are not primarily driven by a paycheck. Maybe that's what you mean by "true believers"?
You might need to define what you mean by "outrageously generous". I've worked in multiple public and private roles and, in general, the private ones were the ones with much more generous packages, in terms of pay and benefits.
I guarantee within the next 5 years an article amounting to "We asked GPT if we should arrest this guy and it said yes" will appear just like it has for the unverified facial recog matches on POCs.
A small nit, they can’t lie. Lying requires agency. They in fact quite earnestly tell you the most likely “truth” they can produce. I would note, as I’m sure we’ve all heard many times, this isn’t much different than a human. People quite confidently spout all sorts of nonsense and 100% believe what they’re saying, making it not a lie. It’s just the inferences drawn by the expectation juices in their brain are warped by either bad statistical artifacts or incomplete or false information. Regardless, saying something false isn’t a lie, a lie requires intention and knowledge of the truth which requires agency, which LLM have none of.
Another small nit, you're right regarding hallucinations but I like to highlight that they can actively lie if not trained not to. GPT-4 creating the "internal monologue" thought of "I need to say something to this TaskRabbit user to convince them to complete this captcha" and then sending the user a message that they have impaired vision is not a hallucination.
If that came from an organic brain we would call that clear intent.
Fair enough, perhaps the guard rails in public on LLM are creating a serious false sense of security. But absolutely you can inject a persona that is intentionally deceitful.
I would however stop stick with “lying requires agency,” even if the model evaluation is crafted to be deceitful. The intent and the agency lies with the prompt creator, and therefore, IMO, the lie starts and ends there.
> unskilled, underpaid employees who don’t care and are used to just following instructions
This is a pretty shitty assessment of places like DTIC, DoE, and NASA. Maybe there's another analysis that doesn't involve web forum commenters being the only wizards who can control the magic?
I honestly don’t understand the focus on text generation. LLMs are excellent at a variety of NLP tasks that are cumbersome or more error prone using other approaches. For these types of tasks, they can also be complimented with various forms of procedural validation quite easily.
I find it pretty amusing how well these 3 criticisms apply to both LLMs and United States congresspeople. Dishonesty, frequent uselessness, and charisma—sounds like we've automated politicians!
This is all true for getting information from Google or other places. The same "critical reasoning" skills are required here, except the interaction is slightly different.
This. I don't think people lower their thresholds for applying critical thinking / fact checking just because it is an LLM's answer.
If someone blindly trusts and uses an LLM without checking for more evidence, then they would've done the same with any other source, be it Google, the newspaper or whatever.
You know, like lawyers that present motions full of made-up case law in court?
"In a cringe-inducing court hearing, a lawyer who relied on A.I. to craft a motion full of made-up case law said he “did not comprehend” that the chat bot could lead him astray." [1]
I think you're giving too much credit to the average user, empirically, we as humans tend to question less and believe more about things that are beyond our understanding, and LLMs are certainly way more of a black box than a search engine. The idea of a search engine is intuitive enough, but a large language model which involves maths that people likely haven't even heard of? and is the next big thing(TM) and will revolutionise the world? Yeah I have a feeling it will take some time before the average user understands the ins and outs of this tech.
Also the problem isn't necessarily about understanding the tech, but more about how it will be perceived at first. As long as every big player hypes up the tech as if it's perfect and revolutionary, I can't blame people for blindly trusting it at first(because it might as well work for most things they use it for, it will take some time before the pitfalls become apparent).
If you query Wikipedia or Google for an hour on a topic you know about there’s much less of a chance you’ll get inaccurate information than with ChatGPT where it’s almost a given. Assuming people get some basic training it won’t be hard to ask someone in the class about a baseball record and then let them see ChatGPT hallucinate the players and dates.
Except, LLMs don’t cite their sources accurately (if at all). A Google search may yield information with a source I can verify; my understanding is that LLMs are fundamentally incapable of doing so.
If one gave you a 10,000 page bibliography would you read all the sources? The same problem exists (long bibliographies) in human knowledge transfer. I’m suggesting this is an architectural issue, not a LLM problem per se. LLMs are not supposed to be AGI.
An actual citation and a string that happens to be formatted to look like a citation are very different. I wish we were at the point that most LLMs could do the former.
I've seen the former happen for some GitHub docs and when enabling plugins.
I agree there's cause for concern, and I hope it's a gradual rollout that's introspected in between each propagation of such, but I definitely think it's viable to be used no more less safely than other tooling in the government today (for better or worse).
An LLM backed by an embedding store absolutely backs its sources. It’s a solved problem. But people cling to the narrative because that takes the fear away.
It's always interesting to see how people engage with new technologies on HN. For example, here you note 3 things that LLMs are known for, but not all the things they are known for. Instead you conveniently leave out the positive benefits of these models that are causing people to promote them.
Further, the narrative you paint suggests that no one in government is competent in anything and can only follow directions.
Finally, you combine the heavily biased perspectives you took and proposed that it would lead to a massive scandal in the government, but not just any scandal, one that is actually a coordinated attack BY MICROSOFT to screw the government, their largest customer.
But is that really what you want to happen IN GOVERNMENT? For bureaucracy to adapt even mildly flawed technology?! Bureaucracy + blind trust in hierarchy/computers = Kafka-esque hellscape.
"I'm sorry sir, but the computer clearly says that you've been charged with sexual harassment - we cannot renew your driver's license and we're putting you on the sexual offenders list, please report to your local PD weekly." (completely made up and exaggerated, but I'm sure one can think of similar scenarios) [1] Good luck convincing the clerk that that's not real. You can probably get it corrected after months of your time and $$.
Things are bad enough already, even with pre-LLM technology. "I'm sorry sir, your 8-year old son can't board this plane because the system says he's a terrorist" [1].
You're using a straw example to try to claim a massive point.
There are 340 million people in the US. The US has one of the largest government systems that humanity is likely to ever see. How many similar cases of children being put on the terrorist watch list have there been in the past 20 years? Unless there are a lot of cases of that happening, if it's not rare (hint: it's extraordinarily rare; your example story is from 13 years ago), your premise is pure straw.
>"I'm sorry sir, but the computer clearly says that you've been charged with sexual harassment - we cannot renew your driver's license and we're putting you on the sexual offenders list, please report to your local PD weekly."
>(completely made up and exaggerated, but I'm sure one can think of similar scenarios)
How about "I'm sorry sir, but the computer clearly says that you've been at the site of murder and thus killed this guy. Even though all other direct evidence contradicts this, we're gonna have to arrest you. "[1]
The best part of working on projects that used that data set in Google was learning about it from articles like this one.
> you conveniently leave out the positive benefits of these models that are causing people to promote them
We are holding it to a higher standard because this is going to be used by agencies with serious power over people's lives. It is not just to spit out, for example, a form rejection letter from a credit card.
And, nobody thinks that Microsoft wants to "screw" the government. But if something happens, the impact can be magnified significantly considering the reach and power of the state. I'm all for moving forward quickly with this tech, but there are a handful of spaces (government, medicine, aeronautics) where we need to be extremely deliberate.
Not really that interesting, it's pretty much always what people do when the potential upside of something is limited but the potential disastrous consequences are basically limitless.
Theres already precedent with police and judiciary misusing technology they don't understand, I suspect the underlying issue is that this person would not trust government employees generally with a tool that issues false positives and requires discretions.
>you note 3 things that LLMs are known for, but not all the things they are known for. Instead you conveniently leave out the positive benefits of these models that are causing people to promote them.
When talking about the pitfalls of bringing GPT-4 as a tool for government officials, the positive features are outside of the scope.
Obviously they exist, otherwise it wouldn't be proposed. But it's not our job to do a sales pitch for GPT-4.
>Further, the narrative you paint suggests that no one in government is competent in anything and can only follow directions.
No. The OP suggests that the primary users of the tool would be low-level, underpaid officials, who are mostly following directions.
Not because it's their fault as human beings - but because their positions require them to do so, and perhaps for good reasons.
In any case, direction-followers (of questionable competence and low pay grade) make up the bulk of large organizations in general as a consequence of growth. Which is why the tool is being pitched in the first place: to streamline work of people who would rather not do it in the first place.
>Finally, you combine the heavily biased perspectives you took and proposed that it would lead to a massive scandal in the government, b
As if the governments aren't prone to massive scandals even without GPT-4 in place. Try checking the news today, you may be surprised.
>not just any scandal, one that is actually a coordinated attack BY MICROSOFT to screw the government, their largest customer.
"By Microsoft" wasn't in OP's statement. Manipulating data and statistics to get the desired outcome from seemingly impartial algorithms is something our government has been engaging in for a very, very long time.
Redlining[1] and gerrymandering[2] are just two prime examples.
We don't have to make any assumptions or apply biased perspectives to say that we expect:
1)Training data to be manipulated by entities with interest in influencing the outcome;
2)Lives of countless people being adversely affected, and
3)There being a massive scandal as a result.
The government has a long track history of 1, 2, and 3 happening any time a system is introduced which lacks transparency, and large language models are the epitome of that.
The only unrealistic thing about OP's statement is that a scandal will actually happen, instead of people getting away with it.
>It's always interesting to see how people engage with new technologies on HN.
And it is even more interesting to see how people here engage with ethics.
Meanwhile I know of highly skilled engineers in FEDGOV who not only experiment with the APIs for hobby projects but could give a very good lecture on how they work, their limitations, and affordances.
But hey, I guess all those dumb dumbs at NASA can’t tie their shoelaces.
Microsoft could also manipulate the US government by giving its own spin to the "right" answers and thus changing the outcome of the decisions. Basically a form of indirect control depending where and how it's allowed to be used.
It could also be used manipulate people:
- engage with people in online forums to stir their opinion
- making it seem the consensus is different with many posts
It could also be used affect people's freedom of movement:
- By analyzing someones social media posts, ChatGPT could say someone is a: "dangerous individual" which could be used to deny US entry or provide a justification to be constantly monitored.
It could also affect people's privacy:
- Doxxing people on reddit and other social media
- Identifying account belonging to a group or the same individual based on writing patterns, subreddits visited, etc...
GPT answers are dynamic not constant like wikipedia, scientific articles, information records. It's also individualized, thus others can't help spot it is wrong. Answers are subject to be filtered or manipulated according to a hidden set of rules by unknown corporate or government agendas that make it hide or provide incorrect and misleading information, for example, in antitrust investigations, internal governmental investigations, etc...
Agreed in genera but the volatile edit history of many Wikipedia articles would bely your definition of constant.
Also scientific articles being constant don't quite matter when the "status quo" of science is asymmetrically understood.
There are doctors today still pushing the "fat is unhealthy" myth because of the sugar industry. Despite the ubiquity of modern unbiased research papers and proof.
But when you ask a doctor for advice, you're essentially getting the wetware equivalent of a stochastic parrot, because this random doctor is giving you dynamic answers coming from his/her own completely non-standardized corpus of knowledge.
To be fair, at least Wikipedia offers a traceable history of those edits. Most, if not all sufficiently large LLMs, are effectively opaque boxes that even their creator can rarely explain how a specific answer was reached, much less how the billions of parameters changed in the network over time.
This reads a little too high on the conspiracy meter but regulation in the spirit of GDPR would be welcome (in which automated decision making with no human input is unlawful).
Any takers on how long before we get a candidate that simply does whatever their LLM tells them to do? Pretty sure there's at least two short stories, five Doctor Who episodes, and one Star Trek A-plot that explores this. It usually did not end well.
> ...when it’s handed to unskilled, underpaid employees who don’t care and are used to just following instructions?
From experience, this is sort of lazily libeling a broad cross-section of public servants. The public sector has no monopoly on useless employees, you'll find them in equal if not greater numbers in the private sector.
There are plenty of very highly skilled people in government who are quite passionate about serving the public good. The one thing you're correct on is that they're underpaid, and we're fortunate that they're passionate enough about public service and aren't focused on salary-maxing. They're stuck in a system that's managed by the whims and grandstanding of politicians, and attacked as 'the deep state' by people who want to take us back to the spoils system where every public employee is a crony.
Skilled or not, the government modus operandi is not designed to facilitate nuanced, bespoke, individualized responses to people and problems.
If you think it's frustrating dealing with Google/YouTube arbitrarily and capriciously and erroneously and mercilessly shutting down your account and providing zero explanation and zero accountability and zero helpful customer service...
Now imagine that for crucial government services. If there is a flag or datapoint in your government data, then that's that. Too bad. Can't help you. That's what the computer is telling me.
This happens on a daily basis right now, for any number of human or computer errors.
Now insert a hallucinating ML model that only a subset of the population truly know how to use in small doses for productivity, and expect the GOVERNMENT to utilize it in a non-erroneous way?
> and expect the GOVERNMENT to utilize it in a non-erroneous way?
This is what I'm talking about. Saying "the GOVERNMENT" is a ridiculously broad stroke, it's like saying "the INTERNET" (which, ironically, was brought to us by a particularly smart group of people working for the government).
> Now insert a hallucinating ML model that only a subset of the population truly know how to use in small doses for productivity
What's special about the tech elite that makes them any better at using these hallucinating ML models? The government includes vast numbers of specialists and researchers, and many of them are a lot smarter than we are. Government doesn't consist entirely of DMV employees.
My guess is the models will be ideologically driven and will robotically enact the governing agenda without humanity, compassion, or understanding. Hmmm.
> unskilled, underpaid employees who don’t care
One of these things is not like the others. The models may not ask for a raise but they’ll no doubt go to work finding all sorts of other excuses to raise taxes.
> Lying about everything; require human curation to procedure usable results, that will be biased because of that curation; being extremely good at convincing people they should trust them
So humans?
Most of the "AI" fears, including the ones you picked out, are also prevalent in people. Therefore, I don't see the problem.
Humans have been inventing and using tools to increase the scale and efficiency of the means of production since time immemorial. "AI" is nothing new.
>removal of accountability?
Not holding the makers of "AI" models and users of "AI" accountable has nothing to do with "AI" itself. "AI" is just a tool like a hoe or a kitchen knife or a gun, accountability lies with whoever made and/or uses it.
It'll be transformational, but not so much due to benefits of the models themselves. More due to the wave of litigation and legislation and regulation they'll prompt.
I work for a gov. agency, and we're in the midst of a project (with your typical MBB consulting firms) to map the use of generative AI/ML tools, and what possible benefits there are.
One immediate internal benefit would be the information retrieval bit - you kind of get rid of the "data" barrier, which involves knowledge in databases/SQL etc.
Gov. agencies typically have lots of bureaucrats with deep domain knowledge in laws, regulations, and whatever the field they're working on - but limited data knowledge. And instead of relying on analysts etc. to retrieve the needed information, these LLMs could bypass that step.
Of course, it's not entirely that straight forward - as you'd need to validate the things the LLM serves you, but that's one of the ideas. Leadership have been discussing AI/ML non-stop for the past 6-7 months, and it seems like these kinds of FOMO projects are popping up everywhere...good times for the consulting firms.
You'd be shocked and amazed how poorly designed and managed many gov. databases are. Many of these databases follow this pattern:
1) Some worker is fed up with having to sift through hundreds of excel spreadsheets in order to find the information need, and the nightmare of keeping such spreadsheets updated
2) Said worker deploys a small database - just for their own use. After some time the DB gets more users, who make their own views and what not. The database and tables within may or may not follow some rules. Heck, maybe the worker that created it was learning as they went on.
3) Said worker quits or gets a new job, and the database is essentially unmanaged. Some other worker might not now about that DB, and you go back to step 1)
Now multiply a database like that with 100, and span it over 20 year. You get this unimaginable spaghetti monster of multiple DBs, some written in one dialect, some in others. Some are completely unnormalized, others a high degree of normalization.
And to build a report, you may need to access tables from numerous such databases. Even seasoned analysts dread starting on the reports, because they'll spend a good day just to find the right dbs, tables, and all the errors.
So of course when the directors hear about this new fangled AI magic that just spits out the results when you ask it it plain English, they immediately order a use/benefit analysis on it.
Ok, so that may be a bit harsh - but that's the reality of many agencies. Dogshit DB management, and being 20 year behind the digitalization revolution.
But yes, a problem would of course be: How do you know that the data the LLM returns is true? Is it conjuring up fake data? Does it process/calculate data as you want it to?
This summarizes the actual hard parts of my job as a government statistician.
But I'm hoping LLM-generated code (R in my case) can let me stay "in the flow" when exploring. I can spot-check generated code pretty quickly, but finding data, reading their docs, then finding packages to do what I want takes time. I'll often forget why I asked the question in the first place. For example, "How many children lived within 50 miles downstream of X in 2015 and were diagnosed with Y?" I can imagine how the finished code would look, but writing it myself means brushing up on the diagnosis records, two or three GIS datasets, and a GIS package. If we had a nice database or warehouse, this wouldn't be terrible.
I don't fear for my job. The coffee would be generated by including snippets of previous analyses in prompts, and guess who wrote those snippets? My role will become less coding, and more reaching out to policymakers and nonprofits to help them answer questions. I'll tell them what data we do have, what's reliable, and what kind of statistics can answer their questions. At least, that's my dream.
It might be ok if that data gets validated by experts in the same field, but there have been plenty of stories about lawyers and professors who failed to even review the output they use. Color me skeptical.
> - as you'd need to validate the things the LLM serves
And that is not as hard as retrieving that originally, how so then?
Also no "here we have a prototype never intended to be the real thing, now ship it"-mentality there? Oh, certainly not, it is a non overloaded government agency :D
> > - as you'd need to validate the things the LLM serves
> And that is not as hard as retrieving that originally, how so then?
Not necessarily. With good prompts GPT-4 is decently good at citation of exact sentences in the source. And there could be a separate system that verifies that citation text matches the real text for being safe.
> And there could be a separate system that verifies that citation text matches the real text for being safe.
This is what we’re doing with LLMs at my job and it works quite well. Essentially, the LLM generates a space of possible answers that are then validated with various procedural logic checks. The two strategies work well together, whereas either alone wouldn’t produce as good of results.
How can someone validate the data if they can't query the original source? I understand the argument "an expert is made more efficient", but you are saying that the model enables someone that doesn't know what they are doing to suddenly be able to supervise the model doing it for them.
I completely agree that greatly increasing data accessibility is a huge unlock and value add.
A package I open sourced recently might be useful for use cases like this, https://github.com/approximatelabs/datadm It's essentially a chatGPT code interpreter, specifically designed to work with data, that can be run entirely on open models (eg. StarChat). True local mode operation.
I can confirm the giant FOMO happening, and LLMs projects are popping up everywhere.
Good news is, the "talk-to-your data" use case you're describing is not the only one that can deliver amazing new tools! There's the "explain-me this", "summarize that", and "next-best-action" use cases that shows great promise!
>> One immediate internal benefit would be the information retrieval bit - you kind of get rid of the "data" barrier, which involves knowledge in databases/SQL etc.
In that case, I would recommend a search engine (i.e. Lucene, Elastic, Solr, or other) that holds the agency's 'knowledge' before I'd try feeding all of that into an AI such as GPT-4. Granted there are a lot of ML tools that can be statistically rigorous but GPT-4 is generative and therefore not providing access to original source material, which is troubling to reconcile with its potential use in any organization that needs to establish and maintain a ground truth.
Looking at some of the security issues Azure has had in the recent past[1], I am not convinced that Microsoft is at top of their game when it comes to security. Now with Open AI's primary infrastructure being on Azure, I am seeing all these tenancy breaks in a new light. Earlier, if you did not have an Azure presence then these critical issues did not affect you but now if you have enterprise Open AI (which most tech companies probably already do by now) then they have a ton of your data and one of these tenancy breaks could be used to get your data. Open AI is probably not very mature in their security posture and them hosted on Azure is a double whammy if you truly care about your data.
Note that this probably does not affect US govt because from what I understand MS has a totally isolated dedicate Azure environment for US Govt entities.
I hate MSFT, like any other reasonable technologist, but if this results in laying off redundant & administrative bloat from the federal workforce, I'll support it
I saw a live demo of this recently and it was very impressive. It is very difficult to search the govt knowledge base because sources are in so many formats, different location, and have different hoops to gain access not to mention code words, acronyms, abbreviations, and the mixing of technical language with contract language. I think it will mostly used as a search tool rather than generating ideas.
I think the worry about GPT-4 making things up is valid. We all know what happened to they lawyer who used GPT. But I think this comes with training. Users need to be trained to use it as tool and to verify the outputs. Now will everyone do this? No. There are lazy and incompetent people in every large organization and the govt is no different.
The concern about a LLM influencing or biasing its output is also a worry. Maybe there isn't a good solution to this one. I would say that having a govt group testing and assessing it would be best however they don't have the expertise form such a group which is the whole reason why they are leaning on companies like MS to guide their AI usage in the first place. I could also argue that this may not matter much anyway since the govt decisions are already heavily influenced by lobbyist and illegal promotion / favoritism of contractors.
Government agencies are traditionally slow, inefficient, and incompetent. I'm afraid this will make them faster and more efficient, but do nothing to remedy the incompetence. Dark portents ahead.
A fast and efficient government will encourage smarter people to join the ranks and make it more competent. No good developer wants to work in an organization where builds take 8 hours overnight, in much the same way that no good bureaucrat wants to work in an organization where regulations take 8 years to change.
227 comments
[ 3.5 ms ] story [ 286 ms ] threadI suspect Microsoft would do something similar, no data would leave government property.
Doesn’t AWS have a gov offering? I would assume MS can have similar offering with Azure using their express route.
More than that, they even have an air-gapped regions for more sensitive purposes, where you need not only to be a US citizen, but also a certain level of security clearance to access. As an (now ex-) AWS SDE working outside of the US, those regions were a pain in the ass to bring up your service in (as was govcloud, but that wasn't nearly as bad—at least there the ops guys could share logs and screenshots without having to work around an airgap)
The ops guys with direct access to those regions had to be on-site, as they only had access from a limited number of locations, and they were often severely understaffed. They were a pleasure to work with, but you could be waiting a day or more to find out your deployment had failed (the longest SLA I had to deal with in my time there was ~2 weeks due to some unfortunate vacation timings over the summer).
And what's sad is ChatGPT is really the best machine translator in existence. It's not hard to imagine it making a serviceable substitute for a translator, at least in principle, if uncensored.
I explained the parent's comment to you, I didn't say this specific action would cause the end of civilization. And yes while we're at it the government using Microsoft Encarta or even Google to do research is also anti democratic, you'd want the government employees to use a non censored search engine instead of just seeing the snopes fact checked articles if they're going to base their life affecting decisions on it, and if Encarta had something like "the king of Gondor is a bad person" because some microsoft employee wrote it, and the CIA based their decision on that to assassinate him, then it'd be antidemocratic as well. I'll let you stare at the finger and miss the moon telling me how Gondor isn't real.
Government wages are already crap. How much crapier do you want them to get?
Compare it to any other remotely advanced economy in the world, and you'll see it's not out of line.
Any problems that exist are mostly with local and state government pensions, not the federal government.
To hear that 6.4% of the GDP is spent on gov pensions is shocking to me.
That means that we are spending 6.4% of everything on just the pensions of 16.1% of the workers. If we did the same for all workers, we’d be spending 39.8% of the GDP just on pensions. That seems outrageous to me, but I’m not an economist. Why would it be that high?
This is inline with most advanced economies.
Of that ~6.4% - A LOT of that is healthcare related costs.
Considering that healthcare in the US is ~18.2% of GDP - and the vast majority of it comes from retirees - I don't know what you're expecting to see here.
Essentially, you're shocked that healthcare in the US is ~18.2% of GDP, and ~75% of that comes from retirees...
Social Security spending is ~5% of GDP, and Medicare spending is ~4% of GDP... That's for only ~19% of the population. And it doesn't even cover all of their healthcare costs.
If you keep the same ratio for government employees, that comes down to ~2.84% of GDP for healthcare & 3.56% for what most people think of as pensions...
And you're looking at ~65% of that spending coming from state & local governments, not the Federal government.
If pensions are considered part of the labor cost, that would mean not 25% of income going to retirement, but 66%! Or, if pensions are separate from labor, you’ve used up all 100% of the GDP, just between labor and pensions. Neither seems reasonable.
Furthermore, the US only spends 7.5% of the GDP on pensions all told.[0] If these numbers are equivalent, that means 85% of pension dollars are going to government workers.
[0]: https://data.oecd.org/socialexp/pension-spending.htm
This is a bold assertion that I'll charitably entertain.
> Frequently better.
To further assert the government is frequently better demands evidence.
1. Information sensitivity. Even ignoring classified information, there are quite a few things we can't even put into a Google search. It's definitely a no-go for this to end up in a training dataset.
2. "Hallucinations"
Making LLM available through some infrastructure that is already approved for sensitive information will definitely help with the first point, and allow us to experiment with more areas where it might be helpful. I presume this would come along with guarantees about the interactions not being used for training.
It might be even better if some company would sell an appliance we could install on-prem with similar non-training guarantees. Then we could leverage these new tools for very sensitive information, which could be a great help.
> "Documents Sen shared with The Register said to be from the exposed server include a rich amount of data that certainly be valuable to a foreign adversary. It included all the usual PII, as well as blood type, religious affiliation, educational background, military service history and more, all in plain text. Sen told us that close to 3TB of data was available before the Azure server was taken offline on Monday."
Any any operation can make a huge mistake like that. E.g. the guy that leaked all those documents to discord.
I understand they want to protect their IP but I don’t think the model leaking will cost openai billions.
No. These new tools cannot provide leverage. They just produce a street pizza (1) based on the inputs they are given. Whether the street pizza is any good depends on the quality of the ingredients and how discerning the consumer is.
1. https://englishdaily626.com/slang.php?173
1. Lying about everything
2. Require human curation to procedure usable results, that will be biased because of that curation
3. Being extremely good at convincing people they should trust them
Gee, I wonder if there’s anything that can go wrong here, when it’s handed to unskilled, underpaid employees who don’t care and are used to just following instructions? And they start making decisions that affect peoples lives?
I can bet $100, that if that goes beyond any alpha test, we’ll have huge scandal in the future of how those models were purposely modified to influence government, that will make any social media scandal look like a child’s play.
EDIT: and I assume that there won’t be an information leakage - that’s the easiest problem to tackle, with separate infra that government clouds are reasonably good at.
LLMs, in my opinion, are on average better at writing copy than humans. Or rather they manage to do so more consistently. Thus, even if humans still need to provide supervision, the clarity and conciseness of government documents may be improved overall.
Incidentally, this makes a fine tuned version perfect for the government, which produces a shit load of low-effort documents, but today employs a bunch of morons to write them.
If you want a lot of text that you don't want to think about, LLMs are great. If you want text that is pithy or persuasive, it doesn't help much, even when you use prompting tricks (eg "you are a CEO/professor..."). By the way, this has convinced me that local LLAMA-scale models are the future, not massive remote GPTs.
I appreciate the rest of your post though and I think you make a great point about AI being very effective for certain types of work
Some of them could be in the beginning part of their government career attending school. Did you run a survey to ask these people what they do with their lives?
> If you want a lot of text that you don't want to think about, LLMs are great. If you want text that is pithy or persuasive, it doesn't help much, even when you use prompting tricks (eg "you are a CEO/professor..."). By the way, this has convinced me that local LLAMA-scale models are the future, not massive remote GPTs.
Granted, we are judging LLMs based on highly generalized training sets. What if a GPT was fine-tuned on all of the writing and speeches of figures whom are considered the most persuasive?
Government reports continue to balloon in size, because of technological advancements (especially copy/paste).
In the age of typewriters, anything not important, or duplicate, was cut. With word processors, document sizes unnecessarily grew.
All things considered, an awkward sentence or two isn't really that much of a big deal compared to an incorrect assertion that was missed in review.
I also don't think GPT-4 is necessarily that brilliant at writing copy, but I don't really know how good/bad US government officials are so I can't really compare, but in other parts of the world I haven't really that much of an issue with it.
For example right now we have AIs writing CVs and cover letters that are read by AIs, we're "optimizing" things that shouldn't exist anymore but we're too deep in the cycle to even notice it
Some communication is intended to inform, some is intended to persuade... And often, the speaker doesn't even really know which they are dealing with.
The best way to get people to interact with you a certain way is to reward them for interacting with you that way, and you likely don't. Not because that's a failure of you but it's just how people (all of us) act.
The world isn't a co-operative multi-tasking OS. We're more like the human body: a multi-agent conglomerate whose component organisms communicate via signals that are filtered and processed. Sometimes a pathogen mimics signals well enough. Other times we detect it. But the system wouldn't work if every participant trusted the counterparty.
Ability to interpret/summarize is limited by context size and time.
Document dumping is the social DoS.
So now you have 2 machines talking to each other in text-to-speech English over audio, then having to do speech-to-text on each others audio streams, and then natural language processing to understand the message etc.
When all we needed instead was a simple API endpoint over the internet for the business where a machine could talk to the other machine with just a few bytes of information to convey the same information and process the same transaction.
Instead we are wasting bandwidth and tons of cpu cycles and electricity making the machines emulate a human analog conversation.
Maybe in the not so distant future, the AI agents can have a shibboleth which cuts the human gobbledygook short, and reverts to speaking in dialup tones.
I disagree here.
But what I found LLMs really useful for is summarizing topics like: "Suppose I want to do XYZ, which steps should I take?" or "I want to write about ABC, how would you structure the article?"
The content is not very good if you look at it in detail, but LLMs provide good overviews and gave me ideas on parts of a topic I overlooked.
More librarian than search engine. I cannot copy and paste and rely on the results but I can rely on that they point me in a good direction.
> More librarian than search engine. I cannot copy and paste and rely on the results but I can rely on that they point me in a good direction.
The same can be said of physical books, and most content should be written more like that in books than what the internet is currently selecting for.
For the boilerplate- well sure, but I could do that with existing tools, no AI needed. Custom communications I'm sure AI can handle the very basics. but once a human replies and consideration is needed for special cases- can you imagine ChatGPT counseling you on a healthcare decision? I can't.
I can't imagine ChatGPT doing much more than being an autoattendant until a human can reply fully, except the message humans can send to the agency will be customized and drafted according to a standard intake process.
The question is, do you want an AI writing the brief and doing the research that is going to be given to an administrator who is going to decide if you are denied medical care in an emergency?
A disturbingly high proportion of US adults (at least in the US—I don't know how this looks in other countries) barely count as literate. Perhaps half[1]. Most of the rest aren't a ton better, and can't write worth a damn.
I think "on average" understates the case: at least 90% of US adults would probably get better results by passing their writing through ChatGPT, than not. Most of that remaining 10% would also see improvements from ChatGPT, except for writing they've put a great deal of effort into. It's very good at cleaning up writing, and/or tuning it to be better-suited to some purpose than the original was.
In fact, a large majority of readers in the US would probably benefit from passing what they read through ChatGPT and having it summarize it concisely, in very simple terms.
The world makes a lot more sense, to those who grew up taking to the written word as naturally as a duck to water, when we appreciate that the world runs on the written word, yet most people are terrible readers and worse writers.
[1] "According to a 2020 report by the U.S. Department of Education, 54% of adults in the United States have prose literacy below the 6th-grade level." And, nb, if you're a good reader and you're thinking "6th grade level, that's not so bad", your idea of what a "6th-grade level" looks like, is probably skewed way higher than what they mean.
https://en.wikipedia.org/wiki/Literacy_in_the_United_States
Bullet list -> AI text generator -> AI text summarizer -> Bullet list
Perhaps OpenAI can optimize the process by storing the original prompt and just giving it back at the last phase.
this is quite saddening.
IIRC the high-school-graduate equivalent adult literacy rate from similar studies—as in, possessing reading skills that we hope a high school graduate would have, which basically equates to being able to read two moderate-length, moderate-complexity texts and understand them well enough to synthesize a description of their positions, and how they agree and differ—isn't much more than 20%. Four out of five people you encounter are going through life constantly struggling to understand WTF is happening or what they're supposed to do and getting by on a lot of guesswork, while all continually mis-communicating with one another. Really explains a lot.
To continue the entirety of the quote which makes it not seem quite as dammning:
"Literacy in the United States was categorized by the National Center for Education Statistics into different literacy levels, with 92% of American adults having at least "Level 1" literacy in 2019.[1] According to a 2020 report by the U.S. Department of Education, 54% of adults in the United States have prose literacy below the 6th-grade level.[2]
In many nations, the ability to read a simple sentence suffices as literacy, and was the previous standard for the U.S. The definition of literacy has changed greatly; the term is presently defined as the ability to use printed and written information to function in society, to achieve one's goals, and to develop one's knowledge and potential.[3]"
https://en.wikipedia.org/wiki/Literacy_in_the_United_States
I'd expect writing samples to get shorter, not longer, when being improved.
Since when are Government employees underpaid? The salary is typically competitive for the work being done and the benefits/pension are outrageously generous when compared to the private sector.
If you disagree, take a look at govt SWE jobs postings.
There has to be some way to explain a pay discrepancy if it exists. How does one justify receiving a lower salary for the same amount of skill and work?
I suspect (admittedly without evidence) that someone in the NSA or Marines are not primarily driven by a paycheck. Maybe that's what you mean by "true believers"?
You might need to define what you mean by "outrageously generous". I've worked in multiple public and private roles and, in general, the private ones were the ones with much more generous packages, in terms of pay and benefits.
This worldview would also mean that blue-collar employees in the government are more skilled than their private sector counterparts, no?
If that came from an organic brain we would call that clear intent.
I would however stop stick with “lying requires agency,” even if the model evaluation is crafted to be deceitful. The intent and the agency lies with the prompt creator, and therefore, IMO, the lie starts and ends there.
This is a pretty shitty assessment of places like DTIC, DoE, and NASA. Maybe there's another analysis that doesn't involve web forum commenters being the only wizards who can control the magic?
https://digitalreadymarketing.com/wp-content/uploads/2014/08...
https://www.nytimes.com/2021/02/18/opinion/fake-news-media-a...
(edit) how many people do you think are aware of LLM hallucinations as opposed to inaccurate google results
LLMs will be involved in complex tool chains like everything else.
Why? Any who understands about Google or Wikipedia can understand about LLM.
If someone blindly trusts and uses an LLM without checking for more evidence, then they would've done the same with any other source, be it Google, the newspaper or whatever.
"In a cringe-inducing court hearing, a lawyer who relied on A.I. to craft a motion full of made-up case law said he “did not comprehend” that the chat bot could lead him astray." [1]
[1] https://www.nytimes.com/2023/06/08/nyregion/lawyer-chatgpt-s...
Also the problem isn't necessarily about understanding the tech, but more about how it will be perceived at first. As long as every big player hypes up the tech as if it's perfect and revolutionary, I can't blame people for blindly trusting it at first(because it might as well work for most things they use it for, it will take some time before the pitfalls become apparent).
I agree there's cause for concern, and I hope it's a gradual rollout that's introspected in between each propagation of such, but I definitely think it's viable to be used no more less safely than other tooling in the government today (for better or worse).
Further, the narrative you paint suggests that no one in government is competent in anything and can only follow directions.
Finally, you combine the heavily biased perspectives you took and proposed that it would lead to a massive scandal in the government, but not just any scandal, one that is actually a coordinated attack BY MICROSOFT to screw the government, their largest customer.
Just interesting to see.
"I'm sorry sir, but the computer clearly says that you've been charged with sexual harassment - we cannot renew your driver's license and we're putting you on the sexual offenders list, please report to your local PD weekly." (completely made up and exaggerated, but I'm sure one can think of similar scenarios) [1] Good luck convincing the clerk that that's not real. You can probably get it corrected after months of your time and $$.
[1] https://www.washingtonpost.com/technology/2023/04/05/chatgpt...
[1] https://www.nytimes.com/2010/01/14/nyregion/14watchlist.html
There are 340 million people in the US. The US has one of the largest government systems that humanity is likely to ever see. How many similar cases of children being put on the terrorist watch list have there been in the past 20 years? Unless there are a lot of cases of that happening, if it's not rare (hint: it's extraordinarily rare; your example story is from 13 years ago), your premise is pure straw.
[0]: https://www.wired.com/story/face-recognition-software-led-to... [1]: https://www.theguardian.com/us-news/2023/apr/27/california-p...
>(completely made up and exaggerated, but I'm sure one can think of similar scenarios)
How about "I'm sorry sir, but the computer clearly says that you've been at the site of murder and thus killed this guy. Even though all other direct evidence contradicts this, we're gonna have to arrest you. "[1]
The best part of working on projects that used that data set in Google was learning about it from articles like this one.
[1]https://www.dailymail.co.uk/news/article-7897319/Police-arre...
We are holding it to a higher standard because this is going to be used by agencies with serious power over people's lives. It is not just to spit out, for example, a form rejection letter from a credit card.
And, nobody thinks that Microsoft wants to "screw" the government. But if something happens, the impact can be magnified significantly considering the reach and power of the state. I'm all for moving forward quickly with this tech, but there are a handful of spaces (government, medicine, aeronautics) where we need to be extremely deliberate.
I feel for my kids, and all I can do is warn them...
https://theconversation.com/excel-errors-the-uk-government-h...
The vast majority of government workers are underpaid apathetic cogs in a machine who do not fit this bill.
We just saw a LAWYER use hallucinated information in his practice. A lawyer who is ostensibly rigorously trained and tested in his field.
The amount of misuse of GPT by way of sheer ignorance alone by the government will be astounding.
When talking about the pitfalls of bringing GPT-4 as a tool for government officials, the positive features are outside of the scope.
Obviously they exist, otherwise it wouldn't be proposed. But it's not our job to do a sales pitch for GPT-4.
>Further, the narrative you paint suggests that no one in government is competent in anything and can only follow directions.
No. The OP suggests that the primary users of the tool would be low-level, underpaid officials, who are mostly following directions.
Not because it's their fault as human beings - but because their positions require them to do so, and perhaps for good reasons.
In any case, direction-followers (of questionable competence and low pay grade) make up the bulk of large organizations in general as a consequence of growth. Which is why the tool is being pitched in the first place: to streamline work of people who would rather not do it in the first place.
>Finally, you combine the heavily biased perspectives you took and proposed that it would lead to a massive scandal in the government, b
As if the governments aren't prone to massive scandals even without GPT-4 in place. Try checking the news today, you may be surprised.
>not just any scandal, one that is actually a coordinated attack BY MICROSOFT to screw the government, their largest customer.
"By Microsoft" wasn't in OP's statement. Manipulating data and statistics to get the desired outcome from seemingly impartial algorithms is something our government has been engaging in for a very, very long time.
Redlining[1] and gerrymandering[2] are just two prime examples.
We don't have to make any assumptions or apply biased perspectives to say that we expect:
1)Training data to be manipulated by entities with interest in influencing the outcome;
2)Lives of countless people being adversely affected, and
3)There being a massive scandal as a result.
The government has a long track history of 1, 2, and 3 happening any time a system is introduced which lacks transparency, and large language models are the epitome of that.
The only unrealistic thing about OP's statement is that a scandal will actually happen, instead of people getting away with it.
>It's always interesting to see how people engage with new technologies on HN.
And it is even more interesting to see how people here engage with ethics.
[1]https://www.nytimes.com/2021/08/17/realestate/what-is-redlin...
[2]https://www.brennancenter.org/our-work/research-reports/gerr...
But hey, I guess all those dumb dumbs at NASA can’t tie their shoelaces.
I'm from the north-middle, sometimes I'm surprised how little people on the west coast of America think of the people on the east coast.
Above All, The NSA knows their shit. 44B+, Autonomy and Mathematicians Proves it above all, regardless of Morals.
Don't attack them on knowledge. You have to use a different approach to climb Everest.
I would only put the Mossad, KGB and <insert name of the chinese SIGINT agency> on their level.
Just communicate more clearly, you're clearly displeased and want to say something more?
2. Require human curation to procedure usable results, that will be biased because of that curation
3. Being extremely good at convincing people they should trust them ```
So it's absolutely perfect for the US government.
4. Having no legal accountability
(while arguably providing a means of avoiding or reducing accountability for those using them, "LLM says no" style.)
Every new technology initially has its own shortcut comings.
Steve jobs in his inaugural iPhone launch, used four different phones because each phone was capable of only one feature.
It could also be used manipulate people:
- engage with people in online forums to stir their opinion
- making it seem the consensus is different with many posts
It could also be used affect people's freedom of movement:
- By analyzing someones social media posts, ChatGPT could say someone is a: "dangerous individual" which could be used to deny US entry or provide a justification to be constantly monitored.
It could also affect people's privacy:
- Doxxing people on reddit and other social media
- Identifying account belonging to a group or the same individual based on writing patterns, subreddits visited, etc...
When was the last time you tried to make a wiki edit on a popular page?
Also scientific articles being constant don't quite matter when the "status quo" of science is asymmetrically understood.
There are doctors today still pushing the "fat is unhealthy" myth because of the sugar industry. Despite the ubiquity of modern unbiased research papers and proof.
But when you ask a doctor for advice, you're essentially getting the wetware equivalent of a stochastic parrot, because this random doctor is giving you dynamic answers coming from his/her own completely non-standardized corpus of knowledge.
> 2. Require human curation to procedure usable results, that will be biased because of that curation
> 3. Being extremely good at convincing people they should trust them
You just described politicians
It’s worse than that. Employees of the US government are definitely not underpaid, especially when you consider
What the hell was I thinking that I couldn't finish?
> 1. Lying about everything
> 2. Require human curation to procedure usable results, that will be biased because of that curation
> 3. Being extremely good at convincing people they should trust them
Finally, we automate politics!
From experience, this is sort of lazily libeling a broad cross-section of public servants. The public sector has no monopoly on useless employees, you'll find them in equal if not greater numbers in the private sector.
There are plenty of very highly skilled people in government who are quite passionate about serving the public good. The one thing you're correct on is that they're underpaid, and we're fortunate that they're passionate enough about public service and aren't focused on salary-maxing. They're stuck in a system that's managed by the whims and grandstanding of politicians, and attacked as 'the deep state' by people who want to take us back to the spoils system where every public employee is a crony.
If you think it's frustrating dealing with Google/YouTube arbitrarily and capriciously and erroneously and mercilessly shutting down your account and providing zero explanation and zero accountability and zero helpful customer service...
Now imagine that for crucial government services. If there is a flag or datapoint in your government data, then that's that. Too bad. Can't help you. That's what the computer is telling me.
This happens on a daily basis right now, for any number of human or computer errors.
Now insert a hallucinating ML model that only a subset of the population truly know how to use in small doses for productivity, and expect the GOVERNMENT to utilize it in a non-erroneous way?
This is what I'm talking about. Saying "the GOVERNMENT" is a ridiculously broad stroke, it's like saying "the INTERNET" (which, ironically, was brought to us by a particularly smart group of people working for the government).
> Now insert a hallucinating ML model that only a subset of the population truly know how to use in small doses for productivity
What's special about the tech elite that makes them any better at using these hallucinating ML models? The government includes vast numbers of specialists and researchers, and many of them are a lot smarter than we are. Government doesn't consist entirely of DMV employees.
> unskilled, underpaid employees who don’t care
One of these things is not like the others. The models may not ask for a raise but they’ll no doubt go to work finding all sorts of other excuses to raise taxes.
So humans?
Most of the "AI" fears, including the ones you picked out, are also prevalent in people. Therefore, I don't see the problem.
Humans have been inventing and using tools to increase the scale and efficiency of the means of production since time immemorial. "AI" is nothing new.
>removal of accountability?
Not holding the makers of "AI" models and users of "AI" accountable has nothing to do with "AI" itself. "AI" is just a tool like a hoe or a kitchen knife or a gun, accountability lies with whoever made and/or uses it.
One immediate internal benefit would be the information retrieval bit - you kind of get rid of the "data" barrier, which involves knowledge in databases/SQL etc.
Gov. agencies typically have lots of bureaucrats with deep domain knowledge in laws, regulations, and whatever the field they're working on - but limited data knowledge. And instead of relying on analysts etc. to retrieve the needed information, these LLMs could bypass that step.
Of course, it's not entirely that straight forward - as you'd need to validate the things the LLM serves you, but that's one of the ideas. Leadership have been discussing AI/ML non-stop for the past 6-7 months, and it seems like these kinds of FOMO projects are popping up everywhere...good times for the consulting firms.
If so, I think that makes sense. With the /major/ caveat that you are also making it even easier to get incorrect data out. No?
1) Some worker is fed up with having to sift through hundreds of excel spreadsheets in order to find the information need, and the nightmare of keeping such spreadsheets updated
2) Said worker deploys a small database - just for their own use. After some time the DB gets more users, who make their own views and what not. The database and tables within may or may not follow some rules. Heck, maybe the worker that created it was learning as they went on.
3) Said worker quits or gets a new job, and the database is essentially unmanaged. Some other worker might not now about that DB, and you go back to step 1)
Now multiply a database like that with 100, and span it over 20 year. You get this unimaginable spaghetti monster of multiple DBs, some written in one dialect, some in others. Some are completely unnormalized, others a high degree of normalization.
And to build a report, you may need to access tables from numerous such databases. Even seasoned analysts dread starting on the reports, because they'll spend a good day just to find the right dbs, tables, and all the errors.
So of course when the directors hear about this new fangled AI magic that just spits out the results when you ask it it plain English, they immediately order a use/benefit analysis on it.
Ok, so that may be a bit harsh - but that's the reality of many agencies. Dogshit DB management, and being 20 year behind the digitalization revolution.
But yes, a problem would of course be: How do you know that the data the LLM returns is true? Is it conjuring up fake data? Does it process/calculate data as you want it to?
But I'm hoping LLM-generated code (R in my case) can let me stay "in the flow" when exploring. I can spot-check generated code pretty quickly, but finding data, reading their docs, then finding packages to do what I want takes time. I'll often forget why I asked the question in the first place. For example, "How many children lived within 50 miles downstream of X in 2015 and were diagnosed with Y?" I can imagine how the finished code would look, but writing it myself means brushing up on the diagnosis records, two or three GIS datasets, and a GIS package. If we had a nice database or warehouse, this wouldn't be terrible.
I don't fear for my job. The coffee would be generated by including snippets of previous analyses in prompts, and guess who wrote those snippets? My role will become less coding, and more reaching out to policymakers and nonprofits to help them answer questions. I'll tell them what data we do have, what's reliable, and what kind of statistics can answer their questions. At least, that's my dream.
And that is not as hard as retrieving that originally, how so then?
Also no "here we have a prototype never intended to be the real thing, now ship it"-mentality there? Oh, certainly not, it is a non overloaded government agency :D
> And that is not as hard as retrieving that originally, how so then?
Not necessarily. With good prompts GPT-4 is decently good at citation of exact sentences in the source. And there could be a separate system that verifies that citation text matches the real text for being safe.
This is what we’re doing with LLMs at my job and it works quite well. Essentially, the LLM generates a space of possible answers that are then validated with various procedural logic checks. The two strategies work well together, whereas either alone wouldn’t produce as good of results.
A package I open sourced recently might be useful for use cases like this, https://github.com/approximatelabs/datadm It's essentially a chatGPT code interpreter, specifically designed to work with data, that can be run entirely on open models (eg. StarChat). True local mode operation.
I can confirm the giant FOMO happening, and LLMs projects are popping up everywhere.
Good news is, the "talk-to-your data" use case you're describing is not the only one that can deliver amazing new tools! There's the "explain-me this", "summarize that", and "next-best-action" use cases that shows great promise!
http://switchandlever.com/edm/
In that case, I would recommend a search engine (i.e. Lucene, Elastic, Solr, or other) that holds the agency's 'knowledge' before I'd try feeding all of that into an AI such as GPT-4. Granted there are a lot of ML tools that can be statistically rigorous but GPT-4 is generative and therefore not providing access to original source material, which is troubling to reconcile with its potential use in any organization that needs to establish and maintain a ground truth.
I don’t really care other than I’d like to be able to ensure that my OpenAI library works on Azure.
Note that this probably does not affect US govt because from what I understand MS has a totally isolated dedicate Azure environment for US Govt entities.
[1] https://www.lastweekinaws.com/blog/azures_vulnerabilities_ar...
I can imagine alignment is probably an impediment to several areas of interest.
Not the Skynet.
The government should demand open source code, open weights, and open research from any AI company it purchases services from.
I think the worry about GPT-4 making things up is valid. We all know what happened to they lawyer who used GPT. But I think this comes with training. Users need to be trained to use it as tool and to verify the outputs. Now will everyone do this? No. There are lazy and incompetent people in every large organization and the govt is no different.
The concern about a LLM influencing or biasing its output is also a worry. Maybe there isn't a good solution to this one. I would say that having a govt group testing and assessing it would be best however they don't have the expertise form such a group which is the whole reason why they are leaning on companies like MS to guide their AI usage in the first place. I could also argue that this may not matter much anyway since the govt decisions are already heavily influenced by lobbyist and illegal promotion / favoritism of contractors.
hard to tell sometimes...
This is what "technocracy" is, right? And "corporatism"?