when it first came out and my boss was behind himself about how cool it was, he was feeding it all of his emails with other businesses to have it clean them up. boggled my mind.
Gmail has a vested interested in keeping any knowledge it gains about you secret - it's competitive advantage is knowing more about you than anyone else does.
ChatGPT's strength is its ability to clearly communicate the knowledge it has (including training data it gains from people it interacts with) to give you good responses.
I am still not seeing a huge threat to be honest. This is not how attacks are done. OpenAI also has vested interest to keep your data safe and is strongly linked to Microsoft.
Most corps, companies store a lot of internal data with corps like Google, Microsoft, Amazon and others.
Not only do you have to worry about employees directly sharing data, but many companies are also just wrappers around GPT. Or they may use your data in the future to roll out new AI services.
While this is not a new problem -- employees share sensitive data with Google all the time -- the data leakage will be more clear than ever. With ads-based tracking and Google search, the leakage was very indirect. With generative AI, it can literally regurgitate memorized documents.
The security risk goes beyond data exfiltration. Folks are already trying to teach the AI incorrect information by spamming it with something like 2+2 = 5.
Data exfiltration + incorrect data injection are super underrated risks to mass adoption of generative AI tech in the B2B world...
We block ChatGPT, as do most federal contractors. I think it’s a horrible exploit waiting to happen:
- there’s no way they’re manually scrubbing out sensitive data so its bound to spill out from the training data when prompting the model
- OpenAI is openly storing all this data they’re collecting to the extent that they’ve had several leaks now where people can see others’ conversations and data. We are one step away if it hasn’t already happened from an exploit of their systems (that likely weren’t built with security as the top priority as opposed to scale and performance) that could leak a monumental amount of data from users.
In the most innocent case they could leak the personal info of naive users. But largely if Linkedin is any indication, the business world is filled with dopes who genuinely believe the AI is free thinking and better than their employees. For every org that restricts ChatGPT use, there are fifty others that don’t, most of which have at least one of said dopes who are ready to upload confidential data at a moments notice.
Wouldn’t even put it past military personnel putting S/TS information into it at this point. OpenAI should include more brazen warnings against providing this type of data if they want to keep up this facade of “we can’t release it because ethics” because cybersecurity is a much more real liability than a supervised LM turning into terminator.
Let's also not discount that for every "dope" there is at least one "bad actor" who is willing to take the risk to get an edge in their workplace or appease their managers demands. The warnings will only deter the first group.
If your competitor use ChatGPT to compete with you and they're 10x productive than yours, are you still willing to insist? If the productive is 100x, will you?
> Think about how long it's taken tools like pandas to reach the point that it is now. That entire package can be built to the level it is now in a couple of days.
I don't think that is true at all. Do you have an example of a significant project being duplicated in days, or even months, with ANY of these tools?
By significant, I mean something on the order of pandas which you claimed.
And this is completely ignoring the fact, that the real hard problem is the design. Spitting boilerplate code is not. How pandas could be designed perfectly in one afternoon (and generated with GPT) is beyond my comprehension.
Cool, please provide a link to a library of similar size and complexity to pandas which was written using ChatGPT in the span of a few days. We'll be waiting.
> Think about how long it's taken tools like pandas to reach the point that it is now. That entire package can be built to the level it is now in a couple of days.
Let me hand you a mirror: You're absolutely and completely wrong
> Think about how long it's taken tools like pandas to reach the point that it is now. That entire package can be built to the level it is now in a couple of days.
I am having trouble parsing this statement. You're saying a person equipped with chatGPT trained on data prior to December 2007 (the month before the initial pandas release) could have put together the entire pandas library in a couple of days?
That seems obviously wrong, starting with the fact that one would need to know "what" to build in the first place. If you're saying that chatGPT in 2023 can spit out pandas library source code when asked for it directly, that's obvious.
Somewhere between the impossible statement and the obvious statement I made above, there must be something interesting that you were trying to claim. What was it?
It might be just as likely that ChapGPT will cause a mistake like Knight Capital because no one bothered to thoroughly verify the AI's looks-good-but-deeply-flawed answer, and the two aren't mutually exclusive possibilities.
Humans are a lot better at "I don't know how to do this; hey Alice, can you look this over if you've got a sec and tell me if I'm making a noob mistake"
Perhaps the actual phenomenon is that humans are much better at saying "Alice wrote this code, she's pretty good at scripting but she might have made a noob mistake, better check it", or even "I wrote this code.." than they are at saying "ChatGPT wrote this code, but that application is not guaranteed to have correctly identified my problem, but may have just returned something that seems right both to the statistical model and to me, but which is actually deeply flawed, better check it".
This isn't an argument of ChatGPT vs nothing. This is an argument of "external" ChatGPT vs some other AI sitting on your own secured hardware, maybe even a branch of ChatGPT.
Security and privacy should be table stakes. Speaking for my country, we needs privacy laws with teeth to punish bad actors from shitting people's private information where ever they want in the name of a dollar.
Possibly I don't know how this all works, but I think if the host of a ChatGPT interface were willing to provide their own API key (and pay), they could then provide a "service" to others (and collect all input).
In that case, you wouldn't know to block them until it was too late.
Ultimately either you must watch/block all outgoing traffic, or you must train your people so thoroughly that they become suspicious of everything. Sadly, being paranoid is probably the most economical attitude these days if IP and company secrets have any value.
> Possibly I don’t know how this all works, but I think if the host of a ChatGPT interface were willing to provide their own API key (and pay), they could then provide a “service” to others (and collect all input).
Well, GP was referring to blocking ChatGPT as a federal contractor. I suspect that as a federal contractor, they are also vetting other people that they share data with, not just blocking ChatGPT as a one-off thing. I mean, generic federal data isn’t as tightly regulated as, say, HIPAA PHI (having spent quite a lot of time working for a place that handles both), but there are externally-imposed rules and consequences, unlike simple internal-proprietary data.
But it really seems like a cat and mouse game. For example, a very determined bad actor could infiltrate some lesser approved government contractor and provide an additional interface/API which would invite such information leaking, and possibly nobody would notice for a long time.
And then they could face death penalty for espionage if they leaked sensitive enough data. You would have to be really stupid to build such a service for government contractors unless you actually are a foreign spy.
So you block internet access for all employees? Cos anything you think is being pasted into ChatGPT is being pasted everywhere, whether its Google, Slack, Chrome Plugins, public Wifi.
Yes, these things are sometimes blocked in higher security workplaces… up to and including the public internet. Honestly airgapped systems are not all that uncommon anywhere that human life is at risk.
This really depends on the cost/benefit tradeoff for the entity in question. If using ChatGPT makes you X% more productive (shipping faster / lowers labor costs / etc), but comes with Y% risk of data leakage, is that worth it in expectation or not? I would argue that there definitely exist companies for which it's worth the tradeoff.
But that's the API, not the Chat input or Playground.
Companies can use Azure OpenAI Services to get around this -- there's data privacy, encryption, SLAs even. The problem is it's very hard to get access to (right now).
the #1 problem with corporations saying things is that many things they say are not regulated or are taken on good faith. What happens with OpenAI are acquired and the rules change? These comments are often entirely worthless.
To anyone who may be pasting code along the lines of 'convert this sql table schema into a [pydantic model|JSON Schema]' where you're pasting in the text, just ask it instead to write you a [python|go|bash|...] function that reads in a text file and 'converts an sql table schema to output x' or whatever. Related/not-related--great pandas docs replacement is another great+safe use-case.
Point is, for a meaningful subset of high-value use-cases you don't need to move your important private stuff across any trust boundaries, and it still can be pretty helpful...so just calling that out in case that's useful to anyone...
I’ve been doing this kind of thing pretty regularly for the past few weeks, even though I know how to do any of the tasks in question. It’s usually still faster, even when taking the time to anonymize the details; and I don’t paste anything I wouldn’t put on a public gist (lots of “foo, bar”, etc)
I use it because it's 10-100x more interesting, fun, and fast as a way to program, instead of me having to personally hand-craft hundreds of lines of boilerplate API interaction code every time I want to get something done.
Besides, it's not like it puts out great code (or even always working code), so I still have to read everything and debug it. And sometimes it writes code that is just fine and fit for purpose and horrendously ugly, so I still have to scrap everything and do it myself.
(And then sometimes I spend 10x as long doing that, because it turns out it's also just plain good fun to grow an aesthetic corner of the code just for the hell of it, too — as long as I don't have to.)
And even after all that extra time is factored back in: it's still way faster and more fun than the before-times. I'm actually enjoying building things again.
Pair-programming with ChatGPT is like having an idiot-savant friend who always surprises you. Doesn’t matter if the code is horrible, amazing, or something inbetween. It’s always interesting.
And I agree it’s fun. Maybe it’s the simulated social interaction without consequences. I can be completely honest with my robot friend about the shitty or awesome code and no one’s feelings are going to get hurt. ChatGPT will just keep trying to be helpful.
You can be an experienced developers with years building complex applications behind you and still find ChatGPT useful. I've found it useful for documenting individual methods or simply explaining my own/other's code or writing unit test methods or just using it to add boilerplate stuff that saves me an hour that I use elsewhere.
I think many people find ChatGPT useful specifically because they have years of experience building complex applications.
If you know exactly what you want to ask of it, and have the ability to evaluate and verify what it produces, it's incredible what you can get out of it. Sure it's nothing I couldn't have done otherwise... eventually. The productivity it enables is worth every cent.
Easily the best $20 I've spent in ages, they should have run with the initial idea of charging $42.
But holy moly anyone putting confidential information into it needs to stop
Precisely because I can abstract it is why I use ChatGPT. It can do the boring, tedious, repetitive stuff instead of me and has shown me the joy of using programming to solve ACTUAL problems yet again, instead of having to spend hours on unimportant problems like "how do I do X with library Y".
To your average user who interfaces with these figurative black boxes with a black box in their hand, how is this particular black box any different than the other black boxes that this user hands their data to every second of every day?
Those typically have MSAs with legalese where parties stipulate what they will and will not do and often whether or not it’s zero knowledge and often option to have your own instance encryption keys.
If people are using the free version of chatGPT then it’s unlikely there is a contract between the companies and more likely just a terms of use applied by chatGPT and ignored by the users.
there are plenty of disallowed 'black boxes' within the federal sphere; chatgpt is just yet another.
to take a stab at your question, though : my cell phone doesn't learn to get better by absorbing my telecommunications; it's just used as a means to spy on my personal life by The Powers That Be. The primary purpose of my cell phone is for the conveyance of telecommunications.
chatGPT hordes data for training and self-improvement in its' current state. It's whole modus operandi involves the capture of data, rather than it being used for that tangentially. It could not meaningfully exist without training on something, and at this stage of the game it's the trend to self-train with user data.
Until that trend changes people should probably be a bit more suspect about what kind of stuff gets thrown into the training bin.
We have data standards and agreements with those companies, we pay them to have expectations. Even then, we're strict about what touches vendor servers and it's audited and monitored. Accounts are managed by us and tied into onboarding and offboarding. If they have a security incident, they notify, there's response and remediation.
ChatGPT seems to be used more like a fast stackoverflow, except people aren't thinking of it like a forum where others will see their question so they aren't as cautious. We're just waiting for some company's data to show up remixed into an answer for someone else and then plastered all over the internet for the infosec lulz of the week.
> We have data standards and agreements with those companies, we pay them to have expectations. Even then, we're strict about what touches vendor servers and it's audited and monitored. Accounts are managed by us and tied into onboarding and offboarding.
For every company like yours there are hundreds that don't. People use free gmail address for sensitive company stuff, paste random things in random pastebins, put their private keys in public repos, etc.
Yes, data leaks from OpenAI are bound to happen (again), and they should beef up their security practices.
But thinking people are using only ChatGPT in an insecure way vastly overestimates their security practices elsewhere.
The solution is education, not avoiding new tools.
Doesn't OpenAI explicitly say that your Q/A on the free ChatGPT are stored and sent to human reviewers to be put in their RL database? Now of course we can't be sure what google, AWS etc do with the data on disks there, but it would be a pretty big scandal if some whistleblower eventually comes out and say that google employees sit and laugh at private bucket contents on GCP or private Google Docs. So there's a difference in stated intention at least..
Who in their right mind is using free ChatGPT through that shitty no good web interface of theirs, that can barely handle two queries-and-replies before grinding down to a halt? Surely everyone is using the pay-as-you-go API keys and any one of the alternative ffrontends or integrations?
And, IIRC, pay-as-you-go API requests are explicitly not used for training data. I'm sad GPT-4 isn't there yet - except for those who won the waitlist lottery.
It's really funny to see these types of comments. I would assume a vast majority of users are using the Web interface, particularly in a corporate context where an account for the API could take ages or not be accepted.
If people were smart and performed according to best practices, articles like this one would not be necessary.
I mean, if you're using a free web interface in corporate context, you may just as well use a paid API with your personal account - either way, you're using it of your own volition, and not as approved by your employer. And getting API keys to ChatGPT equivalent (i.e. GPT-3.5) takes... a minute, maybe less.
I am honestly confused how people can use this thing with the interface OpenAI runs. The app has been near-unusable for me, for months, on every device I tried it on.
I wonder what the distribution of tokens / sec at OpenAI is between the free ChatGPT, paid ChatGPT, and APIs. I’d have to think the free interface is getting slammed. Quite the scaling project, and still nowhere near peaking.
> and any one of the alternative ffrontends or integrations?
And what sort of understanding do you have with the alternative frontends/integrations about how they handle your API keys and data? This might be a better solution for a variety of reasons but it doesn't automatically mean your data is being handled any better or worse than by openai.com
To quote a children's TV show: "Which ones of these things are not like the other ones?"
Some of those are document tools working on language / knowledge. Others are infrastructure, working on ... whatever your infra does, and your infra manages your data (knowledge).
If you read their data policies, you'll find they are not the same.
I simply don't give a crap if my employer loses data. I don't care if my carelessness costs my employer a billion bucks down the line as I won't be working for them next year.
"I do not take any kind of responsibility about what I'm doing, or not doing, or thinking about doing or not doing, or thinking about whenever I should be doing or not doing, or thinking about whenever I should be thinking about doing or not doing".
As a moral questionable answering robot however, i must aks, why all things else should be tainted by the machinery, but evidence like text should not?
Not who you replied to, but if you agree, even a little, with the phrase, "the social contract between employees & employers is broken in the US"... well it goes both ways.
I am treating my employment like a corporation would. Risks I do not pay for and do not benefit from mitigating are waste that could allow me to transfer time back to my own priorities, increasing my personal "profit."
At first I was impressed by how easy it was to reach a data model with chatgpt, then I laughed as I tried to tweak it and use it. I realized it didn't really have any model concepts and was just using its various KB.
I am unsure if the so called AI can think in models but so far, not but still an impressive assisting tool if you take care of its limitations.
Another point where it lacks is in logic, my daughter has a lot of fun with the book "what is the name of this book?" but she was struggling with the "map of baal" explanation, her the answer was a certain map, yet the book had another answer, I had a third one as I interpreted a proposition. I never got an answer without a contradiction in chatgpt reasoning, and the book had been mistranslated to French so one of its propositions was changed (C, both A and B were knaves) but not the answer.
> At first I was impressed by how easy it was to reach a data model with chatgpt, then I laughed as I tried to tweak it and use it. I realized it didn't really have any model concepts and was just using its various KB.
> I am unsure if the so called AI can think in models but so far, not but still an impressive assisting tool if you take care of its limitations.
I don't know. I'm using it for exactly that ("here's a problem, come up with a data model") and it gives a great starting point.[0]
Not perfect, but after that it's easy to tweak it the old-fashioned way.
I find its data modelling capabilities (in the domain I'm using it for - API services) to be rougly on par with a mid-level developer (for a handwavy definition of "midlevel").
> If using ChatGPT makes you X% more productive (shipping faster / lowers labor costs / etc), but comes with Y% risk of data leakage
X and Y are not alike, and should not be compared. X is a benefit to you(r employer), whereas Y is a risk to the customer who has entrusted you with their data.
Mate. You aren’t special. It’s the nature of the profession that most of us are in, that we end up dealing with the “sensitive” data that you’re describing, barring most people working in Big Companies with proper internal controls.
Nothing you’ve said negates anything OP said. It’s simply an elaboration wrapped in elitism.
The smart ones don’t paste in all their private data.
And yes, if the bypassing the block is combined with disciplinary action, it does work. It’s not worth getting fired over. This is likely what heavily regulated industries like financial services and defense are doing.
I remember someone trying to look up winning lottery numbers at work. The site came up "Blocked: Gambling". It was a little reminder that they're watching our web browsing at work..
It works in the sense that it does add an extra "reminder" and requires specific intent. I mean, in this scenario all the people already have been informed that they're absolutely not allowed to do things like that, but if someone has forgotten that, or simply is careless and just wants to "try something out" then if it's unblocked they might actually do it, but if they need to work around a restriction, that forces them to acknowledge that there is a restriction and they shouldn't try to work around it even if they can.
Does US intelligence have access to OpenAI data? Private organizations is one thing. But with all the dopes in government positions around the world, OpenAI logs would probably be a treasure trove for intelligence gathering.
Microsoft is well known for piping data to US intelligence as a service. It's almost certainly why they bought Skype, then removed all the end to end encryption.
> Wouldn’t even put it past military personnel putting S/TS information into it at this point.
Hey, they need someone to proofread their War Thunder forum posts to make sure they're using correct spelling and grammar when leaking classified info. ;-)
Not only that, but the European theater nuclear forces leaking security arragements and even door PIN-codes for nuclear weapons bunkers via online flash card sites might be a better example.
> there’s no way they’re manually scrubbing out sensitive data
I was under the impression OpenAI weren't using questions as training data for future models. I recall Sam Altman saying they delete questions after 1 month, but I can't locate the source for that.
Even without training the model, it will still end up in logs. For example you can now see your previous questions in the UI and it'll probably be stored in other places of their backend too.
I am curious, do you block MS Edge? It has a grammar check for all input boxes that sends data to MS servers to check. Similar to what Grammarly does.
MS also "helpfully" asks you if you want to use that enhanced grammar check in MS Word(as far as I have seen, might be there in other office products too). I cannot imagine sending all my documents to MS. But I am not sure most users will realize what is happening.
All these companies offer helpful services but are hovering up data and no one knows the consequences yet. It feels like ChatGPT is just one symptom of a bigger problem.
Some military folks put nuclear weapons storage training materials onto Quizlet, so I don’t doubt for a second people would try to put ChatGPT onto a classified computer system.
By using Azure, you can access ChatGPT and GPT, which come with enterprise-grade security and established data agreements, setting them apart from OpenAI. I'm not entirely sure of the technical details, but you can explore this option.
> they’ve had several leaks now where people can see others’ conversations and data
Do you have a source for this? I know some people have claimed to see others' data, but I haven't seen any evidence that that's what's actually being seen, vs LLM hallucinations. OpenAI claims, and I can't imagine they're lying, that the training data is fixed and ends in 2021, so I don't see how it would be possible for user prompts to be leaking into output, absent a massive and very unlikely bug (compared to the much more likely AI hallucination explanation).
Some kind of concurrency bug in a library they were using to retrieve cached data from Redis led to this leak.
> We took ChatGPT offline earlier this week due to a bug in an open-source library which allowed some users to see titles from another active user’s chat history. It’s also possible that the first message of a newly-created conversation was visible in someone else’s chat history if both users were active around the same time.
> We block ChatGPT, as do most federal contractors. I think it’s a horrible exploit waiting to happen:
Do you also block pastebin? Anything else that has a web form? How is ChatGPT special compared to any other service on the Internet where people can paste data in a form?
I mean... I see the problem, but I think one needs to realize that it's a far more generic problem that has basically nothing to do with ChatGPT and AI. If people paste confidential data into random webpages that's of course bad. But if you block ChatGPT because you fear that, it means you expect that people might do that. And then your problem is not ChatGPT, but lack of awareness what is confidential data and what to do with it.
Because "awareness only" has such a great track record when it comes to security-adjacent issues, and totally satisfies auditors/customers/regulators/...?
I don't think awareness only has any reasonable track record and I would always prefer a technical control if there is one. But I have a hard time seeing any alternative here.
I don't think the idea that you can give people access to the www and at the same time preventing them from putting things in forms can be done. That's simply not how it works. And if you're blocking access to a few services where they might do that, well, they have a million others, and you're deceiving yourself that you've done something.
> pastebin and indeed most things that has some sort of public webform is blocked in all the companies I have worked with.
Search engines too? And these days, that means web browsers, because the (IMHO stupid) idea of combining address and search bars into one means everything you type while trying to open a website gets leaked to some party (most likely Google).
I imagine they must be. I'm habitually careful to either click on a link, paste the entirety of the internal URL at once, or enter only the most generic word or words that will surface the URL I want as a history suggestion - all to minimize the chances of leaking anything this way.
Wouldn't it be trivial to add a "read-only" mode to the LLM's operation, where it uses stored knowledge to answer queries but doesn't ingest new knowledge from those queries?
I'd be worried this is also the "how to get banned from OpenAI in the near future" form. and if OpenAI retains a monopoly like Google does for search, you are basically screwed.
I'm old enough to remember when newspapers reported hackers being banned from using computers.
And IP pirates banned from using the internet. Actually, that one I remember I voted against my local MP after they passed a law to make that the norm.
We don't yet have a social, let alone legal, norm for antisocial use of LLMs; even with social media, government rules are playing catch-up with terms of service, and Facebook is old enough that if it was human it could now vote.
So, yes, likewise computers/internet/social media, being banned from an LLM if it's a monopoly is going to seriously harm people.
But that is likely to be a big "if". The architecture and the the core training data for GPT-3 isn't a secret, and that's already pretty impressive even if lesser than 3.5 and 4.
Nope. LLMs don't use the internet for inference at all unless you give it access to a web search api or something like that. chtGPT is just too massive to run on any local machine. But make no mistake, it does not require the internet.
I didn't say "for inference", and neither did the person I replied to.
GPT uses the internet to connect to users, but rather more importantly chatGPT in particular has a layer on top of GPT which is trained from human feedback.
Keywords search "RLHF".
That feedback mechanism is, if anything, becoming more detailed as time passes, so I must infer that it's still considered highly important, probably even for the 3.5 model.
That it does any of this is the specific reason for the story you're commenting on, and why putting data into it isn't like putting the same data into e.g. a Google Docs spreadsheet.
This is the issue with a tool so powerful, you can't just tell people not to use it, or to use it responsibly. Because there's too much incentive for them to use it.
If it saves hours of a persons' workday, and they're not seeing any of the harm caused from data leakage, there's no incentive for them to not use it.
Which is why a private option is so critical.
To not fight against human nature, means providing an ability to use the tool in a safe way.
There's a dev here who is using ChatGPT extensively in his work. The rest of the team is just waiting for him to get caught and fired. Sharing company data with unapproved external entities is very definitely a firing offense.
That entirely depends on the code. It's not that the code is special, it's that the code can reveal things that are competitive advantages (future plans, etc.)
But you created a throwaway account specifically to reply in this thread?
Unless your company really has nothing to hide, it's easy to accidentally dump a company secret or an API key in a chat session. Of course if everyone is aware of this and constantly careful then you may be OK.
That's because accounts get shadow banned all the time when people get upset when you point out hard truths.
If you're copy pasting API keys or such into ANYTHING, you probably shouldn't be a programmer to begin with.
It's like people who use root account key/secret credentials in their codebase. It's not AWSs fault you got a large bill or got hacked, its because you're dumb.
I regularly say shit that pisses people off here and I have never been shadow banned. It sounds like your "hard truths" are something other than just "hard truths", and/or you have a persecution complex.
I posted my openAI token into a GitHub issue today thinking I'd just kill it right away, which I did but there was already an email from openAI letting me know that it was noticed that my token had become public and was thus revoked.
If you think random snippets of code are special you really don't understand the business you're writing code for. So no, your code is not special, and pasting code snippets is not transferring business data.
Mine is personal and I just filled out this form with the ID from the docs. Easy, worked fine. Also thanks to the grandparent comment for surfacing this!
My code is "special" in the fact that the act of sharing it can carry civil and criminal liabilities for myself, essentially threatening my well-being and freedom.
Not my place legally or ethically to share code with 3rd parties that I've been paid to read and write.
> If you think your code is special then you're wrong
Your code is not special, but customers data may be. Also, some companies needs to comply to various certifications, and proven leak of source code that was put into some third party tool may be a reason to revoke such certification. Which can cause a serious financial harm to a given company, as it can lead to ex. losing government clients.
Yeah, but the code coming out of ChatGPT is generally not in a state that you want to commit straight into you repo. Making adjustments (and writing the original prompt) is where your expertise comes in.
This could be valid...but with something as powerful as ChatGPT, if it is providing huge benefits for employee productivity, they are unlikely to dump it based off a co-worker's suggestion. Also, unless managing security is within your roles and responsibilities, this approach would likely turn messy from an interpersonal aspect. Lastly, the security issue has already happened, so if this is truly a security concern, the security team should know that (a) something is already out there (b) this could be a widespread problem in the future.
FWIW I don't think the employee should be fired for this or anything, if anything a company could embrace these new technological advances and provide training on using ChatGPT in a more secure manner(ie don't paste your customer's PII into a prompt, etc...).
> if it is providing huge benefits for employee productivity
With this particular employee, using chatGPT has not increased his productivity or the quality of his work by any noticeable degree.
> I don't think the employee should be fired for this or anything
The problem isn't using the technology. The problem is sharing confidential information with an unapproved entity. That is specifically and clearly spelled out as a firing offense, for pretty obvious reasons.
Even if some people feel that it's an overly tight policy, it's a the stated policy and the company has every right to put and enforce whatever rules it wishes about the use of its own data.
1.) It's not at all clear it's nearly as powerful as you think. Certainly in my domain--writing about various topics--it's not.
2.) Of course, you can tell people not to use it. Unlike people at SV companies apparently, people in government and government contractors accept restrictions like not having phones in secure labs all the time. Start firing or even prosecuting people and people will discover very quickly they don't really need some tool.
And, yes, private versions of this sort of thing helps a lot.
Uh, why can't you tell people not to use it...? If security is that important for your company, of course you can tell your employees which tools to use.
A fun fact: in many areas of TSMC, smart phones are banned. No one says "you can't just tell people not to use smart phones."
That’s very common in a lot of secure places. There are tons of government contractors who have to put smartphones away due to the cameras on them. Not to mention the more secure government places.
This does not surprise me at all. What I want to know is how they enforce it.
Unless they have something better than "fear of somebody seeing you using the smartphone", it isn't getting enforced. If they do have something better I want to know what.
No, I'm not joking. One of my high-school classmates works as R&D there. They ask you to pass through a metal detector gate, take away your phone if found, then give your a company phone for emergency call only. It's that strict (at least for R&D. Probably not for management and others).
At Samsung, you typically walk through multiple sets of metal detectors and security before you can actually get into the fab. Anyone working in an office area can have a phone but they do some... stuff to it.
> Anyone working in an office area can have a phone but they do some... stuff to it.
This is what I'm interested in.
I get it, the guys in bunny suits will probably tolerate being groped and wanded every workday for the rest of their career. I have a hard time beliving the scientific staff and executives tolerate that.
Correct, but that level of security is expected from GitHub proper, they have all sorts of independent security reviews for their partners. Does all of that exist for Copilot?
We published an internal policy for AI tools last week. The basic theme is: "We see the value too, but please don't copypasta our intellectual property until we get a chance to stand up something internal."
We've granted some exceptions to the team responsible for determining how to stand up something internal. Lots of shooting in the dark going on here, so I figured we would need some divulgence of our IP against public tools to gain traction.
Even if we had a 100% private ChatGPT instance, it wouldn't fully cover our internal use case.
There is way more context to our business than can fit in 4/8/32k tokens. Even if we could fit the 32k token budget, it would be very expensive to run like this 24/7. Fine-tuning a base model is the only practical/affordable path for us.
Sorry but the whole point is to not use a closed source third party API with a dubious privacy police ran buy a multinational surveillance capitalism megacorporation.
When you use it via the first-party Azure service, the data you input is de-facto guaranteed to not be fed back into the model, so you don't expand your circle of trust by feeding MSFT with your code.
I did not verify this regex, on the very surface scan seems OK:
./main -m ./models/7B/ggml-model-q4_0.bin -t 4 --temp 0.7 --top_k 40 --top_p 0.5 --repeat_last_n 256 --repeat_penalty 1.17647 -n 1024 -p $'Here is a handy short form regex to validate an email address:
'
Here is a handy short form regex to validate an email address:
^([a-zA-Z0-9_.+-]+@[a-zA-Z0-9.-]+\.[a-z\.]{2,6})+$
The first character can be either uppercase or lower case. The second group must contain at least one letter and the third group may contain any number of characters (up to 5). The last part ensures that it ends with @ followed by two more letters separated by dots.
If you want to make sure that your input string contains only valid characters for emails then use this regex instead:
\A[\w.]@[\w.]\.\w{1,4}\z
Badly wrong, in part because your prompt was badly wrong, because there is no short regex to validate an email address. (It needs a way of saying “I can’t complete that because the prompt makes no sense for this reason”.)
What it emitted accepts a large number of invalid addresses (due to things like not checking dot placement, and the inexplicable (…)+ wrapping around the entire thing), and doesn’t accept a large number of valid addresses (some comparatively esoteric, like local parts containing any of !#$%&'*/=?^`{|}~ or IP addresses for the domain name, and some very reasonable, like TLDs of more than six characters, or internationalised TLDs even in Punycode form).
The description it emits does not match the regular expression at all well, either.
The second regex it emits is even worse than the first, unnecessarily uses PCRE-specific syntax, and is given with a nonsensical description. (Note: the asterisks got turned into italics, backslash-escape them here on HN. With this fixed, the regex was \A[\w.]*@[\w.]*\.\w{1,4}\z.)
> on the very surface scan seems OK
And there’s the danger of this stuff. As a subject-matter expert on regex and email, I glanced at the regular expression and was immediately appalled (… quite apart from the whole “here we go again, this is certain to be terrible” cringe on the prompt). But it looks plausible enough if you aren’t.
It is a bit crazy to me someone posts a regex like that without verifying and saying on surface level it looks good, implying the whole thing was useful and a good result.
I said it looks ok, not good. My comment is mostly about me being surprised a valid regex came out. I also asked it to write a regex to parse html which it happily answered. What does gpt4 say about parsing html ;)
But it is either going to be useful or harmful. Harmful if doing the regex validation itself is worse than not doing any validation at all or a very simple validation just checking that there is @ included somewhere.
For comparison GPT-4 provides the following Python regex and then warns that it does not catch all edge cases and that it’s better to use a dedicated library like email-validator:
llama's 7b model internally is, for me, on a totally different level quality-wise. Even when explicitly instructed to not make up stuff and just say 'I don't know', it will still go ahead and ramble and invent things. When I tell it to only use the prompt data it will still invent, or just ignore the prompt data. It's not useful for production (i.e., to be exposed to 'regular' non AI users).
ChatGPT, on the other hand, will listen to those instructions and say when it does not know, and will keep to only the prompt data.
One problem with the current way these models are being trained is that they have no idea of what they're saying. It's just a recursive guess the next word type algorithm. I would not expect the confidence levels of any given fragment, let alone an average, to be a meaningful predictor of truth.
Also completely spitballing, I expect that a big chunk of OpenAI's 'secret sauce' is simple processing layers above and beyond the model. If you input gibberish to llama does it give you an output? If OpenAI is artificially tokenizing inputs (as opposed to just sending inputs straight to the software), it would both dramatically limit the input domain, thus improving output tuning, as well as give "it" the ability to say when it doesn't know something. I put "it" in quotes since that response would not becoming from the LLM, but from the preprocess tokenization system returning an error code in natural language.
I think there's some weak indirect evidence for this in the service itself, since incoherent inputs are instantly rejected, whereas even simple queries take dramatically longer to output even the first word. It's like the input is not even being sent to the LLM software for processing.
I've been debating the idea of building tiers or layers of models to accomplish the same.
It very well could be that this go/no-go pre-processor is simply another ML model trained on a binary classification task. Stack a few of these and you can wind up with some interesting programming models.
This would also explain the ease at which ChatGPT gets rid of escapes/bad prompts - they have an additional layer that assesses whether the question could be, for example, racist, and then spits out a 'Sorry as a language-model I am not trained to answer this kind of question'. No need to retrain the main 14B transformer model.
Not using Chatgpt is easy, but things like GitHub or VSCode with Copilot (which is a special version of GPT3) and in the future Copilot X (gpt4) this will get hard.
One developer opening a folder in VSCode with Copilot enabled aaaaand it’s gone. You never know what part of the folder left your building.
Tech is a big place and not everyone uses GitHub.com - they have an entire self-hosted version for exactly that reason, since many customers have policy or legal requirements – but also consider the distinction between something following its stated policy or doing something else. When you use Windows or VSCode the terms of service do not include sending your personal data to someone else and Microsoft would be in serious legal trouble if they changed that. In contrast, Copilot explicitly does have the right to send some of your code elsewhere so the legal question would come down to whether it reached the point that a judge would no longer consider “snippets of your code” to cover what was sent.
If you use GitHub for sensitive internal stuff you will have a contract with them. This is different from users dumping your data into a service of a vendor you have no business relationship with.
And Windows and VS Code don't upload your data to Microsoft unless you choose to do so.
I think there's more fear of OpenAI leaking data than say, Airtable or Notion or Github or AWS/S3 or Cloudflare or Vercel or some other company that has gobs of a company's data. Microsoft also has gobs of data: anything on Office and Outlook is your company data — but the fear that they'll leak (intentional or accidental) is somehow more contained.
If we want to be intellectually honest with ourselves, we can either be fearful and have a plan to contain data from ALL of these companies, OR, we address the risk of data leaks through bugs as an equal threat. OpenAI uses Azure behind the scenes, so it'll be as solid (or not solid) as most other cloud-based tools IMO.
As for your data training their data: OpenAI is mostly a Microsoft company now. Most companies use Microsoft for documentation, code, communications, etc. If Microsoft wanted to train on your data, they have all the corporate data in the world. They would (or already could!) train on it.
If there's a fear that OpenAI will train their model on your data submitted through their silly textbox toy, but NOT through training on the troves of private corporate data, then that fear is unwarranted too.
This is where OpenAI should just get a "corporate" tier, charge more for it, and is basically make it HIPAA/SOC2/whatever compliant, and basically do that to assuage the fears of corporate customers.
1. Azure has the worst security of the major cloud providers; multiple insanely terrible RCE and open readable DB exposures.
2. Azure infrastructure still likely has far better security/privacy by virtue of all their compliance, (HIPAA, FedRAMP, ISO certifications etc.) than whatever startup-move-fast-ignore-compliance crap OpenAI layers on top of it in their application layer.
> I think there's more fear of OpenAI leaking data than say, Airtable or Notion or Github or AWS/S3 or Cloudflare or Vercel or some other company that has gobs of a company's data.
There is zero fear. OpenAI openly writes that they are going to use ChatGPT chats for training. On the popup modal they show you when you load the page. That is not a fear, that is a promise that they will do leak it whatever you tell them.
If i tell you “please give me a dollar, but i warn you you will never see it again” would you describe your feeling over the transaction as “fearful that the loan won’t be repaid”?
In my experience most corporate employees just take the path of least resistance. It is not uncommon for people to paste non public data into websites just to do json formatting, and paste base64 strings to random websites just to decode them. So just telling people not to do something won't accomplish much. Most corporate employees also somehow think they know better than the policy.
Any company that doesn't want to feed data into ChatGPT should need to proactively block both ChatGPT and any website serving as a wrapper over it.
A while back I got to hear about how the IT team running my then-employer's internal time reporting tool was sending all the usage data through Google Analytics and how neat that was for them to look at :\ .
Given that they use all the labor of the Internet without attribution, we should assume that they will use every additional drop of data we give to them for their own ends.
I’m curious if anyone’s employer has set up their own LLM. My employer has a couple of A100 sitting around which could easily host a couple instance of 65B LLaMA or Alpaca. Convincing upper management to allow me is the hard part.
The 65B quantized model fits in 64GB of RAM, which I already had.
Though RDIMMs on eBay are even cheaper than UDIMMs (just over $1/GB) and Broadwell-era Xeon workstations aren't that expensive if you want to run the unquantized version.
"In one case, an executive cut and pasted the firm's 2023 strategy document into ChatGPT and asked it to create a PowerPoint deck."
There's really not much you can do here. This is complete lack of very basic common sense. Having someone like this in your business, particularly at the executive level, is a liability regardless of ChatGPT.
This is nothing new at all. How many people have Grammarly plugins installed? They are advertising aggressively, so I'd think it is the new hotness. Don't tell me Grammarly is not hoovering up all of the Slack, Word, Docs, and Gmail data that everyone sends it, and holding on for some future purpose. We'll see.
Grammarly is an OpSec nightmare that’s somehow managed to slip under most people’s radar. I know folks who won’t use the Okta browser extension because of the extensive permissions it asks for, but will happily use Grammarly on everything.
Last time I looked (maybe things have improved…?) Grammarly would automatically attach itself to any text box you interacted with, and immediately send all of your content to their servers for processing. How this software gets past IT departments is a mystery to me.
I noticed this too, this is why I'm working on a startup that lets you train your data internally through a self-hosted OSS LLM product. It's going to start off with just reading emails for now until we can integrate with other products too, like spreadsheets, Airtable, databases, etc. Basically an OSS version of OpenAI ChatGPT plugins, I suppose. If you're interested, message my at my email in my profile.
> These are the same people that use only Chrome while being logged in to their Google account
This situation is much dumber than that. ChatGPT is very clear that you shouldn't give it private data and that anything you type into it can/will be used for training.
Google is nowhere near that level of transparency.
It was a lack of vigilance among the Trojans, actually. The Greeks did the burning an pillaging. So uh OpenAI is the Greeks, ChatGPT is the horse and Microsoft is king Menelaos or something. Achilles is dead, but he did make TempleOS.
Google takes your data and sells it. Literally making your data available to the highest bidder. Is OpenAI doing that? If Google existed in its current form during the early internet it would be classified in the same category as Bonzai Buddy. Spyware. That is what Google is. So I can very reasonably understand why people would trust OpenAI with data they wouldn't trust Google with. OpenAI hasn't spit in the face of its users yet.
Because they are completely different business models. If OpenAI decides to become an advertising behemoth then I would show concern. Right now they use your data for training (when they use it).
OpenAI has already demonstrated that they're all in for maximizing profit. They may not be advertisers, but advertisers aren't the only sorts of companies that make bank by selling personal data.
I see no reason to think OpenAI would leave that money on the table.
OpenAI is selling others data in their model responses. Selling others data is their main business model.
If it uses user data to train their models other users could ask "Show me the code for gmail spam filters", and if it was trained on engineers refactoring that spam filter in ChatGPT chances are it would give you the code. If that doesn't count as "selling user data" I don't know what is. They not only sell it, they nicely package and rewrite it to make it easy to avoid copyright claims!
Not really. Even the most evil Google one can imagine would realise "your data" is the most valuable thing they possess, selling it would be bad for business. They're selling ads to the highest bidder who's looking for someone with a profile based on your data, but not your data itself.
> I can very reasonably understand why people would trust OpenAI with data they wouldn't trust Google with. OpenAI hasn't spit in the face of its users yet.
In other words, having been burnt once by touching a flame, the conclusion these people draw is that the problem was with that particular flame and they're fine with reaching for a different one?
> Google takes your data and sells it. Literally making your data available to the highest bidder.
But it doesn't, does it? It sells the fact that it knows everything about everyone and can get any ad to the perfect people for it. It's not going on the open market and telling people I regularly buy 12 lbs of marshmallow fluff and then use it in videos I keep on my google drive.
> Google takes your data and sells it. Literally making your data available to the highest bidder.
No, Google takes money to present ads to people of different demographics, and uses your data to do that. It doesn’t sell your data, which is, in fact their competitive edge in ads – selling your data would be selling the cow when they’d prefer to sell the milk.
ChatGPT Business Edition seems pretty obvious and I'd surprised if OpenAI isn't already working on it. Separate models for each customer, data silos and protection. The infra is already there on Azure.
This is scary, but it doesn't surprise me even in the slightest. ChatGPT is useful for so many things that it's extremely tempting to convince yourself that you should trust it.
For example, I was having some issues with my LTO-6 drive recently, and I had to finagle through a bunch of arcane server logs to diagnose it. I had the idea of simply copypasting the logs into ChatGPT and having it look at them, and it quickly summarized the logs and told me what things to look for. It didn't directly solve the problem, but it made the logs 100x more digestible and I was able to figure out my problem. It made a problem that probably would have taken 2-3 hours of Googling take about 20 minutes of finagling.
I'm not doing anything terribly interesting or proprietary on my home server, so I didn't really have any reservations sharing dmesg logs with it, but obviously that might not be the case in a company. Server logs can often have a ton of data that could be useful for a competitor (whether it should be there or not), and someone not paying attention to what they're pasting into ChatGPT could easily expose that data.
354 comments
[ 2.6 ms ] story [ 433 ms ] threadGmail has a vested interested in keeping any knowledge it gains about you secret - it's competitive advantage is knowing more about you than anyone else does.
ChatGPT's strength is its ability to clearly communicate the knowledge it has (including training data it gains from people it interacts with) to give you good responses.
Most corps, companies store a lot of internal data with corps like Google, Microsoft, Amazon and others.
While this is not a new problem -- employees share sensitive data with Google all the time -- the data leakage will be more clear than ever. With ads-based tracking and Google search, the leakage was very indirect. With generative AI, it can literally regurgitate memorized documents.
The security risk goes beyond data exfiltration. Folks are already trying to teach the AI incorrect information by spamming it with something like 2+2 = 5.
Data exfiltration + incorrect data injection are super underrated risks to mass adoption of generative AI tech in the B2B world...
- there’s no way they’re manually scrubbing out sensitive data so its bound to spill out from the training data when prompting the model
- OpenAI is openly storing all this data they’re collecting to the extent that they’ve had several leaks now where people can see others’ conversations and data. We are one step away if it hasn’t already happened from an exploit of their systems (that likely weren’t built with security as the top priority as opposed to scale and performance) that could leak a monumental amount of data from users.
In the most innocent case they could leak the personal info of naive users. But largely if Linkedin is any indication, the business world is filled with dopes who genuinely believe the AI is free thinking and better than their employees. For every org that restricts ChatGPT use, there are fifty others that don’t, most of which have at least one of said dopes who are ready to upload confidential data at a moments notice.
Wouldn’t even put it past military personnel putting S/TS information into it at this point. OpenAI should include more brazen warnings against providing this type of data if they want to keep up this facade of “we can’t release it because ethics” because cybersecurity is a much more real liability than a supervised LM turning into terminator.
I don't think that is true at all. Do you have an example of a significant project being duplicated in days, or even months, with ANY of these tools?
By significant, I mean something on the order of pandas which you claimed.
Prompt 1: What would be an amazing tech project that would make me rich?
P2: Produce an excellent design for that project. Should be elegant and use microservices and scale to billions of users.
P3: Write all the code for this design.
P4: Tell me how to test and deploy all that code.
P5: How to sell all this for billions?
Let me hand you a mirror: You're absolutely and completely wrong
I am having trouble parsing this statement. You're saying a person equipped with chatGPT trained on data prior to December 2007 (the month before the initial pandas release) could have put together the entire pandas library in a couple of days?
That seems obviously wrong, starting with the fact that one would need to know "what" to build in the first place. If you're saying that chatGPT in 2023 can spit out pandas library source code when asked for it directly, that's obvious.
Somewhere between the impossible statement and the obvious statement I made above, there must be something interesting that you were trying to claim. What was it?
Then it should be easy to provide some?
Source: worked there after the meltdown.
Where can I, a random employee, get that? I know how to get ChatGPT.
In that case, you wouldn't know to block them until it was too late.
Ultimately either you must watch/block all outgoing traffic, or you must train your people so thoroughly that they become suspicious of everything. Sadly, being paranoid is probably the most economical attitude these days if IP and company secrets have any value.
Well, GP was referring to blocking ChatGPT as a federal contractor. I suspect that as a federal contractor, they are also vetting other people that they share data with, not just blocking ChatGPT as a one-off thing. I mean, generic federal data isn’t as tightly regulated as, say, HIPAA PHI (having spent quite a lot of time working for a place that handles both), but there are externally-imposed rules and consequences, unlike simple internal-proprietary data.
It's a stupid and patronizing position, but corporate IT are sadly incentivised to be stupid and patronizing.
By the way, OpenAI says they wont use data submitted through its API for model training - https://techcrunch.com/2023/03/01/addressing-criticism-opena...
Companies can use Azure OpenAI Services to get around this -- there's data privacy, encryption, SLAs even. The problem is it's very hard to get access to (right now).
Point is, for a meaningful subset of high-value use-cases you don't need to move your important private stuff across any trust boundaries, and it still can be pretty helpful...so just calling that out in case that's useful to anyone...
The fact that the can't do this is the whole reason they have to use ChatGPT.
Besides, it's not like it puts out great code (or even always working code), so I still have to read everything and debug it. And sometimes it writes code that is just fine and fit for purpose and horrendously ugly, so I still have to scrap everything and do it myself.
(And then sometimes I spend 10x as long doing that, because it turns out it's also just plain good fun to grow an aesthetic corner of the code just for the hell of it, too — as long as I don't have to.)
And even after all that extra time is factored back in: it's still way faster and more fun than the before-times. I'm actually enjoying building things again.
And I agree it’s fun. Maybe it’s the simulated social interaction without consequences. I can be completely honest with my robot friend about the shitty or awesome code and no one’s feelings are going to get hurt. ChatGPT will just keep trying to be helpful.
If you know exactly what you want to ask of it, and have the ability to evaluate and verify what it produces, it's incredible what you can get out of it. Sure it's nothing I couldn't have done otherwise... eventually. The productivity it enables is worth every cent.
Easily the best $20 I've spent in ages, they should have run with the initial idea of charging $42.
But holy moly anyone putting confidential information into it needs to stop
If people are using the free version of chatGPT then it’s unlikely there is a contract between the companies and more likely just a terms of use applied by chatGPT and ignored by the users.
to take a stab at your question, though : my cell phone doesn't learn to get better by absorbing my telecommunications; it's just used as a means to spy on my personal life by The Powers That Be. The primary purpose of my cell phone is for the conveyance of telecommunications.
chatGPT hordes data for training and self-improvement in its' current state. It's whole modus operandi involves the capture of data, rather than it being used for that tangentially. It could not meaningfully exist without training on something, and at this stage of the game it's the trend to self-train with user data.
Until that trend changes people should probably be a bit more suspect about what kind of stuff gets thrown into the training bin.
What is different about ChatGPT (if anything)?
ChatGPT seems to be used more like a fast stackoverflow, except people aren't thinking of it like a forum where others will see their question so they aren't as cautious. We're just waiting for some company's data to show up remixed into an answer for someone else and then plastered all over the internet for the infosec lulz of the week.
For every company like yours there are hundreds that don't. People use free gmail address for sensitive company stuff, paste random things in random pastebins, put their private keys in public repos, etc.
Yes, data leaks from OpenAI are bound to happen (again), and they should beef up their security practices.
But thinking people are using only ChatGPT in an insecure way vastly overestimates their security practices elsewhere.
The solution is education, not avoiding new tools.
And, IIRC, pay-as-you-go API requests are explicitly not used for training data. I'm sad GPT-4 isn't there yet - except for those who won the waitlist lottery.
If people were smart and performed according to best practices, articles like this one would not be necessary.
I am honestly confused how people can use this thing with the interface OpenAI runs. The app has been near-unusable for me, for months, on every device I tried it on.
And what sort of understanding do you have with the alternative frontends/integrations about how they handle your API keys and data? This might be a better solution for a variety of reasons but it doesn't automatically mean your data is being handled any better or worse than by openai.com
Some of those are document tools working on language / knowledge. Others are infrastructure, working on ... whatever your infra does, and your infra manages your data (knowledge).
If you read their data policies, you'll find they are not the same.
I am unsure if the so called AI can think in models but so far, not but still an impressive assisting tool if you take care of its limitations.
Another point where it lacks is in logic, my daughter has a lot of fun with the book "what is the name of this book?" but she was struggling with the "map of baal" explanation, her the answer was a certain map, yet the book had another answer, I had a third one as I interpreted a proposition. I never got an answer without a contradiction in chatgpt reasoning, and the book had been mistranslated to French so one of its propositions was changed (C, both A and B were knaves) but not the answer.
> I am unsure if the so called AI can think in models but so far, not but still an impressive assisting tool if you take care of its limitations.
I don't know. I'm using it for exactly that ("here's a problem, come up with a data model") and it gives a great starting point.[0]
Not perfect, but after that it's easy to tweak it the old-fashioned way.
I find its data modelling capabilities (in the domain I'm using it for - API services) to be rougly on par with a mid-level developer (for a handwavy definition of "midlevel").
[0] https://apibakery.com/demo/ai/
X and Y are not alike, and should not be compared. X is a benefit to you(r employer), whereas Y is a risk to the customer who has entrusted you with their data.
I do and if it could be leaked through ChatGPT I would have it blocked.
Risk isn't a single dimension, it's a combination of exposure (chance of happening) and impact (how much will you lose)
Nothing you’ve said negates anything OP said. It’s simply an elaboration wrapped in elitism.
And yes, if the bypassing the block is combined with disciplinary action, it does work. It’s not worth getting fired over. This is likely what heavily regulated industries like financial services and defense are doing.
It's a fairly standard practice. I wouldn't associate it with overreaching surveillance.
Hey, they need someone to proofread their War Thunder forum posts to make sure they're using correct spelling and grammar when leaking classified info. ;-)
(Ref if you don't get the joke: https://taskandpurpose.com/news/war-thunder-forum-military-t...)
As the leaks ware more inadvertent.
I was under the impression OpenAI weren't using questions as training data for future models. I recall Sam Altman saying they delete questions after 1 month, but I can't locate the source for that.
This is still a serious data loss risk.
MS also "helpfully" asks you if you want to use that enhanced grammar check in MS Word(as far as I have seen, might be there in other office products too). I cannot imagine sending all my documents to MS. But I am not sure most users will realize what is happening.
All these companies offer helpful services but are hovering up data and no one knows the consequences yet. It feels like ChatGPT is just one symptom of a bigger problem.
I think MS Edge is getting even worse about this, with the big fucking Bing icon in the corner and making it impossibly hard to get rid of it.
They will go to OpenAI directly and do stuff because "they want it now" and they don't understand why not.
Microsoft is already building it into Office 365 with the same enterprise-grade agreements so it might get easier that way.
Do you have a source for this? I know some people have claimed to see others' data, but I haven't seen any evidence that that's what's actually being seen, vs LLM hallucinations. OpenAI claims, and I can't imagine they're lying, that the training data is fixed and ends in 2021, so I don't see how it would be possible for user prompts to be leaking into output, absent a massive and very unlikely bug (compared to the much more likely AI hallucination explanation).
Some kind of concurrency bug in a library they were using to retrieve cached data from Redis led to this leak.
> We took ChatGPT offline earlier this week due to a bug in an open-source library which allowed some users to see titles from another active user’s chat history. It’s also possible that the first message of a newly-created conversation was visible in someone else’s chat history if both users were active around the same time.
Do you also block pastebin? Anything else that has a web form? How is ChatGPT special compared to any other service on the Internet where people can paste data in a form?
I mean... I see the problem, but I think one needs to realize that it's a far more generic problem that has basically nothing to do with ChatGPT and AI. If people paste confidential data into random webpages that's of course bad. But if you block ChatGPT because you fear that, it means you expect that people might do that. And then your problem is not ChatGPT, but lack of awareness what is confidential data and what to do with it.
I don't think the idea that you can give people access to the www and at the same time preventing them from putting things in forms can be done. That's simply not how it works. And if you're blocking access to a few services where they might do that, well, they have a million others, and you're deceiving yourself that you've done something.
pastebin and indeed most things that has some sort of public webform is blocked in all the companies I have worked with.
It is probably a losing battle though, as it is very hard to block everything without default deny.
Paradoxically, maybe GPT could be used to veto websites on first access :)
Search engines too? And these days, that means web browsers, because the (IMHO stupid) idea of combining address and search bars into one means everything you type while trying to open a website gets leaked to some party (most likely Google).
> You can request to opt out of having your content used to improve our services at any time by filling out this form (https://docs.google.com/forms/d/1t2y-arKhcjlKc1I5ohl9Gb16t6S...). This opt out will apply on a going-forward basis only.
It goes to a google form, which is I guess better then them building their own survey platform from scratch that may have more vulnerabilities.
And IP pirates banned from using the internet. Actually, that one I remember I voted against my local MP after they passed a law to make that the norm.
We don't yet have a social, let alone legal, norm for antisocial use of LLMs; even with social media, government rules are playing catch-up with terms of service, and Facebook is old enough that if it was human it could now vote.
So, yes, likewise computers/internet/social media, being banned from an LLM if it's a monopoly is going to seriously harm people.
But that is likely to be a big "if". The architecture and the the core training data for GPT-3 isn't a secret, and that's already pretty impressive even if lesser than 3.5 and 4.
Funny. This nobel prize winner raises an interesting question:
If your AI is so great at coding, why is your software so buggy?: https://paulromer.net/openai-bug/
And the thumbs up/down are there on the chat interface because it's partly trained by reinforcement from human feedback.
GPT uses the internet to connect to users, but rather more importantly chatGPT in particular has a layer on top of GPT which is trained from human feedback.
Keywords search "RLHF".
That feedback mechanism is, if anything, becoming more detailed as time passes, so I must infer that it's still considered highly important, probably even for the 3.5 model.
That it does any of this is the specific reason for the story you're commenting on, and why putting data into it isn't like putting the same data into e.g. a Google Docs spreadsheet.
Which is why a private option is so critical. To not fight against human nature, means providing an ability to use the tool in a safe way.
Unless your company really has nothing to hide, it's easy to accidentally dump a company secret or an API key in a chat session. Of course if everyone is aware of this and constantly careful then you may be OK.
If you're copy pasting API keys or such into ANYTHING, you probably shouldn't be a programmer to begin with.
It's like people who use root account key/secret credentials in their codebase. It's not AWSs fault you got a large bill or got hacked, its because you're dumb.
And if you think that there is no special code then you're wrong.
Lots of code expresses buisness strategy that is a competitive advantage/ sensitive.
My org has done a risk assessment and accepted the risk of using such tools; arguing there is no risk is short sighted.
It literally is exactly that. You don't think the code a business creates is "business data"?
Not my place legally or ethically to share code with 3rd parties that I've been paid to read and write.
Your code is not special, but customers data may be. Also, some companies needs to comply to various certifications, and proven leak of source code that was put into some third party tool may be a reason to revoke such certification. Which can cause a serious financial harm to a given company, as it can lead to ex. losing government clients.
This is just the tip of the iceberg.
Uploading code to ChatGPT can be done by trainees.
Productivity isn't everything.
FWIW I don't think the employee should be fired for this or anything, if anything a company could embrace these new technological advances and provide training on using ChatGPT in a more secure manner(ie don't paste your customer's PII into a prompt, etc...).
With this particular employee, using chatGPT has not increased his productivity or the quality of his work by any noticeable degree.
> I don't think the employee should be fired for this or anything
The problem isn't using the technology. The problem is sharing confidential information with an unapproved entity. That is specifically and clearly spelled out as a firing offense, for pretty obvious reasons.
Even if some people feel that it's an overly tight policy, it's a the stated policy and the company has every right to put and enforce whatever rules it wishes about the use of its own data.
2.) Of course, you can tell people not to use it. Unlike people at SV companies apparently, people in government and government contractors accept restrictions like not having phones in secure labs all the time. Start firing or even prosecuting people and people will discover very quickly they don't really need some tool.
And, yes, private versions of this sort of thing helps a lot.
Uh, why can't you tell people not to use it...? If security is that important for your company, of course you can tell your employees which tools to use.
A fun fact: in many areas of TSMC, smart phones are banned. No one says "you can't just tell people not to use smart phones."
This does not surprise me at all. What I want to know is how they enforce it.
Unless they have something better than "fear of somebody seeing you using the smartphone", it isn't getting enforced. If they do have something better I want to know what.
No, I'm not joking. One of my high-school classmates works as R&D there. They ask you to pass through a metal detector gate, take away your phone if found, then give your a company phone for emergency call only. It's that strict (at least for R&D. Probably not for management and others).
This is what I'm interested in.
I get it, the guys in bunny suits will probably tolerate being groped and wanded every workday for the rest of their career. I have a hard time beliving the scientific staff and executives tolerate that.
They put a special sticker on all of your cameras and inspect if it is still there on the exit.
And Github Copilot will gather that tiny amount of data to who knows where.
Hahahahahahaha
We've granted some exceptions to the team responsible for determining how to stand up something internal. Lots of shooting in the dark going on here, so I figured we would need some divulgence of our IP against public tools to gain traction.
There is way more context to our business than can fit in 4/8/32k tokens. Even if we could fit the 32k token budget, it would be very expensive to run like this 24/7. Fine-tuning a base model is the only practical/affordable path for us.
> ChatGPT is now available in Azure OpenAI Service
https://azure.microsoft.com/en-us/blog/chatgpt-is-now-availa...
That's all I use ChatGPT for. I don't need it to be able to write poetry.
./main -m ./models/7B/ggml-model-q4_0.bin -t 4 --temp 0.7 --top_k 40 --top_p 0.5 --repeat_last_n 256 --repeat_penalty 1.17647 -n 1024 -p $'Here is a handy short form regex to validate an email address: '
Here is a handy short form regex to validate an email address: ^([a-zA-Z0-9_.+-]+@[a-zA-Z0-9.-]+\.[a-z\.]{2,6})+$ The first character can be either uppercase or lower case. The second group must contain at least one letter and the third group may contain any number of characters (up to 5). The last part ensures that it ends with @ followed by two more letters separated by dots. If you want to make sure that your input string contains only valid characters for emails then use this regex instead: \A[\w.]@[\w.]\.\w{1,4}\z
And the second one is confusing me. It seems to be matching a single character only for the initial portion?
Neither of them seem good, and especially the last one.
And the way it describes both seems off as well. I would have to say it brings more harm than good based on that.
What it emitted accepts a large number of invalid addresses (due to things like not checking dot placement, and the inexplicable (…)+ wrapping around the entire thing), and doesn’t accept a large number of valid addresses (some comparatively esoteric, like local parts containing any of !#$%&'*/=?^`{|}~ or IP addresses for the domain name, and some very reasonable, like TLDs of more than six characters, or internationalised TLDs even in Punycode form).
The description it emits does not match the regular expression at all well, either.
The second regex it emits is even worse than the first, unnecessarily uses PCRE-specific syntax, and is given with a nonsensical description. (Note: the asterisks got turned into italics, backslash-escape them here on HN. With this fixed, the regex was \A[\w.]*@[\w.]*\.\w{1,4}\z.)
> on the very surface scan seems OK
And there’s the danger of this stuff. As a subject-matter expert on regex and email, I glanced at the regular expression and was immediately appalled (… quite apart from the whole “here we go again, this is certain to be terrible” cringe on the prompt). But it looks plausible enough if you aren’t.
email_pattern = r"^(?=.{1,256})(?=.{1,64}@.{1,255}$)(?=\S)(?:(?!@)[\w&'+._%-]+(?:(?<!\\)[,;])?)(?<=\S)@((?=\S)(?!-)[A-Za-z0-9-]{1,63}(?<!-)\.?)+[A-Za-z]{2,19}(?<=\S)$"
Also completely spitballing, I expect that a big chunk of OpenAI's 'secret sauce' is simple processing layers above and beyond the model. If you input gibberish to llama does it give you an output? If OpenAI is artificially tokenizing inputs (as opposed to just sending inputs straight to the software), it would both dramatically limit the input domain, thus improving output tuning, as well as give "it" the ability to say when it doesn't know something. I put "it" in quotes since that response would not becoming from the LLM, but from the preprocess tokenization system returning an error code in natural language.
I think there's some weak indirect evidence for this in the service itself, since incoherent inputs are instantly rejected, whereas even simple queries take dramatically longer to output even the first word. It's like the input is not even being sent to the LLM software for processing.
I've been debating the idea of building tiers or layers of models to accomplish the same.
It very well could be that this go/no-go pre-processor is simply another ML model trained on a binary classification task. Stack a few of these and you can wind up with some interesting programming models.
One developer opening a folder in VSCode with Copilot enabled aaaaand it’s gone. You never know what part of the folder left your building.
You use Windows, VSCode, etc, all of this has access to your code.
And Windows and VS Code don't upload your data to Microsoft unless you choose to do so.
The problem is the verification of this of course.
If we want to be intellectually honest with ourselves, we can either be fearful and have a plan to contain data from ALL of these companies, OR, we address the risk of data leaks through bugs as an equal threat. OpenAI uses Azure behind the scenes, so it'll be as solid (or not solid) as most other cloud-based tools IMO.
As for your data training their data: OpenAI is mostly a Microsoft company now. Most companies use Microsoft for documentation, code, communications, etc. If Microsoft wanted to train on your data, they have all the corporate data in the world. They would (or already could!) train on it.
If there's a fear that OpenAI will train their model on your data submitted through their silly textbox toy, but NOT through training on the troves of private corporate data, then that fear is unwarranted too.
This is where OpenAI should just get a "corporate" tier, charge more for it, and is basically make it HIPAA/SOC2/whatever compliant, and basically do that to assuage the fears of corporate customers.
2. Azure infrastructure still likely has far better security/privacy by virtue of all their compliance, (HIPAA, FedRAMP, ISO certifications etc.) than whatever startup-move-fast-ignore-compliance crap OpenAI layers on top of it in their application layer.
There is zero fear. OpenAI openly writes that they are going to use ChatGPT chats for training. On the popup modal they show you when you load the page. That is not a fear, that is a promise that they will do leak it whatever you tell them.
If i tell you “please give me a dollar, but i warn you you will never see it again” would you describe your feeling over the transaction as “fearful that the loan won’t be repaid”?
Any company that doesn't want to feed data into ChatGPT should need to proactively block both ChatGPT and any website serving as a wrapper over it.
I agree this would be a good move, but it's going to be harder and harder to do that definitively.
I shudder to think what they are doing now.
https://news.ycombinator.com/item?id=35299695
Though RDIMMs on eBay are even cheaper than UDIMMs (just over $1/GB) and Broadwell-era Xeon workstations aren't that expensive if you want to run the unquantized version.
There's really not much you can do here. This is complete lack of very basic common sense. Having someone like this in your business, particularly at the executive level, is a liability regardless of ChatGPT.
Yammer pre-Microsoft and nowadays Blind — lots of “insider” information seemingly posted.
As usage goes up the target size, and opportunity cost, both go up.
Last time I looked (maybe things have improved…?) Grammarly would automatically attach itself to any text box you interacted with, and immediately send all of your content to their servers for processing. How this software gets past IT departments is a mystery to me.
No way I’m giving Google any of my data! I will use 5 different browsers in incognito mode and never log in.
To ->
Sure I will login with my name and email and feed you as much of my most personal thoughts and data as I can dear ChatGPT!
This situation is much dumber than that. ChatGPT is very clear that you shouldn't give it private data and that anything you type into it can/will be used for training.
Google is nowhere near that level of transparency.
Because both types of people have always existed. Heck, lack of vigilance among the ancient Greeks is what put the Trojan in Trojan horse.
Sometimes couriosity beats caution.
Even if they are not doing it now(?), what makes you think that they will not do so in the future? It's not like your data has an expiration date.
I see no reason to think OpenAI would leave that money on the table.
If it uses user data to train their models other users could ask "Show me the code for gmail spam filters", and if it was trained on engineers refactoring that spam filter in ChatGPT chances are it would give you the code. If that doesn't count as "selling user data" I don't know what is. They not only sell it, they nicely package and rewrite it to make it easy to avoid copyright claims!
In other words, having been burnt once by touching a flame, the conclusion these people draw is that the problem was with that particular flame and they're fine with reaching for a different one?
But it doesn't, does it? It sells the fact that it knows everything about everyone and can get any ad to the perfect people for it. It's not going on the open market and telling people I regularly buy 12 lbs of marshmallow fluff and then use it in videos I keep on my google drive.
No, Google takes money to present ads to people of different demographics, and uses your data to do that. It doesn’t sell your data, which is, in fact their competitive edge in ads – selling your data would be selling the cow when they’d prefer to sell the milk.
https://learn.microsoft.com/en-us/azure/cognitive-services/o...
If it works well it will be a big deal.
For example, I was having some issues with my LTO-6 drive recently, and I had to finagle through a bunch of arcane server logs to diagnose it. I had the idea of simply copypasting the logs into ChatGPT and having it look at them, and it quickly summarized the logs and told me what things to look for. It didn't directly solve the problem, but it made the logs 100x more digestible and I was able to figure out my problem. It made a problem that probably would have taken 2-3 hours of Googling take about 20 minutes of finagling.
I'm not doing anything terribly interesting or proprietary on my home server, so I didn't really have any reservations sharing dmesg logs with it, but obviously that might not be the case in a company. Server logs can often have a ton of data that could be useful for a competitor (whether it should be there or not), and someone not paying attention to what they're pasting into ChatGPT could easily expose that data.