Honestly, it hasn't. Last night I was trying to use it/Bing to write a simple API request based on a JSON schema, and it failed miserably. Nothing super complicated either, it was just a poor fit. Both GPT3 and GPT4 repeatedly generated nonworking code, which surprised (and frustrated) me.
I've had astoundingly inconsistent results myself.
Sometimes it'll write really elegant code that just nails the request. Other times it insists the task is impossible ("too complex"). Days later it'll do it without fuss.
I suspect there's some sort of model shell game going on behind the scenes. The best results I get are always outside of business hours.
> Other times it insists the task is impossible ("too complex").
This is the lobotomization. It's interesting to watch the interviews with some red teamers and bing integration devs who had access before they did this to it. They are fully Blake Lemoine'd.
That i'm finally breaking my "just google it" habits for technical stuff.
Had it write non blocking micro-controller code for a project (about 75% of the output was put into production) it helped making variable names and general structure. The first output was bad because it blocked the button read while "breathing" and led but then I told it: "non blocking code please".
I asked it to draft a legal document based on some bullet points and it created it, in a different language, in keeping with the legal norms of that country. I sent it to a local lawyer for review and they only came back with very minor things.
Did the reverse with some regulatory documents too, asking it to summarise in vernacular then asking specific questions to better wrap my head around the docs. I wouldn't trust it fully for this of course, more like a supplementary learning aid.
I keep hitting limits with context size though. Eg. When coding it becomes cumbersome once your project hits a certain complexity. You have to carefully construct prompts to get useful outputs. Being able to add the entire source code into a prompt to have more contextual responses would really increase utility.
Sure, it was to specifically mention the account numbers and banks involved in some edge cases instead of saying something more general like "transfer to the bank account of person x".
ChatGPT4 blew my mind when I asked it some specific details about my PhD thesis. It was a relative simple question, but you needed to know some domain knowledge to give the correct answer. ChatGPT not only gave a wrong answer, it completely made up a story was beyond repair. It is concerning (blew my mind) how confident ChatGPT can provide a completely wrong answer.
Sometimes either just relevant or actual citations, I've seen both. If you ask for specific citations for a specific phrase it's basically a coin flip what you get, what you get could be fake citations
A recent thesis is a single datapoint at the edge of human knowledge. These bots will necessarily be better at parsing information with more interconnectedness closer to the core of existing human knowledge.
This week between ChatGPT and Phind I asked an AI four questions. Two were about SMT-LIB, one was about homomorphic encryption, and one was about software-defined networking.
Both ChatGPT and Phind insisted on giving me example SMT problems and then incorrect solutions to them. The HE answer was so vague as to be useless, and I don’t actually know enough about SDN to verify the last one but it seemed useful.
Overall it didn’t give me a lot of confidence in the two systems.
I see a lot of replies here talking about asking GPT questions in a domain an individual is an expert in, and being disappointed in the results. I think this is missing the point a bit. The impressive part of GPT isn’t that it’s an expert in anything, but that it’s better than an average human on many topics. For example - someone brought up GPT not comprehending homomorphic encryption fully. Well, I have a MS in CS and have been professionally developing software for 15 years, and I’ve even READ articles on homomorphic encryption multiple times, but I can’t articulate what it is very well. I just remember it’s inefficient and infeasible at this time.
The thing that is mind blowing to me is that I can talk with GPT about basically any topic and it can help get me in the right direction, I can talk with it to get clarity on anything that’s confusing, and it’s like a way to collaborate on thought without a human. That’s mindblowing to me!
I describe GPT as really good at bridging contexts. As you point out, it does a great job pointing you in the right direction on pretty much any topic. I was using it with my 7 yo daughter and asked it to explain calculus in a way a 7 yo could understand and it did a pretty good job.
If ChatGPT is impressive when discussing any subject, unless you’re an expert, it should be a journalist.
Okay, I know that sounds like a sleight but I mean it in reverse: we give it a hard time for failing at something humans can’t do either: be an expert at everything. Journalists do a great job despite regularly driving SMEs nuts.
I use it not for querying immutable truths but for exploring topics. It trivializes what used to be paralyzing for me: “where do I begin if I want to learn about X?”
GPT-4 has once solved a bug in an hour that took me a day.
It wasn't because it was a superintelligent entity, but because it was an intelligent scientific logbook.
It would break down possible root causes of the bug. I could add more if it misses any. It would give me the simplest possible solution to test each one. Sometimes it's not right and I have a simpler test.
But it'll track which ones we have tried so we don't go in circles with debugging. It's really a superior rubber duck + notepad.
Who is wiser? One who doesn't know much, but knows what he doesn't know and points this out when asked. Or the one who knows a lot, but doesn't know what he doesn't know, and (confidently) makes up something when he doesn't know? (also see Apology from Plato)
I'm not disappointed that GPT doesn't know the answers to questions in my domain of expertise. I'm disappointed that GPT doesn't know that it isn't an expert in my domain, and makes up a confident but wrong answers.
As a side project I've been writing a tool that drives GPT4 for coding purposes. It finds FIXME comments and equips the LLM with a set of high level shell commands it can use to explore the codebase and edit the files. This works well and I find it superior to the (often semi-broken) chat UI that OpenAI provide. There's no need for copy paste, and it can be given instructions like
// FIXME: Implement this class paying attention to the unit tests
at which point it will go look up the unit test class, read it, understand what the code needs to do and proceed to edit the necessary algorithms into the code. The tool is a mix of response parsing/execution and prompt building, so it can do builds/tests/fix cycles. At the end you get a git branch with the work it's done.
By this point I'm kinda developing an intuition for what GPT4 can and can't do, but whilst pushing against what it can do I still often find myself being impressed. Maybe not "mind blown" anymore because it's amazing how quickly you can get used to this stuff, but still. Not only does it do an excellent job of figuring out what to do when the instructions are clear enough, but it also has commands for adding library dependencies and will use relevant open source libs to make its job easier.
The hard part about using this sort of tool is that you can quickly become bottlenecked on figuring out what it is you actually want. It's easy to run ahead of yourself and end up with a mess. This sort of very rapid "painting with code" feel is a bit new; probably the slowness of typing allows your subconscious to think ahead a bit when doing programming normally. Also, I'm still teaching it how to explore the codebase efficiently so things that require a lot of context aren't quite there yet. I've got a whole plan for how to make that work better though.
I expect at some point soon some big company or some $100M "seed round" startup will come along and do this better, but I didn't want to wait.
I've been automating some of our team's most menial tasks and since we're in a corporate environment we don't have control over what is and isn't installed on our servers, so using bash is the sane default choice.
None of the scripts it produced are incredible or whatever (the tasks being more menial than lohic-based), but considering the intricacies of bash and how often I just faceplant into them as an experienced sysadmin, if feel as though the ability to just say "write a script that compares a list of agents against every manager and outputs the ones not found anywhere" and have it output basically a working script first try is impressive.
Here’s a paper that constitutes the state of the art in this domain in 2018. [1] They trained a model on a corpus, and the results were about 50% accurately executed commands. That’s only single commands (not complete scripts) and you couldn’t go back and forth with NL2Bash to fix mistakes.
I think this is a great example of the strides of LLMs. It’s absolutely leaps and bounds ahead of what was unwieldily cutting edge just 5 years ago.
I recently used it (in combination with copilot) to help me implement weighted, blended, order-independent transparency for a network visualization project I'm working on... in just one long evening.
Only to realize in the process that all I really needed was to modify the shader program's depth checking behavior. Even just disabling it altogether looks fine and makes everything way faster. Never in my wildest imagination a year ago would I even dream of implementing WBOIT in one night and then just leaving it in a branch for a one line change.
In the process I realized a little trick on my own, using what I learned, that modifying the depth values in the fragment shader could allow selective alpha-blending/occlusion. Basically the idea was to put stuff I wanted to blend order-independently all at an equal z position and then set the depth function to "less than or equal" instead of "less than". I used the occluding geometry's depth buffer as a source to modify the translucent stuff's depth buffer in the fragment shader (with a slight offset to get it behind the occluding stuff).
There's absolutely no way I'd have had the guts to try all that in one night without the robot tools.
That’s what I’ve been using it for - unblocking my own mind. It almost never had the answer for me. But it breaks down my mental barrier. I’ve achieved more in the last three months than three years for this reason alone.
Compare it to reading stackoverflow or a tutorial on something that you can't quite do yet - the answer on SO might be wrong, or it might work in a different context from yours, or might work but not be the optimal solution; the tutorial might be out of date or it might have been incorrect from the start. But SO and tutorials are still very useful if you're smart about hwo you use them.
Same thing with ChatGPT - having a general interactive "answer-anything, tutor-me-on-anything" tool is a big deal; you just need to be smart about how you use it (prompting it well, verifying the outputs), and the results are surprisingly good especially with GPT-4.
There are a few types of RF receiver architecture and I was asking ChatGPT4 to provide the codes in Mermaid for direct conversion receiver architecture since it does not has support for image generation capability. The provided codes contained some errors when copied/pasted to Mermaid. Then I feedback the errors back to ChatGPT4 it then apologized and duely provided the correct codes that Mermaid can draw the receiver diagram. After that the diagram is fed into Krogi and draw.io for further processing resulting in the direct conversion diagram with editable boxes and texts that is ready to be used or modified with draw.io.
I was at the beach and used it to write a streaming C++ web server integrating whisper.cpp. I should opensource it eventually, but was like totally amazing since I used to write C++ all the time but it's been like 15 years and was like well i'll use c++20 and sure enough even with a few drinks and a lot of sun the code came together really nicely.
I think people are most surprised when using it outside of their own domain.
As an example: I recently showed my mother GPT-4. I asked her what she wanted to see, and having little knowledge of the tool's capabilities, she asked me to surprise her. Apparently I had done this before (with GPT-3), as the poem I generated about Petoskey stones was met with little fanfair: "I've seen it generate poems... what else can it do?"
At that point I showed her a Python file and asked GPT to translate it to Typescript. Needless to say she's not a poet or a programmer, but the fact it could do something like that, entirely outside of her domain, was mind-blowing.
When 4 perfectly followed a complex prompt that I had spent hours trying (unsuccessfully) to get 3.5-turbo to follow even somewhat acceptably. Both models continue to impress me with their ability to tell me how to tune my prompts to get better results more consistently.
It blew my mind when I gave it a TypeScript function with multiple nested awaits, all tangled up into a chain of multiple `map` calls. A comment I wrote above it was “TODO refactor this monstrosity.” It was clever enough to split it into a main function and multiple utils. And it added JSDocs. The result passed my existing tests immediately.
I was testing the pixie observability tool for an internal PoC. It has only Go and Python client libraries. I asked Bing AI, how to the call the API in Java. It made up an fake Java client library and gave instructions for adding maven dependency. When prompted further, it gave some sample code as well.
54 comments
[ 3.2 ms ] story [ 118 ms ] threadSometimes it'll write really elegant code that just nails the request. Other times it insists the task is impossible ("too complex"). Days later it'll do it without fuss.
I suspect there's some sort of model shell game going on behind the scenes. The best results I get are always outside of business hours.
This is the lobotomization. It's interesting to watch the interviews with some red teamers and bing integration devs who had access before they did this to it. They are fully Blake Lemoine'd.
Had it write non blocking micro-controller code for a project (about 75% of the output was put into production) it helped making variable names and general structure. The first output was bad because it blocked the button read while "breathing" and led but then I told it: "non blocking code please".
Did the reverse with some regulatory documents too, asking it to summarise in vernacular then asking specific questions to better wrap my head around the docs. I wouldn't trust it fully for this of course, more like a supplementary learning aid.
I keep hitting limits with context size though. Eg. When coding it becomes cumbersome once your project hits a certain complexity. You have to carefully construct prompts to get useful outputs. Being able to add the entire source code into a prompt to have more contextual responses would really increase utility.
It was nothing special, nor not completely correct, but it is the first time I've been able to pinpoint ChatGPTs source.
For my example I knew of the sources already. ChatGPT didn't make the connection
Both ChatGPT and Phind insisted on giving me example SMT problems and then incorrect solutions to them. The HE answer was so vague as to be useless, and I don’t actually know enough about SDN to verify the last one but it seemed useful.
Overall it didn’t give me a lot of confidence in the two systems.
The thing that is mind blowing to me is that I can talk with GPT about basically any topic and it can help get me in the right direction, I can talk with it to get clarity on anything that’s confusing, and it’s like a way to collaborate on thought without a human. That’s mindblowing to me!
Once discovered, I can go search more conventionally for more reputable sources
Okay, I know that sounds like a sleight but I mean it in reverse: we give it a hard time for failing at something humans can’t do either: be an expert at everything. Journalists do a great job despite regularly driving SMEs nuts.
I use it not for querying immutable truths but for exploring topics. It trivializes what used to be paralyzing for me: “where do I begin if I want to learn about X?”
It wasn't because it was a superintelligent entity, but because it was an intelligent scientific logbook.
It would break down possible root causes of the bug. I could add more if it misses any. It would give me the simplest possible solution to test each one. Sometimes it's not right and I have a simpler test.
But it'll track which ones we have tried so we don't go in circles with debugging. It's really a superior rubber duck + notepad.
I'm not disappointed that GPT doesn't know the answers to questions in my domain of expertise. I'm disappointed that GPT doesn't know that it isn't an expert in my domain, and makes up a confident but wrong answers.
By this point I'm kinda developing an intuition for what GPT4 can and can't do, but whilst pushing against what it can do I still often find myself being impressed. Maybe not "mind blown" anymore because it's amazing how quickly you can get used to this stuff, but still. Not only does it do an excellent job of figuring out what to do when the instructions are clear enough, but it also has commands for adding library dependencies and will use relevant open source libs to make its job easier.
The hard part about using this sort of tool is that you can quickly become bottlenecked on figuring out what it is you actually want. It's easy to run ahead of yourself and end up with a mess. This sort of very rapid "painting with code" feel is a bit new; probably the slowness of typing allows your subconscious to think ahead a bit when doing programming normally. Also, I'm still teaching it how to explore the codebase efficiently so things that require a lot of context aren't quite there yet. I've got a whole plan for how to make that work better though.
I expect at some point soon some big company or some $100M "seed round" startup will come along and do this better, but I didn't want to wait.
Maybe that’s an easy thing to know, but I expected to not get a good answer.
None of the scripts it produced are incredible or whatever (the tasks being more menial than lohic-based), but considering the intricacies of bash and how often I just faceplant into them as an experienced sysadmin, if feel as though the ability to just say "write a script that compares a list of agents against every manager and outputs the ones not found anywhere" and have it output basically a working script first try is impressive.
I think this is a great example of the strides of LLMs. It’s absolutely leaps and bounds ahead of what was unwieldily cutting edge just 5 years ago.
https://homes.cs.washington.edu/~mernst/pubs/nl2bash-corpus-...
Only to realize in the process that all I really needed was to modify the shader program's depth checking behavior. Even just disabling it altogether looks fine and makes everything way faster. Never in my wildest imagination a year ago would I even dream of implementing WBOIT in one night and then just leaving it in a branch for a one line change.
In the process I realized a little trick on my own, using what I learned, that modifying the depth values in the fragment shader could allow selective alpha-blending/occlusion. Basically the idea was to put stuff I wanted to blend order-independently all at an equal z position and then set the depth function to "less than or equal" instead of "less than". I used the occluding geometry's depth buffer as a source to modify the translucent stuff's depth buffer in the fragment shader (with a slight offset to get it behind the occluding stuff).
There's absolutely no way I'd have had the guts to try all that in one night without the robot tools.
Same thing with ChatGPT - having a general interactive "answer-anything, tutor-me-on-anything" tool is a big deal; you just need to be smart about how you use it (prompting it well, verifying the outputs), and the results are surprisingly good especially with GPT-4.
Everyone could be producing way more than they currently are.
Everyone could be generating 1 app every 3 hours.
Everyone could be writing entire novels.
Everyone could be writing meaningful Wikipedia articles.
DIY guides should be 100x larger .
As an example: I recently showed my mother GPT-4. I asked her what she wanted to see, and having little knowledge of the tool's capabilities, she asked me to surprise her. Apparently I had done this before (with GPT-3), as the poem I generated about Petoskey stones was met with little fanfair: "I've seen it generate poems... what else can it do?"
At that point I showed her a Python file and asked GPT to translate it to Typescript. Needless to say she's not a poet or a programmer, but the fact it could do something like that, entirely outside of her domain, was mind-blowing.
For me, I'm surprised how little I use Google...