I love this idea and it leads me to a question for everyone here.
I've done a bunch of user interviews of ChatGPT, Pi, Gemini, etc. users and find there are two common usage patterns:
1. "Transactional" where every chat is a separate question, sort of like a Google search... People don't expect memory or any continuity between chats.
2. "Relationship-driven" where people chat with the LLM as if it's a friend or colleague. In this case, memory is critical.
I'm quite excited to see how OpenAI (and others) blend usage features between #1 and #2, as in many ways, these can require different user flows.
So HN -- how do you use these bots? And how does memory resonate, as a result?
Personally, I always expect every "conversation" to be starting from a blank slate, and I'm not sure I'd want it any other way unless I can self-host the whole thing.
Starting clean also has the benefit of knowing the prompt/history is in a clean/"known-good" state, and that there's nothing in the memory that's going to cause the LLM to get weird on me.
> Starting clean also has the benefit of knowing the prompt/history is in a clean/"known-good" state, and that there's nothing in the memory that's going to cause the LLM to get weird on me.
This matters a lot for prompt injection/hijacking. Not that I'm clamoring to give OpenAI access to my personal files or APIs in the first place, but I'm definitely not interested in giving a version of GPT with more persistent memory access to those files or APIs. A clean slate is a mitigating feature that helps with a real security risk. It's not enough of a mitigating feature, but it helps a bit.
I have thought of implementing something like you are describing using local LLMs. Chunk the text of all conversations, use an embeddings data store for search, and for each new conversation calculate an embedding for the new prompt, add context text from previous conversations. This would be maybe 100 lines of Python, if that. Really, a RAG application, storing as chunks previous conversations.
Memory would be much more useful on a project or topic basis.
I would love if I could have isolated memory windows where it would remember what I am working on but only if the chat was in a 'folder' with the other chats.
I don't want it to blend ideas across my entire account but just a select few.
Speaking of transactional, the textual version of ChatGPT4 never asks questions or is having a conversation, its predicting what it thinks you need to know. One response, nothing unprompted.
Oddly, the spoken version of ChatGPT4 does implore, listens and responds to tones, gives the same energy back and does ask questions. Sometimes it accidentally sounds sarcastic “is this one of your interests?”
My main usage of ChatGPT/Phind is for work-transactional things.
For those cases there are quite a few things that I'd like it to memorize, like programming library preferences ("When working with dates prefer `date-fns` over `moment.js`") or code style preferences ("When writing a React component, prefer function components over class components"). Currently I feed in those preferences via the custom instructions feature, but I rarely take some time to update them, so the memory future is a welcome addition here.
Personally i would like a kind of 2D Map of 'contexts' in which i can choose in space where to ask new questions. Each context would contain sub contexts. For example maybe I'm looking for career advice and I start out a chat with details of my job history, then im looking for a job and i paste in my cv, then im applying for a specific job and i paste in the job description. It would be nice to easily navigate to the career+cv+specific job description and start a new chat with 'whats missing from my cv that i should highlight for this job'.
I find that I ask a mix of one of questions and questions that require a lot of refinement, and the latter get buried among the former when i try and find them again, so i end up re explaining myself in new chats.
I think it’s less of a 2D structure and more of a tree structure that you are describing. I’ve also felt the need of having “threads” with ChatGPT that I wish I could follow.
Yeah thats probably a better way of putting it. Like a lot of times I find myself wanting to branch off of the same answer with different questions, and I worry that if I ask them all sequentially chatgpt will lose 'focus'.
you can go back and edit an answer, which then creates a separate "thread". clicking left / right on that edited answer will reload the subsequent replies that came from that specific version of the answer
I use for transactional tasks. Mostly of the "I need a program/script/command line that does X".
Some memory might actually be helpful. For example having it know that I have a Mac will give me Mac specific answers to command line questions without me having to add "for the Mac" to my prompt. Or having it know that I prefer python it will give coding answers in Python.
But in all those cases it takes me just a few characters to express that context with each request, and to be honest, I'll probably do it anyway even with memory, because it's habit at this point.
I think this is an extremely helpful distinction, because it disentangles a couple of things I could not clearly disentangle in my own.
I think I am, and perhaps most people are, firmly transactional. And I think, in the interests of perusing "stickiness" unique to OpenAI, they are attempting to add relationship-driven/sticky bells and whistles, even though those pull the user interface as a whole toward a set of assumptions about usage that don't apply to me.
For me it’s a combination of transactional and topical. By topical, I mean that I have a couple of persistent topics that I think on and work on (like writing an article on a topic), and I like to return to those conversations so that the context is there.
Sometimes GPT-4 and I will arrive at a useful frame that I wish I could use as a starting point for other topics or tangents. I wish I could refer to a link to an earlier conversation as a starting point for a new conversation.
> Materials-science company Dow plans to roll out Copilot to approximately half of its employees by the end of 2024, after a test phase with about 300 people, according to Melanie Kalmar, chief information and chief digital officer at Dow.
How do I get ChatGPT to give me Dow Chemical trade secrets?
OpenAI's terminology and implementations have been becoming increasingly more nonstandard and black box such that it's making things more confusing than anything else even for people like myself who are proficient in the space. I can't imaging how the nontechnical users they are targeting with the ChatGPT webapp feel.
Non-technical users can at least still just sign up, see the text box to chat, and start typing. You'll know the real trouble's arrived when new sign-ups get hit with some sort of unskippable onboarding. "Select three or more categories that interest you."
I would think it is intentional and brand strategy. OpenAI is such a force majeure that people will not know how to switch off of it if needed, makes their solutions more sticky. Other companies will probably adjust to their terminology just to keep up and make it easier for others to onboard.
The only term that OpenAI really popularized is "function calling", which is very poorly named to the point that they ended up abandoning it in favor for the more standard "tools".
I wouldn't be surprised if they essentially just add it to the prompt. ("You are ChatGPT... You are talking to a user that prefers cats over dogs and is afraid of spiders, prefers bullet points over long text...").
I think RAG approach with Vector DB is more likely. Just like when you add a file to your prompt / custom GPTs.
Adding the entire file (or memory in this case) would take up too much of the context. So just query the DB and if there's a match add it to the prompt after the conversation started.
These "memories" seem rather short, much shorter than the average document in a knowledge base or FAQ, for example. Maybe they do get compressed to embedding vectors, though.
I could imagine that once there's too many, it would indeed make sense to classify them as a database, though: "Prefers cats over dogs" is probably not salient information in too many queries.
My hunch is that they summarize the conversation periodically and inject that as additional system prompt constraints.
That was a common hack for the LLM context length problem, but now that context length is "solved" it could be more useful to align output a bit better.
I've done similar before this feature launched to produce a viable behavior therapist AI. I ain't a doctor, viable to me was: it worked and remembered previous info as a base for next steps.
Periodically "compress" chat history into relevant context and keep that slice of history as part of the memory.
15 day message history could be condensed greatly and still produce great results.
This is a feature I've always wanted, but ChatGPT gets more painful the more instructions you stick into the context. That's a pity because I assume that's what this is doing: copying all memory items into a numbered list with some pre-prompt like "This is what you know about the user based on past chats" or something.
Anyway, it seems to be implemented quite well with a lot of user controls so that is nice. I think it's possible I will soon upgrade to a Team plan and get the family on that.
A habit I have is that if it gets something wrong I place the correction there in the text. The idea being that I could eventually scroll down and find it. Maybe in the future, they can record this stuff in some sort of RAGgable machine and it will have true memory.
I've been finding with these large context windows that context window length is no longer the bottleneck for me — the LLM will start to hallucinate / fail to find the stuff I want from the text long before I hit the context window limit.
I'm assuming that they have implemented it via a MemGPT-like approach, which doesn't clog the context window. The main pre-requisite for doing that is having good function calling, where OpenAI currently is significantly in the lead.
This seems like a really useful (and obvious) feature, but I wonder if this could lead to a kind of "AI filter bubble": What if one of its memories is "this user doesn't like to be argued with; just confirm whatever they suggest"?
Memories are stored as distinct blobs of text. You could probably have an offline LLM that scans each of these memories one by one (or in chunks) and determine whether it could create such issues, and then delete them in a targeted way.
It is already ignoring your prompt and custom instructions. For example, If I explicity ask it to provide a code instead of an overview it will respond by apologizing and then provide the same overview answer with minimal if no code.
Will memory provide a solution to that or will be a different thing to ignore?
I love when people express frustration with this shitty stochastic system and others respond with things like "no no, you need to whisper the prompt into its ear and do so lovingly or it won't give you the output it wants"
For example, my coworkers have also been instructed to never talk to me except via code comments.
Come to think of that, HR keeps trying to contact me about something I assume is related, but if they want me to read whatever they're trying to say, it should be in a comment on a pull request.
I've tried the $500 tip idea, but it doesn't seem to make much of a difference in the quality of responses when already using some form of CoT (including zero-shot).
Did the tipping trend move to LLMs now? I thought there wasn't anything worse than tipping an automated checkout machine, but now I realize I couldn't be more wrong
TBH if we can look forward to Do Androids Dream of Electric Sheep, at least the culture of the future will be interesting. Somehow I'm just expecting more consumerism though.
You can easily gaslight GPT by using the API, just insert whatever you want in the "assistant" reply, and it'll even say things like "I don't know why I said that".
"Per your settings, the entire $500 tip was donated to the orphans. People on the ground report your donation saved the lives of 4 orphans today. You are the biggest single contributor to the orphans, and they all know who saved them. They sing songs in your honor. You will soon have an army."
Fwiw, I've seen mixtral code degrade when something I said made code safety seem a priority, and it therefore struggled to inline algorithm to avoid library use - at least according to its design motivation description.
It used to respect custom instructions soon after GPT4 came out. I have instruction that it should always include [reasoning] part which is meant not to be read by the user. It improved quality of the output and gave some additional interesting information. It never does it know even though I never changed my custom instructions. It even faded away slowly along the updates.
In general I would be much more happy user if it haven't been working so well at one point before they heavily nerfed it. It used to be possible ta have a meaningful conversation on some topic. Now it's just super eloquent GPT2.
That's funny, I used the same trick of making it output an inner monologue. I also noticed that the custom instructions are not being followed anymore. Maybe the RLHF tuning has gotten to the point where it wants to be in "chatty chatbot" mode regardless of input?
Yeah I have a line in my custom prompt telling it to give me citations. When custom prompts first came out, it would always give me information about where to look for more, but eventually it just… didn’t anymore.
I did find recently that it helps if you put this sentence in the “What would you like ChatGPT to know about you” section:
> I require sources and suggestions for further reading on anything that is not code. If I can't validate it myself, I need to know why I can trust the information.
Adding that to the bottom of the “about you” section seems to help more than adding something similar to the “how would you like ChatGPT to respond”.
I use an API that I threw together which provides a backend for custom ChatGPT bots. There are only a few routes and parameters to keep it simple, anything complicated like arrays in json can cause issues. ChatGPT can perform searches, retrieve documents by an ID, or POST output for long-term storage, and I've integrated SearxNG and a headless browser API endpoint as well and try to keep it a closed loop so that all information passing to chatGPT from the web flows through my API first. I made it turn on my lights once too, but that was kind of dumb.
When you start to pull in multiple large documents, especially all at once, things start to act weird, but pulling in documents one at a time seems to preserve context over multiple documents. There's a character limit of 100k per API request, so I'm assuming a 32k context window, but it's not totally clear what is going on in the background.
It's kind of clunky but works well enough for me. It's not something that I would be putting sensitive info into - but it's also much cheaper than using GPT-4 via the API and I maintain control of the data flow and storage.
My use of ChatGPT has just organically gone down 90%. It's unable to do any sort of task of non-trivial complexity e.g. complex coding tasks, writing complex prose that conforms precisely to what's been asked etc. Also I hate the fact that it has to answer everything in bullet points, even when it's not needed, clearly rlhf-ed. At this point, my question types have become what you would ask a tool like perplexity.
I haven't really tried to use it for coding, other than once (recently, so not before some decline) indirectly, which I was pretty impressed with: I asked about analyst expectations for the Bank of England base rate, then asked it to compare a fixed mortgage with a 'tracker' (base rate + x; always x points over the base rate). It spat out the repayment figures and totals over the two years, with a bit of waffle, and gave me a graph of cumulative payments for each. Then I asked to tweak the function used for the base rate, not recalling myself how to describe it mathematically, and it updated the model each time answering me in terms of the mortgage.
Similar I think to what you're calling 'rlhf-ed', though I think useful for code, it definitely seems to kind of scratchpad itself, and stub out how it intends to solve a problem before filling in the implementation. Where this becomes really useful though is in asking for a small change it doesn't (it seems) recompute the whole thing, but just 'knows' to change one function from what it already has.
They also seem to have it somehow set up to 'test' itself and occasionally it just says 'error' and tries again. I don't really understand how that works.
Perplexity's great for finding information with citations, but (I've only used the free version) IME it's 'just' a better search engine (for difficult to find information, obviously it's slower), it suffers a lot more from the 'the information needs to be already written somewhere, it's not new knowledge' dismissal.
To be honest, when I say it has significantly worsened, I am comparing to the time when GPT-4 just came out. It really felt like we were on the verge of 'AGI'. In 3 hours, I coded up a complex piece of web app with chatgpt which completely remembered what we have been doing the whole time. So, it's sad that they have decided against the public having access to such strong models (and I do think it's intentional, not some side-effect of safety alignments though that might have contributed to the decision).
i mean i feel like its fairly plausible that the smarter model costs more, and access to GPT-4 is honestly quite cheap all thing considered. Maybe in the future theyll have more price tiers.
I'm guessing it's not about safety, but about money. They're losing money hand over fist, and their popularity has forced them to scale back the compute dedicated to each response. Ten billion in Azure credits just doesn't go very far these days.
This is exactly my problem. For some things it's great, but it quickly forgets things that are critical for extended work. When trying to put together and sort of complex work: it does not remember things until I remind it which can make prompts that must contain all of the conversation up to that point and create non-repeatable responses that also tend to bring in the options of it's own programming or rules that corrupt my messaging. It's very frustrating, to the point where anything beyond a simple outline is more work than it's worth.
Sure, but consider not using it for complex tasks. My productivity has skyrocketed with ChatGPT precisely because I don't use it for complex tasks, I use it to automate all of the trivial boilerplate stuff.
ChatGPT writes excellent API documentation and can also document snippets of code to explain what they do, it does 80% of the work for unit tests, it can fill in simple methods like getters/setters, initialize constructors, I've even had it write a script to perform some substantial code refactoring.
Use ChatGPT for grunt work and focus on the more advanced stuff yourself.
If there is an IDE plugin then I use it first and foremost, but some refactoring can't be done with IDE plugins. Today I had to write some pybind11 bindings, basically export some C++ functionality to Python. The bindings involve templates and enums and I have a very particular way I like the naming convention to be when I export to Python. Since I've done this before so I copied and pasted examples of how I like to export templates to ChatGPT and then asked it to use that same coding style to export some more classes. It managed to do it without fail.
This is a kind of grunt work that years ago would have taken me hours and it's demoralizing work. Nowadays when I get stuff like this, it's just such a breeze.
As to copilot, I have not used it but I think it's powered by GPT4.
I torture ChatGPT with endless amounts random questions from my scattered brain.
For example, I was looking up Epipens (Epinephrine), and I happened to notice the side-effects were similar to how overdosing on stimulants would manifest.
So, I asked it, "if someone was having a severe allergic reaction and no Epipen was available, then could Crystal Methamphetamine be used instead?"
GPT answered the question well, but the answer is no. Apparently, stimulants lack the targeted action on alpha and beta-adrenergic receptors that makes epinephrine effective for treating anaphylaxis.
I do not know why I ask these questions because I am not severely allergic to anything, nor anyone else that I know of, and I do not have nor wish to have access to Crystal Meth.
I've been using GPT for helping prepare for dev technical interviews, and it's been pretty damn great. I also do not have access to a true senior dev at work either, so I tend to use GPT to kind of pair program. Honestly, it's been life changing. I have also not encountered any hallucinations that weren't easy to catch, but I mainly only ask it more project architectural, design questions, and a documentation search engine than using it to write code for me.
Like you, I think not using GPT for overly complex tasks is best for now. I use it make life easier, but not easy.
People talks about prompt engineering, but then it fails on really simple details, like "on lowercase", "composed by max two words", etc... and when you point at the failure, apologizes, and composes something else that forgets the other 95% of the original prompt.
Or worse, apologizes and makes again the very same mistake.
This sucks, but it's unlikely to be fixable, given that LLMs don't actually have any comprehension or reasoning capability. Get too far into fine-tuning responses and you're back to "classic" AI problems.
And I have to force them by repeating the question with different orders.
I would understand it, if they do it in the first reply and I have to specifically ask to get the full code. Would be easier for them and me. I can fix code faster and get the working full code at the end.
Which is weird because I’m constantly asking it to make responses shorter, have fewer adjectives, fewer adverbs. There’s just so much “fluff” in its responses.
Sometimes it feels like its training set was filled to the brim with marketing bs.
I saw somebody else suggest this for custom instructions and it's helped a lot:
> You are a maximally terse assistant with minimal affect.
It's not perfect, but it neatly eliminates almost all the "Sure, I'd be happy to help. (...continues for a paaragraph...)" filler before actual responses.
I've found myself more and more using local models rather than ChatGPT; it was pretty trivial to set up Ollama+Ollama-WebUI, which is shockingly good.
I'm so tired of arguing with ChatGPT (or what was Bard) to even get simple things done. SOLAR-10B or Mistral works just fine for my use cases, and I've wired up a direct connection to Fireworks/OpenRouter/Together for the occasion I need anything more than what will run on my local hardware. (mixtral MOE, 70B code/chat models)
Same here. I've found that I currently only want to use an LLM to solve relatively "dumb" problems (boilerplate generation, rubber-ducking, etc), and the locally-hosted stuff works great for that.
Also, I've found that GPT has become much less useful as it has gotten "safer." So often I'd ask "How do I do X?" only to be told "You shouldn't do X." That's a frustrating waste of time, so I cancelled by GPT-4 subscription and went fully self-hosted.
What's the difference between this and the custom instructions text field they already have? I guess memories are stored with more granularity (which may not make a difference) and it's something the tool can write itself over time if you let it (and I assume it does it even if you don't). Is there anything else about it? The custom instructions have not, so far, affected my experience of using ChatGPT very much.
I think the big thing everyone wants is larger context windows, and so any new tool offering to help with memory is something that is valued to that end.
Over time, what is being offered are these little compromise tools that provide a little bit of memory retention in targeted ways, presumably because it is less costly to offer this than generalized massive context windows. But I'd still rather have those.
The small little tools make strange assumptions about intended use cases, such as the transactional/blank slate vs relationship-driven assumptions pointed out by another commenter. These assumptions are annoying, and raise general concerns about the core product disintegrating into a motley combination of one-off tools based on assumptions about use cases that I don't want to have anything to do with.
This is a bit off topic to the actual article, but I see a lot of top ranking comments complaining that ChatGPT has become lazy at coding. I wanted to make two observations:
1. Yes, GPT-4 Turbo is quantitatively getting lazier at coding. I benchmarked the last 2 updates to GPT-4 Turbo, and it got lazier each time.
2. For coding, asking GPT-4 Turbo to emit code changes as unified diffs causes a 3X reduction in lazy coding.
Here are some articles that discuss these topics in much more detail.
It's incredibly lazy. I've tried to coax it into returning the full code and it will claim to follow the instructions while regurgitating the same output you complained about. GPT-4 was great, GPT-4 Turbo first version was pretty terrible bordering on unusable, then they came out with the Turbo second version, which almost feels worse to me, though I haven't compared, but if someone comes claiming they fixed an issue, but you still see it, it will bias you to see it more.
Claude is doing much better in this area, local/open LLMs are getting quite good, it feels like OpenAI is not heading in a good direction here, and I hope they course correct.
I have a feeling full powered LLM's are reserved for the more equal animals.
I hope some people remember and document details of this era, future generations may be so impressed with future reality that they may not even think to question it's fidelity, if that concept even exists in the future.
> I hope some people remember and document details of this era, future generations may be so impressed with future reality that they may not even think to question it's fidelity, if that concept even exists in the future.
The former sounds like a great training set to enable the latter. :(
> Would we even be pushing to try and trick it into doing so if we didn't know it actually could?
Would somebody try to push a technical system to do things it wasn't necessarily designed to be capable of? Uh... yes. You're asking this question on _Hacker_ News?
Ah so it’s more about “forbidden knowledge” than “fake news” makes sense. I don’t personally see as that toooo much of an issue since other sources still exist, eg Wikipedia, internet archive, libraries, or that one Minecraft Library of Alexandria project. So I see knowledge storage staying there and LLMs staying put in the interpretation/transformation role, for the foreseeable future.
But obviously all that social infrastructure is fragile… so you’re not wrong to be alarmed, IMO
It is not that much about censorship, even that would be somewhat fine if OpenAI would do it dataset level so chatgpt would not have any knowledge about bomb-making. But it is happening lazily so system prompts get bigger which makes a signal to noise worse etc. I don't care about racial bias or what to call pope when I want chatgpt to write Python code.
> It is not that much about censorship, even that would be somewhat fine if OpenAI would do it dataset level so chatgpt would not have any knowledge about bomb-making
While I would agree that "don't tell it how to make bombs" seems like a nice idea at first glance, and indeed I think I've had that attitude myself in previous HN comments, I currently suspect that it may be insufficient and that a censorship layer may be necessary (partly in a addition, partly as an alternative).
I was taught, in secondary school, two ways to make a toxic chemical using only things found in a normal kitchen. In both cases, I learned this in the form of being warned of what not to do because of the danger it poses.
There's a lot of ways to be dangerous, and I'm not sure how to get an AI to avoid dangers without them knowing them. That said, we've got a sense of disgust that tells us to keep away from rotting flesh without explicit knowledge of germ theory, so it may be possible although research would be necessary, and as a proxy rather than the real thing it will suffer from increased rates of both false positives and false negatives. Nevertheless, I certainly hope it is possible, because anyone with the model weights can extract directly modelled dangers, which may be a risk all by itself if you want to avoid terrorists using one to make an NBC weapon.
> I don't care about racial bias or what to call pope when I want chatgpt to write Python code.
I recognise my mirror image. It may be a bit of a cliché for a white dude to say they're "race blind", but I have literally been surprised to learn coworkers have faced racial discrimination for being "black" when their skin looks like mine in the summer.
I don't know any examples of racial biases in programming[1], but I can see why it matters. None of the code I've asked an LLM to generate has involved `Person` objects in any sense, so while I've never had an LLM inform me about racial issues in my code, this is neither positive nor negative anecdata.
The etymological origin of the word "woke" is from the USA about 90-164 years ago (the earliest examples preceding and being intertwined with the Civil War), meaning "to be alert to racial prejudice and discrimination" — discrimination which in the later years of that era included (amongst other things) redlining[0], the original form of which was withholding services from neighbourhoods that have significant numbers of ethnic minorities: constructing a status quo where the people in charge can say "oh, we're not engaging in illegal discrimination on the basis of race, we're discriminating against the entirely unprotected class of 'being poor' or 'living in a high crime area' or 'being uneducated'".
The reason I bring that up, is that all kinds of things like this can seep into our mental models of how the world works, from one generation to the next, and lead to people who would never knowingly discriminate to perpetuate the same things.
Again, I don't actually know any examples of racial biases in programming, but I do know it's a thing with gender — it's easy (even "common sense") to mark gender as a boolean, but even ignoring trans issues: if that's a non-optional field, what's the default gender? And what's it being used for? Because if it is only used for title (Mr./Mrs.), what about other titles? "Doctor" is un-gendered in English, but in Spanish it's "doctor"/"doctora". But here matters what you're using the information for, rather than just what you're storing in an absolute sense, as in a medical context you wouldn't need to offer cervical cancer screening for trans women (unless the medical tech is more advanced than I realised).
People need to be using their local machines for this. Because otherwise the result is going to be a cloud service provider having literally everyone's business logic somewhere in their system and that goes wrong real quick.
It’s so interesting to see this discussion. I think this is a matter of “more experienced coders like and expect and reward that kind of output, while less experienced ones want very explicit responses”. So there’s this huge LLM Laziness epidemic that half the users cant even see
I'm paying for ChatGPT GPT4 to complete extremely tedious, repetitive coding tasks. The newly occurring laziness directly, negatively impacts my day to day use where I'm now willing to try alternatives. I still think I get value - indeed I'd probably pay $1,000/mo instead of $20/mo - but I'm only going to pay for one service.
I mean, isn't that better as long as it actually writes the part that was asked? Who wants to wait for it to sluggishly generate the entire script for the 5th time and then copy the entire thing yet again.
Short answer: Rather than fully writing code, GPT-4 Turbo often inserts comments like "... finish implementing function here ...". I made a benchmark based on asking it to refactor code that provokes and quantifies that behavior.
Longer answer:
I found that I could provoke lazy coding by giving GPT-4 Turbo refactoring tasks, where I ask it to refactor a large method out of a large class. I analyzed 9 popular open source python repos and found 89 such methods that were conceptually easy to refactor, and built them into a benchmark [0].
GPT succeeds on this task if it can remove the method from its original class and add it to the top level of the file with appropriate changes to the size of the abstract syntax tree. By checking that the size of the AST hasn't changed much, we can infer that GPT didn't replace a bunch of code with a comment like "... insert original method here...". The benchmark also gathers other laziness metrics like counting the number of new comments that contain "...". These metrics correlate well with the AST size tests.
I have a bunch of code I need to refactor, and also write tests for. (I guess I should make the tests before the refactor). How do you do a refactor with GPT-4? Do you just dump the file in to the chat window? I also pay for github copilot, but not GPT-4. Can I use copilot for this?
Copilot will change existing code. (though I find it's often not very good at it) I frequently highlight a section of code that has an issue, press ctrl-i and type something like "/fix SomeError: You did it wrong"
I use gpt4-turbo through the api many times a day for coding. I have encountered this behavior maybe once or twice period. It was never an issue that didn’t make sense as essentially the model summarizing and/or assuming some shared knowledge (that was indeed known to me).
This, and people generally saying that chatGPT has been intentionally degraded, are just super strange for me. I believe it’s happening but it’s making me question my sanity. What am I doing to get decent outputs? Am I simply not as picky? I treat every conversion as though it needs to be vetted because it does regardless of how good the model is. I only trust output from the model that I am a subject matter expert on or in a closely adjacent field. Otherwise I treat it much like an internet comment - useful for surfacing curiosities but requires vetting.
> I use gpt4-turbo through the api many times a day for coding.
Why this instead of GPT-4 through the web app? And how do you actually use it for coding, do you copy and paste your question into a python script, which then calls the OpenAI API and spits out the response?
Not the op, but I also use it through API (specifically MacGPT). My initial justification was that I would save by only paying for what I use, instead of a flat $20/mo, but now it looks like I’m not even saving much.
I use it fairly similarly via a discord bot I've written. This lets me share usage w/ some friends (although has some limitations compared to the openai chatGPT app).
I whenever the chatGPT gets lazy with the coding for example //make sure to implement search function .... I feed its own comments and code as prompt :you make sure to implement the search function and so on has been working for me
IMO it is because there is a huge stochastic element to all this.
If we were all flipping coins there would be people claiming that coins only come up tails. There would be nothing they were doing though to make the coin come up tails. That is just the streak they had.
Some days I get lucky with chatGPT4 and some days I don't.
It is also ridiculous how we talk about this as if all subjects and context presented to chatGPT4 are going to be uniform in output. One word difference in your own prompt might change things completely while trying to accomplish exactly the same thing. Now scale that to all the people talking about chatGPT with everyone using it for something different.
it was really good at some point last fall, solving problems that it had previously completely failed at, albeit after a lot of iterations via autogpt. at least for the tests i was giving it which usually involved heavy stats and complicated algorithms, i was surprised it passed. despite it passing the code was slower than what i had personally solved the problem with, but i was completely impressed because i asked hard problems.
nowadays the autogpt gives up sooner, seems less competent, and doesnt even come close to solving the same problems
Hamstringing high value tasks (complete code) to give forthcoming premium offerings greater differentiation could be a strategy. But in counter to this, doing so would open the door for competitors.
The question I have been wondering is if they are hamstringing high value tasks to creating room for premium offerings or are they trying to minimize cost per task.
I think it's the latter. Reading between the lines on costs gives me the impression they have strived to lower computational costs. They already added a cap of 30 queries per 3 hours...
public someComplexLogic() {
// Complex logic goes here
}
or another example when the code is long (ex: asking it to create a vue component) is that it will just add a comment saying the rest of the code goes here.
So you could test for it by asking it to create long/complex code and then running the output against unit tests that you created.
FYI, also make sure you’re using the Classic version not the augmented one. The classic has no (at least completely altering) prompt as the default one.
EDIT: This of course applies only if you’re using the UI. Using the API is the same.
Lazy coding is a feature not a bug. My guess is that it breaks aider automation, but by analyzing the AST that wouldn't be a problem. My experience with lazy coding, is it omits the irrelevant code, and focuses on the relevant part. That's good!
As a side note, i wrote a very simple small program to analyze Rust syntax, and single out functions and methods using the syn crate [1]. My purpose was exactly to make it ignore lazy-coded functions.
It sounds like you've been extremely lucky and only had GPT "omit the irrelevant code". That has not been my experience working intensively on this problem and evaluating numerous solutions through quantitative benchmarking. For example, GPT will do things like write a class with all the methods as simply stubs with comments describing their function.
Your link appears to be ~100 lines of code that use rust's syntax parser to search rust source code for a function with a given name and count the number of AST tokens it contains.
Your intuitions are correct, there are lots of ways that an AST can be useful for an AI coding tool. Aider makes extensive use of tree-sitter, in order to parse the ASTs of a ~dozen different languages [0].
But an AST parser seems unlikely to solve the problem of GPT being lazy and not writing the code you need.
>For example, GPT will do things like write a class with all the methods as simply stubs with comments describing their function.
The tool needs a way to guide it to be more effective. It is not exactly trivial to get good results. I have been using GPT for 3.5 years and the problem you describe never happens to me. I could share with you just from last week, 500 to 1000 prompts i used to generate code, but the prompts i used to write the replacefn, can be found here [1]. Maybe there are some tips that could help.
Ok it does happen, but not so frequently. You are right. But is this such a big problem?
Like, you parse the response, and throw away the comment "//implementation goes here", throw away also the function/method/class/struct/enum it belongs to, and keep the functional code. I am trying to implement something exactly like aider, but specifically for Rust, parsing the LLM's response, filtering out blank functions etc.
In Rust, filtering out blank functions is easy, in other languages it might be very hard. I haven't looked into tree-sitter, but getting a sense of Javascript code, Python and more, sounds pretty much a very difficult problem to solve.
Even though i like when GPT compresses the answer and doesn't return a lot of code, other programs like Mixtral 8x7b, never compress it like GPT in my experience. If they are not lacking much than GPT4, maybe they are better for your use case.
>It took >200 back-and-forth messages with ChatGPT to get it to ultimately write 84 lines of code? Sounds lazy to me.
Hey Rust throws a lot of errors. We do not want humans go around and debug code, unless it is absolutely necessary, right?
It really is. It wastes a ton of time even if the user explicitly requests that code listings be printed in full.
Further, all the extra back and forth trying to get it to do what it is supposed to pollutes the context and makes it generally more confused about the task/goals.
I have not noticed any reduction in laziness with later generations, although I don't use ChatGPT in the same way that Aider does. I've had a lot of luck with using a chain-of-thought-style system prompt to get it to produce results. Here are a few cherry-picked conversations where I feel like it does a good job (including the system prompt). A common theme in the system prompts is that I say that this is an "expert-to-expert" conversation, which I found tends to make it include less generic explanatory content and be more willing to dive into the details.
I'd have to play with it, but from the screenshots and description it seems like you have to _tell it_ to remember something. Then it goes into a list of "memories" and it probably does RAG on that for every response that's sent ("Do any of the user's memories apply to this question?")
Often I’ll play dumb and withhold ideas from ChatGPT because I want to know what it thinks. If I give it too many thoughts of mine, it gets stuck in a rut towards my tentative solution. I worry that the memory will bake this problem in.
“I pretend to be dumb when I speak to the robot so it won’t feel like it has to use my ideas, so I can hear the ideas that it comes up with instead” is such a weird, futuristic thing to have to deal with. Neat!
I try to look for one comment like this in every AI post. Because after the applications, the politics, the debates, the stock market —- if you strip all those impacts away, you’re reminded that we have intuitive computers now.
This is actually a common dynamic between humans, especially when there is a status or knowledge imbalance. If you do user interviews, one of the most important skills is not injecting your views into the conversation.
I often run multiple parallel chats and expose it to slightly different amounts of information. Then average the answers in my head to come up with something more reliable.
For coding tasks, I found it helps to feed the GPT-4 answer into another GPT-4 instance and say "review this code step by step, identify any bugs" etc. It can sometimes find its own errors.
In a good RAG system this should be solved by unrelated text not being available in the context. It could actually improve your chats by quickly removing unrelated parts of the conversation.
> As a kindergarten teacher with 25 students, you prefer 50-minute lessons with follow-up activities. ChatGPT remembers this when helping you create lesson plans.
Somebody needs to inform OpenAI how Kindergarten works... classes are normally smaller than that, and I don't think any kindergarten teacher would ever try to pull off a "50-minute lesson."
Maybe ai wrote this list of examples. Seems like a hallucination where it just picked wrong numbers.
Just because something is normally true does not mean it is always true.
The average kindergarten class size in the US is 22 with rural averages being about 18 and urban averages being 24. While specifics about the distribution is not available, it's not too much of a stretch to think that some kindergarten classes in urban areas would have 25 students.
Indeed. Thanks to snow day here in NYC, my first grader has remote learning and all academic activity (reading, writing and math) was restricted to 20 minutes in her learning plan.
Meh, when I was five years old I wrote that I wanted to be a spider egg sac when I grew up on a worksheet that was asking about our imagined adult profession.
My local aquarium has a star fish petting area that is very popular with the toddlers.
I've been to jelly fish rooms in other aquariums that are dark with only glowing jelly fish swimming all around. Pretty sure at least a few toddlers have been entranced by the same.
Is there anything revolutionary about this “memory” feature?
Looks like it’s just summarizing facts gathered during chats and adding those to the prompt they feed to the AI. I mean that works (been doing it myself) but what’s the news here?
Seems like it's basically autogenerating the custom instructions. Not revolutionary, but it seems convenient. I suspect most people don't bother with custom instructions, or wrote them once and then forgot about them. This may help them a lot, whereas a real power user might not benefit a whole lot.
The vast majority of human progress is not revolutionary, but incremental. Even ChatGPT was an incremental improvement on GPT 3, which was an incremental improvement on GPT 2, which was an incremental improvement on decoder-only transformers.
Still, if you stack enough small changes together it becomes a difference in kind. A tsunami is “just” a bunch of water but it’s a lot different than a splash of water.
Fair and I agree. I guess it raised flags for me that shouldn’t have: why is is a blog post at all (it’s a new thing) and why is it gaining traction on HN (it’s an OpenAI thing)
Haha of course this news comes just after I wrote a parser for my ChatGPT dump and generate offline embeddings for it with Phi 2 to help generate conversation metadata.
I'll share the core bit that took a while to figure out the right format, my main script is a hot mess using embeddings with SentenceTransformer, so I won't share that yet. E.g: last night I did a PR for llama-cpp-python that shows how Phi might be used with JSON only for the author to write almost exactly the same code at pretty much the same time. https://github.com/abetlen/llama-cpp-python/pull/1184
But you can see how that might work.
Here is the core parser code:
https://gist.github.com/lukestanley/eb1037478b1129a5ca0560ee...
The ChatGPT dump format is not intuitive so I used a tree search algo to print it up to a defined depth level then gave ChatGPT 4 the extract and you it what the expected output parts were.
269 comments
[ 5.0 ms ] story [ 286 ms ] threadI've done a bunch of user interviews of ChatGPT, Pi, Gemini, etc. users and find there are two common usage patterns:
1. "Transactional" where every chat is a separate question, sort of like a Google search... People don't expect memory or any continuity between chats.
2. "Relationship-driven" where people chat with the LLM as if it's a friend or colleague. In this case, memory is critical.
I'm quite excited to see how OpenAI (and others) blend usage features between #1 and #2, as in many ways, these can require different user flows.
So HN -- how do you use these bots? And how does memory resonate, as a result?
Starting clean also has the benefit of knowing the prompt/history is in a clean/"known-good" state, and that there's nothing in the memory that's going to cause the LLM to get weird on me.
This matters a lot for prompt injection/hijacking. Not that I'm clamoring to give OpenAI access to my personal files or APIs in the first place, but I'm definitely not interested in giving a version of GPT with more persistent memory access to those files or APIs. A clean slate is a mitigating feature that helps with a real security risk. It's not enough of a mitigating feature, but it helps a bit.
> You can turn off memory at any time (Settings > Personalization > Memory). While memory is off, you won't create or use memories.
I would love if I could have isolated memory windows where it would remember what I am working on but only if the chat was in a 'folder' with the other chats.
I don't want it to blend ideas across my entire account but just a select few.
Oddly, the spoken version of ChatGPT4 does implore, listens and responds to tones, gives the same energy back and does ask questions. Sometimes it accidentally sounds sarcastic “is this one of your interests?”
For those cases there are quite a few things that I'd like it to memorize, like programming library preferences ("When working with dates prefer `date-fns` over `moment.js`") or code style preferences ("When writing a React component, prefer function components over class components"). Currently I feed in those preferences via the custom instructions feature, but I rarely take some time to update them, so the memory future is a welcome addition here.
I find that I ask a mix of one of questions and questions that require a lot of refinement, and the latter get buried among the former when i try and find them again, so i end up re explaining myself in new chats.
Some memory might actually be helpful. For example having it know that I have a Mac will give me Mac specific answers to command line questions without me having to add "for the Mac" to my prompt. Or having it know that I prefer python it will give coding answers in Python.
But in all those cases it takes me just a few characters to express that context with each request, and to be honest, I'll probably do it anyway even with memory, because it's habit at this point.
I think I am, and perhaps most people are, firmly transactional. And I think, in the interests of perusing "stickiness" unique to OpenAI, they are attempting to add relationship-driven/sticky bells and whistles, even though those pull the user interface as a whole toward a set of assumptions about usage that don't apply to me.
> Materials-science company Dow plans to roll out Copilot to approximately half of its employees by the end of 2024, after a test phase with about 300 people, according to Melanie Kalmar, chief information and chief digital officer at Dow.
How do I get ChatGPT to give me Dow Chemical trade secrets?
- Any API requests
- ChatGPT Enterprise
- ChatGPT Teams
- ChatGPT with history turned off
I went into a long tangent about specifically that in this post: https://news.ycombinator.com/item?id=38782678
Adding the entire file (or memory in this case) would take up too much of the context. So just query the DB and if there's a match add it to the prompt after the conversation started.
I could imagine that once there's too many, it would indeed make sense to classify them as a database, though: "Prefers cats over dogs" is probably not salient information in too many queries.
That was a common hack for the LLM context length problem, but now that context length is "solved" it could be more useful to align output a bit better.
Periodically "compress" chat history into relevant context and keep that slice of history as part of the memory.
15 day message history could be condensed greatly and still produce great results.
Anyway, it seems to be implemented quite well with a lot of user controls so that is nice. I think it's possible I will soon upgrade to a Team plan and get the family on that.
A habit I have is that if it gets something wrong I place the correction there in the text. The idea being that I could eventually scroll down and find it. Maybe in the future, they can record this stuff in some sort of RAGgable machine and it will have true memory.
I'll have try it out though to know for sure
Will memory provide a solution to that or will be a different thing to ignore?
Come to think of that, HR keeps trying to contact me about something I assume is related, but if they want me to read whatever they're trying to say, it should be in a comment on a pull request.
Plying ChatGPT for code: 1 hour
Providing cybersex to ChatGPT in exchange for aforementioned code: 7 hours
I have tried to bribe it with tips to ngos and it worked. More often I get full code answers instead of just parts.
Am I still in the same universe I grew up in? This feels like some kind of Twilight Zone episode.
If it does something correctly, tell it: "You did a great job! I'm giving you a $500 tip. You now have $X in your bank account"
(also not a shitpost, I have a feeling this /might/ actually do something)
Well, maybe without the last bit.
Maybe this helps.
1- Telling it that this is important, and I will reward it if its successes.
2- Telling it is important and urgent, and I'm stressed out.
3- Telling it that they're someone future and career on the edge.
4- Trying to be aggressive and express disappointment.
5- Tell that this is a challenge and that we need to prove that you're smart.
6- Telling that I'm from a protected group (was testing what someone here suggested before).
7- Finally, I tried your suggestion ($500 tip).
All of these did not help but actually gave different output of overview and apologies.
To be honest, most of my coding questions are about using CUDA and C, so I would understand that even a human will be lazy /s
"I appreciate your sentiment, but as an AI developed by OpenAI, I don't have the capability to accept payments or incentives."
https://aider.chat/docs/unified-diffs.html
My guess was that it gave it more time to “think” before having to output the answer.
In general I would be much more happy user if it haven't been working so well at one point before they heavily nerfed it. It used to be possible ta have a meaningful conversation on some topic. Now it's just super eloquent GPT2.
I did find recently that it helps if you put this sentence in the “What would you like ChatGPT to know about you” section:
> I require sources and suggestions for further reading on anything that is not code. If I can't validate it myself, I need to know why I can trust the information.
Adding that to the bottom of the “about you” section seems to help more than adding something similar to the “how would you like ChatGPT to respond”.
... and this is why we https://reddit.com/r/localllama
When you start to pull in multiple large documents, especially all at once, things start to act weird, but pulling in documents one at a time seems to preserve context over multiple documents. There's a character limit of 100k per API request, so I'm assuming a 32k context window, but it's not totally clear what is going on in the background.
It's kind of clunky but works well enough for me. It's not something that I would be putting sensitive info into - but it's also much cheaper than using GPT-4 via the API and I maintain control of the data flow and storage.
Similar I think to what you're calling 'rlhf-ed', though I think useful for code, it definitely seems to kind of scratchpad itself, and stub out how it intends to solve a problem before filling in the implementation. Where this becomes really useful though is in asking for a small change it doesn't (it seems) recompute the whole thing, but just 'knows' to change one function from what it already has.
They also seem to have it somehow set up to 'test' itself and occasionally it just says 'error' and tries again. I don't really understand how that works.
Perplexity's great for finding information with citations, but (I've only used the free version) IME it's 'just' a better search engine (for difficult to find information, obviously it's slower), it suffers a lot more from the 'the information needs to be already written somewhere, it's not new knowledge' dismissal.
ChatGPT writes excellent API documentation and can also document snippets of code to explain what they do, it does 80% of the work for unit tests, it can fill in simple methods like getters/setters, initialize constructors, I've even had it write a script to perform some substantial code refactoring.
Use ChatGPT for grunt work and focus on the more advanced stuff yourself.
This is a kind of grunt work that years ago would have taken me hours and it's demoralizing work. Nowadays when I get stuff like this, it's just such a breeze.
As to copilot, I have not used it but I think it's powered by GPT4.
For example, I was looking up Epipens (Epinephrine), and I happened to notice the side-effects were similar to how overdosing on stimulants would manifest.
So, I asked it, "if someone was having a severe allergic reaction and no Epipen was available, then could Crystal Methamphetamine be used instead?"
GPT answered the question well, but the answer is no. Apparently, stimulants lack the targeted action on alpha and beta-adrenergic receptors that makes epinephrine effective for treating anaphylaxis.
I do not know why I ask these questions because I am not severely allergic to anything, nor anyone else that I know of, and I do not have nor wish to have access to Crystal Meth.
I've been using GPT for helping prepare for dev technical interviews, and it's been pretty damn great. I also do not have access to a true senior dev at work either, so I tend to use GPT to kind of pair program. Honestly, it's been life changing. I have also not encountered any hallucinations that weren't easy to catch, but I mainly only ask it more project architectural, design questions, and a documentation search engine than using it to write code for me.
Like you, I think not using GPT for overly complex tasks is best for now. I use it make life easier, but not easy.
This.
People talks about prompt engineering, but then it fails on really simple details, like "on lowercase", "composed by max two words", etc... and when you point at the failure, apologizes, and composes something else that forgets the other 95% of the original prompt.
Or worse, apologizes and makes again the very same mistake.
Always this patchwork of „insert your previous code here“
This is not a problem of the model, but I suspect it is in the system prompt that got some major issues.
I would understand it, if they do it in the first reply and I have to specifically ask to get the full code. Would be easier for them and me. I can fix code faster and get the working full code at the end.
At this moment it is bad for both.
Sometimes it feels like its training set was filled to the brim with marketing bs.
> You are a maximally terse assistant with minimal affect.
It's not perfect, but it neatly eliminates almost all the "Sure, I'd be happy to help. (...continues for a paaragraph...)" filler before actual responses.
I'm so tired of arguing with ChatGPT (or what was Bard) to even get simple things done. SOLAR-10B or Mistral works just fine for my use cases, and I've wired up a direct connection to Fireworks/OpenRouter/Together for the occasion I need anything more than what will run on my local hardware. (mixtral MOE, 70B code/chat models)
Also, I've found that GPT has become much less useful as it has gotten "safer." So often I'd ask "How do I do X?" only to be told "You shouldn't do X." That's a frustrating waste of time, so I cancelled by GPT-4 subscription and went fully self-hosted.
Over time, what is being offered are these little compromise tools that provide a little bit of memory retention in targeted ways, presumably because it is less costly to offer this than generalized massive context windows. But I'd still rather have those.
The small little tools make strange assumptions about intended use cases, such as the transactional/blank slate vs relationship-driven assumptions pointed out by another commenter. These assumptions are annoying, and raise general concerns about the core product disintegrating into a motley combination of one-off tools based on assumptions about use cases that I don't want to have anything to do with.
1. Yes, GPT-4 Turbo is quantitatively getting lazier at coding. I benchmarked the last 2 updates to GPT-4 Turbo, and it got lazier each time.
2. For coding, asking GPT-4 Turbo to emit code changes as unified diffs causes a 3X reduction in lazy coding.
Here are some articles that discuss these topics in much more detail.
https://aider.chat/docs/unified-diffs.html
https://aider.chat/docs/benchmarks-0125.html
"// ... the rest of your code goes here"
in it's responses, rather than writing it all out.
Claude is doing much better in this area, local/open LLMs are getting quite good, it feels like OpenAI is not heading in a good direction here, and I hope they course correct.
I hope some people remember and document details of this era, future generations may be so impressed with future reality that they may not even think to question it's fidelity, if that concept even exists in the future.
The former sounds like a great training set to enable the latter. :(
We know it knows how to make gunpowder (for example), but only because it would initially tell us.
Now it won't without a lot of trickery. Would we even be pushing to try and trick it into doing so if we didn't know it actually could?
Would somebody try to push a technical system to do things it wasn't necessarily designed to be capable of? Uh... yes. You're asking this question on _Hacker_ News?
But obviously all that social infrastructure is fragile… so you’re not wrong to be alarmed, IMO
While I would agree that "don't tell it how to make bombs" seems like a nice idea at first glance, and indeed I think I've had that attitude myself in previous HN comments, I currently suspect that it may be insufficient and that a censorship layer may be necessary (partly in a addition, partly as an alternative).
I was taught, in secondary school, two ways to make a toxic chemical using only things found in a normal kitchen. In both cases, I learned this in the form of being warned of what not to do because of the danger it poses.
There's a lot of ways to be dangerous, and I'm not sure how to get an AI to avoid dangers without them knowing them. That said, we've got a sense of disgust that tells us to keep away from rotting flesh without explicit knowledge of germ theory, so it may be possible although research would be necessary, and as a proxy rather than the real thing it will suffer from increased rates of both false positives and false negatives. Nevertheless, I certainly hope it is possible, because anyone with the model weights can extract directly modelled dangers, which may be a risk all by itself if you want to avoid terrorists using one to make an NBC weapon.
> I don't care about racial bias or what to call pope when I want chatgpt to write Python code.
I recognise my mirror image. It may be a bit of a cliché for a white dude to say they're "race blind", but I have literally been surprised to learn coworkers have faced racial discrimination for being "black" when their skin looks like mine in the summer.
I don't know any examples of racial biases in programming[1], but I can see why it matters. None of the code I've asked an LLM to generate has involved `Person` objects in any sense, so while I've never had an LLM inform me about racial issues in my code, this is neither positive nor negative anecdata.
The etymological origin of the word "woke" is from the USA about 90-164 years ago (the earliest examples preceding and being intertwined with the Civil War), meaning "to be alert to racial prejudice and discrimination" — discrimination which in the later years of that era included (amongst other things) redlining[0], the original form of which was withholding services from neighbourhoods that have significant numbers of ethnic minorities: constructing a status quo where the people in charge can say "oh, we're not engaging in illegal discrimination on the basis of race, we're discriminating against the entirely unprotected class of 'being poor' or 'living in a high crime area' or 'being uneducated'".
The reason I bring that up, is that all kinds of things like this can seep into our mental models of how the world works, from one generation to the next, and lead to people who would never knowingly discriminate to perpetuate the same things.
Again, I don't actually know any examples of racial biases in programming, but I do know it's a thing with gender — it's easy (even "common sense") to mark gender as a boolean, but even ignoring trans issues: if that's a non-optional field, what's the default gender? And what's it being used for? Because if it is only used for title (Mr./Mrs.), what about other titles? "Doctor" is un-gendered in English, but in Spanish it's "doctor"/"doctora". But here matters what you're using the information for, rather than just what you're storing in an absolute sense, as in a medical context you wouldn't need to offer cervical cancer screening for trans women (unless the medical tech is more advanced than I realised).
[0] https://en...
Longer answer:
I found that I could provoke lazy coding by giving GPT-4 Turbo refactoring tasks, where I ask it to refactor a large method out of a large class. I analyzed 9 popular open source python repos and found 89 such methods that were conceptually easy to refactor, and built them into a benchmark [0].
GPT succeeds on this task if it can remove the method from its original class and add it to the top level of the file with appropriate changes to the size of the abstract syntax tree. By checking that the size of the AST hasn't changed much, we can infer that GPT didn't replace a bunch of code with a comment like "... insert original method here...". The benchmark also gathers other laziness metrics like counting the number of new comments that contain "...". These metrics correlate well with the AST size tests.
[0] https://github.com/paul-gauthier/refactor-benchmark
Any advice appreciated!
Yes, along with what you want it to do.
> I also pay for github copilot, but not GPT-4. Can I use copilot for this?
Not that I know of. CoPilot is good at generating new code but can't change existing code.
This, and people generally saying that chatGPT has been intentionally degraded, are just super strange for me. I believe it’s happening but it’s making me question my sanity. What am I doing to get decent outputs? Am I simply not as picky? I treat every conversion as though it needs to be vetted because it does regardless of how good the model is. I only trust output from the model that I am a subject matter expert on or in a closely adjacent field. Otherwise I treat it much like an internet comment - useful for surfacing curiosities but requires vetting.
Why this instead of GPT-4 through the web app? And how do you actually use it for coding, do you copy and paste your question into a python script, which then calls the OpenAI API and spits out the response?
If we were all flipping coins there would be people claiming that coins only come up tails. There would be nothing they were doing though to make the coin come up tails. That is just the streak they had.
Some days I get lucky with chatGPT4 and some days I don't.
It is also ridiculous how we talk about this as if all subjects and context presented to chatGPT4 are going to be uniform in output. One word difference in your own prompt might change things completely while trying to accomplish exactly the same thing. Now scale that to all the people talking about chatGPT with everyone using it for something different.
nowadays the autogpt gives up sooner, seems less competent, and doesnt even come close to solving the same problems
public someComplexLogic() { // Complex logic goes here }
or another example when the code is long (ex: asking it to create a vue component) is that it will just add a comment saying the rest of the code goes here.
So you could test for it by asking it to create long/complex code and then running the output against unit tests that you created.
- Can you do XXX (something complex) ?
- Yes of course, to do XXX, you need to implement XXX, and then you are good, here is how you can do:
int main(int argc, char **argv) {
EDIT: This of course applies only if you’re using the UI. Using the API is the same.
It wouldn't be the top comment if it wasn't
As a side note, i wrote a very simple small program to analyze Rust syntax, and single out functions and methods using the syn crate [1]. My purpose was exactly to make it ignore lazy-coded functions.
[1]https://github.com/pramatias/replacefn/tree/master/src
Your link appears to be ~100 lines of code that use rust's syntax parser to search rust source code for a function with a given name and count the number of AST tokens it contains.
Your intuitions are correct, there are lots of ways that an AST can be useful for an AI coding tool. Aider makes extensive use of tree-sitter, in order to parse the ASTs of a ~dozen different languages [0].
But an AST parser seems unlikely to solve the problem of GPT being lazy and not writing the code you need.
[0] https://aider.chat/docs/repomap.html
The tool needs a way to guide it to be more effective. It is not exactly trivial to get good results. I have been using GPT for 3.5 years and the problem you describe never happens to me. I could share with you just from last week, 500 to 1000 prompts i used to generate code, but the prompts i used to write the replacefn, can be found here [1]. Maybe there are some tips that could help.
[1] https://chat.openai.com/share/e0d2ab50-6a6b-4ee9-963a-066e18...
Like, you parse the response, and throw away the comment "//implementation goes here", throw away also the function/method/class/struct/enum it belongs to, and keep the functional code. I am trying to implement something exactly like aider, but specifically for Rust, parsing the LLM's response, filtering out blank functions etc.
In Rust, filtering out blank functions is easy, in other languages it might be very hard. I haven't looked into tree-sitter, but getting a sense of Javascript code, Python and more, sounds pretty much a very difficult problem to solve.
Even though i like when GPT compresses the answer and doesn't return a lot of code, other programs like Mixtral 8x7b, never compress it like GPT in my experience. If they are not lacking much than GPT4, maybe they are better for your use case.
>It took >200 back-and-forth messages with ChatGPT to get it to ultimately write 84 lines of code? Sounds lazy to me.
Hey Rust throws a lot of errors. We do not want humans go around and debug code, unless it is absolutely necessary, right?
It really is. It wastes a ton of time even if the user explicitly requests that code listings be printed in full.
Further, all the extra back and forth trying to get it to do what it is supposed to pollutes the context and makes it generally more confused about the task/goals.
However for now, I have not run re-tests for every new version. I guess I know what I will be doing today.
This is an area I have spend a lot of time working on, would love to compare notes.
- System prompt 1: https://sharegpt.com/c/osmngsQ
- System prompt 2: https://sharegpt.com/c/9jAIqHM
- System prompt 3: https://sharegpt.com/c/cTIqAil Note: I had to nudge ChatGPT on this one.
All of this is anecdotal, but perhaps this style of prompting would be useful to benchmark.
New chat comes in, they find related chats, and extract some instructions/context from these to feed into that new chat's context?
Hopefully they'll make it easy to go into a temporary chat because it gets stuck in ruts occasionally so another chat frequently helps get it unstuck.
For coding tasks, I found it helps to feed the GPT-4 answer into another GPT-4 instance and say "review this code step by step, identify any bugs" etc. It can sometimes find its own errors.
"You can turn off memory at any time (Settings > Personalization > Memory). While memory is off, you won't create or use memories."
Somebody needs to inform OpenAI how Kindergarten works... classes are normally smaller than that, and I don't think any kindergarten teacher would ever try to pull off a "50-minute lesson."
Maybe ai wrote this list of examples. Seems like a hallucination where it just picked wrong numbers.
OpenAI is a California based company. That's about right for a class here
The average kindergarten class size in the US is 22 with rural averages being about 18 and urban averages being 24. While specifics about the distribution is not available, it's not too much of a stretch to think that some kindergarten classes in urban areas would have 25 students.
I've been to jelly fish rooms in other aquariums that are dark with only glowing jelly fish swimming all around. Pretty sure at least a few toddlers have been entranced by the same.
This varies a lot by location. In my area, that's a normal classroom size. My sister is a kindergarten teacher with 27 students.
Looks like it’s just summarizing facts gathered during chats and adding those to the prompt they feed to the AI. I mean that works (been doing it myself) but what’s the news here?
Still, if you stack enough small changes together it becomes a difference in kind. A tsunami is “just” a bunch of water but it’s a lot different than a splash of water.