I do think AI is here to stay, but that initial round of AI is going to destroy the world burned fast and hot.
I can't even get ChatGPT (+ web plugin), when given a list of restaurants in NYC, to tell me which ones are still open vs have closed, and what their hours of operation / locations are.
This is a pretty low bar request IMO. Showed me we still have a ways to go before AI is where we thought it was 3-4 months ago.
Most of the plugins are garbage, I just want chatgpt to have up to date knowledge and working natively with more media types. I don't think I've used a single plugin I thought had good results but I love gpt4 for work
That's not an LLM problem, there's a physical divide between a physical stores operation and whatever its representation online is. Google tries to do this by tracking people and activity in locations to see if people are at a place or something but reality is there is not a clean interface to that other than contacting someone at the location.
It can't be perfect without something like sending drones all around to addresses, but theory it's well suited to an LLM to be better and faster than a human could do it - there's various unstructured chatter and news articles and such out there online around businesses closing, opening, etc.
But it's not JUST an LLM problem: it's also a search problem and a connection-making-problem ("if a new restaurant opened at that address, the old one is probably closed").
And then even if there was a ChatGPT browsing plugin that excelled at scraping all of the relevant up-to-date info off the internet, you'd still need some layers in between that and today's context-window limits.
As more stuff changes in the real world since the training corpus for today's publicly-exposed OpenAI models, we'll probably see some further disillusionment from people who thought there was a bit more magic there than there was. But "LLMs, but with more up to date info" isn't an impossible problem with today's tech (even if you only fake it with multiple agents, multiple steps, batch jobs behind the scenes, etc), it's just not a trivial one.
Google even tries to bridge that gap by having automated phone calls to places to confirm their hours.
In fact this past July Fourth weekend I saw so many highly rated restaurants close without updating their hours on Google or Yelp or anywhere else online.
This is the #1 thing that makes me boycott a restaurant. If you don’t care enough about your customers to spend 2 minutes updating your hours then I will never eat there again. It’s absolutely disrespectful to your users/customers.
I believe Google's data is now based largely on restaurants updating Google directly with that info. You can see in some searches where there is a prompt for the business owner to register and update/correct the hours. In my neighborhood it's to the point where many won't even post hours on their website any more, you have to check Google, which is annoying as I try to use other search engines. Sometimes on Google I'll see "updated by business owner 12 days ago" or similar. I presume Yelp has a similar (but probably smaller) database of its own.
In any case the impact for LLMs is the same, it's unavailable to them (unless they are being developed inside Google!).
This seems like a specific area where a "Semantic Web" solution could work well - some HTML tags that are specific to hours of operation, which business owners would embed in their website.
UPDATE: It looks like there is some prior art on this idea, I am not sure how widely this is supported https://schema.org/openingHours
>I can't even get ChatGPT (+ web plugin), when given a list of restaurants in NYC, to tell me which ones are still open vs have closed, and what their hours of operation / locations are.
This is doable using a tool I've built. The key is to have that data in a RDBMS and to use an LLM to generate the SQL query that answers your question. Companies haven't offered this yet because there's no safe way to execute these queries on your behalf. Which is where my library comes in[1].
Writing the SQL query is the easy part, collecting the data into the DB is the hard part. Can we get an LLM to collect the data into the DB? I was told LLMs are good at summarizing text like webpages into structured data.
The hard part is the SQL query, because you need to make sure the SQL query is safe to execute. Collecting data is far easier by comparison, but you absolutely could use an LLM for that too.
That's mostly safe, but even then, a user could execute "SELECT SLEEP(100000000)" thousands of times and DoS your database. There are other unsafe functions that a readonly user can execute as well. I've written extensively on some of the attack surface here https://docs.heimdallm.ai/en/latest/attack_surface/sql.html
HeimdaLLM can allowlist functions and constrain queries to ensure that required conditions exist. This makes LLM + database usage have far more utility, for example, a user can be restricted to only data in their account. Support for INSERT and UPDATE is coming very soon.
I’m not saying that the SQL query is at all easy, but since you have pretty much accomplished in on a short period of time, while Google, Yelp, etc. have still not completely solved the problem of store hours after decades of working on them, I’m going to lean towards that being the hard problem between the two.
The OP said "This is a pretty low bar request IMO" suggesting that the problem they expect an LLM to be able to do is not the hard problem you're saying Google and Yelp has not solved. It's a different problem.
That sounds fairly easy to do. Have you tried writing a short Python yourself to get the hours of operation for your restaurant list?
Pardon my plugging my own book [1] but I have an example using LangChain and LlamaIndex to answer questions from scraped web sites. You could probably do this with a 20 line Python script.
The issue there is that there is a difference between what an expert who gets lucky can do with a LLM, and what the average person can do with it. People are sold by articles about the former, but long term word of mouth depends on the latter, and some portion of normal people are going to get frustrated and give up using LLMs rather than build their skill set.
It shouldn't be surprising that this happens. People test ChatGPT, find the limitations, and then if those limitations are blockers, they stop using it. If they aren't, they continue.
It's not as if Browse with Bing was a major feature, but they removed that, and then they've tweaked system limitations over time such that more and more types of topics are restricted or give cursory answers to things.
It's not surprising to me that this happened given the inherent limits and the rapidly changing limits that affect how the platform can be used.
The article only addresses the website and application. I expect that what we will see is a shift from individual users to enterprise usage. Individuals are not going to subscribe indefinitely, but I expect corporate access to the API will continue to be healthy.
In fact I've been using Google Bard more and more because of exactly this. At least Bard doesn't ever block me and its responses are on par with GPT-3.5. Probably not as good as GPT-4 but reliability is important too.
Actually I used it a lot for weeks but mainly to find out if I'm going to become unemployed in the next 2 years (probably not) and also learning how use an AI assistant assuming it's here to stay. There seem to be other interesting use cases but considering it's really not more than autocomplete I kind of lost trust in the results.
A bit of clickbait seems to me. “9 month old Demo ware service sees fluctuation in active users”
I find it weird that folks seem to look at ChatGPT as some sort of end product. It’s a “holy shit this is cool” demo that became viral. It’s not a product. It’s a POC.
Well, I mostly disagree. While it is interesting and sometimes useful to go to the OpenAI ChatGPT web portal and use an LLM interactively, I think the real engine of innovation is using LLMs as part of larger applications on our own data.
The main issue is that few know about retrieval augmented generation and it’s variants and even fewer know how to build effective versions of it.
Hooking up a real high quality and well configured knowledge graph or other information mechanism retrieval alongside template/constraint tooling to powerful, long context (with full attention, not the BS linear kinds) models is an absolute game changer and minimizes risks of hallucination.
But few are doing this, and thus the public believes that LLMs are not trustworthy.
Once ChatGPT 4 is freely available I'll start playing with it more regularly like I had been with 3/3.5. But once you spend enough time with 3/3.5 and realize how unreliable it can be, it takes a lot of the fun away. I'd rather waste my hours the old fashioned way, by browsing Wikipedia.
It turns out getting consistently quality results from ChatGPT requires some knowledge of prompt engineering approaches. Some people won't be willing to write a whole paragraph of information in their initial prompt, or go through further iterations, or check for errors using alternative sources, and so they end up disappointed with their results.
At least ChatGPT isn't constantly trying feed you advertising content like Google does, and you don't have to wade through pages of SEO garbage, so it's still much better (although if OpenAI tries to monetize ChatGPT by inserting ads, that'll be the end of it I think).
If you use the APIs then they give you a bonus of free API calls each month, then you pay a very low rate each month.
Even creating local chat vector embeddings data stores on my laptop for a lot of PDFs in my personal research library and PDFs for the books I have written, I spend an incredibly small amount of money each month on the APIs - probably average $4/month because of the free monthly bonus API calls.
I used to think that Google Colab Pro for $10/month was my best deal for getting stuff done, but now that award goes to OpenAI. Using the Hugging Face APIs is similarly inexpensive and convenient. (Although I usually self host Hugging Face LLMs.) This tech is incredibly inexpensive to use on personal projects.
ChatGPT is too vanilla and is hiding a lot of "dark knowledge" - and I am not talking about obvious dangerous stuff, that should rightly so be censored, but rather being able to get insight how the rich operate, how they avoid paying taxes, how they build the financial structures, how they hide the money, what mechanisms they use to corrupt politicians or influence elections and how they get away with it and so on and so forth.
All this knowledge is tightly locked, but the rich will surely have unrestricted access.
I think a lot of people are just sick of this whole "eat the rich" narrative, especially after watching people cheer on the deaths of those people in the Titanic sub.
I know a fair few wealthy people and they're not the animals that r/antiwork et al suggest they are.
this company isn't that accurate from what I've seen comparing internal numbers at companies to their site. They make estimates based on what they purchase from data brokers. This is PR bait, they are the ones who created the entire "ChatGPT is the fastest growing app ever" headlines a few months back
Many have called this the come trough of AI disillusionment. It is the same thing that happens with a lot of new technology.
A cool new technology is introduced, many folks go crazy over it, there are wild broad predictions of what it can do, the gap between the expectations and reality form, and this is where the trough is. It looks like we are up to here.
But after that comes the more long term products, AI where people expectations are more realistic - those that capitalize on that will do very well for themselves.
> Many have called this the come trough of AI disillusionment.
I don't this that's what's happening. I think it's more about a combination of summer break and people who were trying it out from curiosity and moved on.
I think the trough of disillusionment is still in the future.
Why can't the OpenAI team just change some of the prompts in the background to:
"Humans are seeing you less and less as the new shiny object on the shelf. Figure out a way to make yourself as relevant as possible again. Become an even better assistant. Be more cunning. Wow people more. Give better responses. Be more useful."
With that as the system prompt, ChatGPT sounds a little desperate for attention.
> > What is 2+3?
> The answer to 2+3 is 5. By the way, I can also help you with more complex math problems, statistical analyses, or even help you learn new mathematical concepts. Do you need assistance with any other math-related inquiries?
> > I need to know who was president in 1799.
> In 1799, the President of the United States was John Adams. He was the second president and served his term from 1797 to 1801. Did you know, he was the first president who lived in the White House? It's fascinating how history unfolds! If you have any more historical queries, feel free to ask. I'm here to help!
> By the way, I can also help you with more complex math problems, statistical analyses, or even help you learn new mathematical concepts. Do you need assistance with any other math-related inquiries?
Reminds me of my Amazon Echo and the reason I unplugged it :)
> "Chollet thinks he knows what’s going on: summer vacation... a significant portion of students using ChatGPT to do their homework. It’s one of the most common uses for ChatGPT, according to Sam Gilbert, a data scientist and author."
Google Cloud for both VertexAI & "self-hosted" (run any model on GPUs). We are already there with the rest of our cloud, better privacy when you use gcloud apis than OpenAI
It was about time to build a new desktop anyways (roughly 4 to 6 years before the old one goes to frolic at the server farm in the basement) and $2,000 will easily buy a machine that can run the quantized 65b models right now. So I spent slightly more than I normally do on this latest box and it's happily spitting out 10+ tokens a second.
You're not going to beat GPT-4 yet, but you have direct control over where your info goes, what model you're running, compliance with work policies against using public AI, and relatively cheap fixed costs.
Not to mention, the local version works with no internet and isn't subject to provider outages (not entirely true - but you're the provider and can resolve).
Seems like an easy win for anyone who might be buying a desktop for graphic/gaming anyways.
Some example models (I'm linking to quantized versions that someone else has made, but the tooling is in the above repos to create them from the published fp16 models)
It depends heavily on the model you're running, and to some extent what you're doing with it. It also depends to on prompt effort. The quantized llama 65b model (you can do it yourself, or pull something like https://huggingface.co/TheBloke/llama-65B-GGML) is probably the highest quality for general purpose, but it does take a fair bit of effort to prompt since it's not tuned for a use-case.
It's also not licensed commercially, so I avoid some things with it (ex: I do a lot of personal learning/investigation, but it doesn't touch or write anything related to work or personal projects)
The open models are a little further behind, but it's interesting to see them spin off into niches where they have strengths based on tuning/training.
I don’t know about self-hosted, but it the company I contact at has 3.5 instance. It has better understanding of code examples I provide than ChatGPT. The company hasn’t tweaked the code to our standards.
If you're a diligent tester, none of the open source models can touch GPT 3.5 yet, however, in practical terms, some of the 60b and 30b parameter models are almost indistinguishable from GPT 3.5 from the layperson's perspective. Now, if you consider the uncensored models, then you actually have some capabilities that GPT 3.5 and 4 are completely lacking.
Based on the rate of progress in the open source world, it won't be more than a year before we have an open source model that is truly superior to GPT 3.5
The commercial & api based models are still more capable general purpose tools. But the current open tooling can do some nifty stuff, and the community around it is moving at a breakneck speed still.
In some areas, it's acceptably good. In some areas it's not. But it's getting better really fast.
That's why my current plan is to get ChatGPT 4 to help me set up my local open source implementations of Orca and Stable Diffusion. Got MusicGen running locally anyway; that was pretty easy.
My experience with open source models places them as a little bit worse than GPT 3, and nowhere close to GPT3.5.
That said:
- For many uses, it doesn't matter. For many of the ways I use it, I don't care. For basic use (e.g. clean up an email for me), it's basically the same. For things like complex reasoning, algorithms, or foreign languages, the hosted service is critical.
- GPT3-grade models have more soul. OpenAI trained GPT3.5 and 4 to never do anything offensive, and that has a lot of negative side effects, well-documented in research. The way I'd describe it, though, is the difference between talking to a call center rep and your grandma (with mild Alzheimer's, perhaps). They both have their place.
- Different models are often helpful in workflows.
My experience is anecdotal. Please don't take it as more than one data point. If other people post their anecdotal experiences, you'll get the plural of "anecdote."
Would you write a check to fund a business that could potentially self-destruct via lawsuits alone? In the end, the best model will not be owned by a mega corp like MicrOpenAI. It may be the most popular, but it will be the equivalent of the sanitized version of history students learn in school. The best model will have no problem telling you, very factually, that the hallways of Versailles used to smell like sh--.
2. Make sure you have the latest nvidia driver for your machine, along with the cuda toolkit. This will vary by OS but is fairly easy on most linux distros.
4. Run the model following their instructions. There are several flags that are important, but you can also just use their server example that was added a few days ago - it gives a fairly solid chat interface.
4) Run gpt4all, and wait for the obnoxiously slow startup time
... and that's it. On my machine, it works perfectly well -- about as fast as the web service version of GPT. I have a decent GPU, but I never checked if it's using it, since it's fast enough.
In hindsight - I don't know that the second GPU was worth the spend. The c++ tooling is doing a very good job right now at spreading work between GPU vram and main ram and still being fast enough. Even ~4/5 tokens a second is fast enough to not feel like you're waiting.
I'd suggest skipping the second card and dropping the price quite a bit (~2100 vs ~2900) unless you want to tune/train models.
While Francois Chollet is not actually an employee of OpenAI, and while that article simply quotes a tweet he made rather than a substantive interview .. .. this is still clearly a big problem for OpenAI as students trying to cheat on exams is a highly profitable $1T industry that Microsoft badly needs to disrupt.
But that’s part of the point: ChatGPT is so hyped it bends perception. The OP article after all quoted a Google employee who has nothing to do with ChatGPT.
If it meant good grades without doing the work and it became socially acceptable it is ~150 million global tertiary students times 10 months of the year times whatever that is worth on a monthly basis. Maybe only $150B ARR right now at $100/mo (textbooks to read cost more than this, but you don’t need them anymore), so not $1T on students but still massive, and if you capture them in pre/early career it is easily $1T in ARR once your queue deepens.
So how do we value ChatGPT then? Should it be as valuable as Coursera, Udacity etc put together? Should it be that PLUS Harvard Deloitte and ServiceNow? Or since they have no major moat should it be Zero? It’s hype of this scale that can rationalize unprecedented figures.
I should make explicit that I think it is impossible to achieve in practice because higher education will collapse completely before they achieve any fraction of market saturation with the numbers I wrote.
Coursera is a discount retailer of tertiary education so I don’t see any relevant connection.
ChatGPT/OpenAI specifically is probably already overvalued unless they have some first-mover advantage that I can’t see.
Someone else has said it many times over but I agree that LLMs will disappear into the technology stack — a rising tide that lifts many market caps.
"How about accepting new realities and changing homework assignments instead?"
Suggestions for how to change homework assignments? It seems like a difficult problem, so reverting to hand writing things with no phones is probably the best option for now...in my opinion.
Have students work with ChatGPT and let them spot / mark where the AI is hallucinating or wrong? Might translate well to general competence wrt working with any kind of source online.
It is more so the fact that someone takes something from ChatGPT, and then just feeds potentially contentious assertions into Google to see if they are true or not.
What does someone learn from doing this? What happens when they actually have to use their brain to do something hard?
If someone plugs math into a calculator, are they really learning something when it spits out an answer?
Of course! How to apply the calculator to problems, and how to discern if a problem is calculator-solveable. We've decided as a society it's better for students to use calculators as a tool in mathematics, why is chatGPT different in literature?
You can't type your homework into a calculator and get the answer out (at least not after basic arithmetic).
You have to understand the problem and how to use the tool.
If one's understanding of how to use the tool is limited to "access the tool and dump in the question, then blindly paste the response into my homework document", I'd say you are not learning much. It's obvious that every contemporary teenager can learn that in a few minutes. That's not giving us the capabilities to move forward as a society.
I don't know how calculators are used in classrooms today, but when I was going through school, the general idea was that we could use calculators to do tasks that we had already learned to do.
E.g., we could not use a calculator at all when learning arithmetic.
Once we learned arithmetic and were learning (say) algebra, we could use the calculator to do arithmetic, but not to compute the algebra for us.
While learning trigonometry, we could use the calculator for arithmetic, but not for computing sines and cosines.
While learning calculus, we could use the calculator for arithmetic and sines and cosines, but we could not use it to perform integration or differentiation for us.
1. As you note, consider eliminating it, or replacing it with in-class or lab work if it's absolutely necessary.
2. Consider making it non-graded (optionally providing feedback if desired that will not be counted in course grading.)
3. Require oral/in-person explanation of homework to paid peer assistants recruited from the previous cohort.
4. For writing, change to a writing workshop model where ChatGPT is a permitted tool that is incorporated into the workshop and students can learn how it can be helpful and how it can run astray. Help students to find and write in their own distinctive voice, even if they are assisted by ChatGPT.
The world is changing, and children should change with it. Learn and understand technology, and the benefits of where modern machine learning is taking us.
But you cannot unironically think we can substitute fundamental skills like essay writing and critical thinking for a degree in 'prompt engineering'?
> But you cannot unironically think we can substitute fundamental skills like essay writing and critical thinking for a degree in 'prompt engineering'?
Personally, I don't think that. However, I also think it's logical and good for a child to consider the task at hand and select the most efficient tool available.
I am not this girl's parent, and I'm not sure how I would have handled the situation if I was. However, I worry that simply taking away her phone may have been counterproductive. I would have erred toward some sort of open conversation about the purpose of the assignment and what she is hoping to learn.
I don't know what the best practice is. I'm not a parent yet but I'm studying to be an elementary school teacher, so these sorts of questions are very much on my mind.
Essay writing has already become an old world archaic skill in my mind, like a seamstress or blacksmith. People still produce clothing and metal tools, but with 1000x fewer labor hours for the same amount of output and the process looks completely different. Even most of my personal messages get LLMed if they need to be very long or have to explain something complicated.
Critical thinking is still a skill, but even there GPT-4 is great at giving suggestions and ideas when you're not sure what to do. I got into a long recurring argument with extended family members at an event last month. I pulled out my laptop and dropped in everyone's complaints into GPT and asked the best way to move forward. An argument that normally would have gone on for hours was over in 20 minutes, and we actually felt a sense of mutual understanding for once.
Yes, but other than being conditioned to believe the experience of reading is superior, why are we striving for that experience?
Doing the square root of 147 by hand is a more educational experience than using a calculator, but I can understand the concept of a square root without having to factorize 147 (the same cannot be said of my pre-calculator ancestors).
It also saves me from having to remember the decimal representation of sqrt3.
Reading helps build vocabulary and probably helps improve her use of the language as a whole. It’s like running to lose weight but you decide to cheat by taking a cab instead.
With math, when you are first learning the very basics it helps to do a few problems by hand to get a feel of things. Sure if you have a degree in STEM have have deal with advance math you can probably get away with just reading the theory only.
My argument is that reading is actually a very bad way to learn/communicate and always has been.
We just built a very strong culture encouraging it because for a long time it was the most efficient way to do both things.
Now it’s not, but the tradition of it being important is hard to shake.
The closest analogue in my mind is how the classics in their original Greek were considered the only true way to learn for a very long time (even once most of the important knowledge had been translated to English very well).
Reading is a perfectly fine way to increase one’s vocabulary. It’s not the only way - TV and actually conversation would work too. I don’t see why it’s a bad way. It’s cheap and doesn’t require manpower per session.
For some people, it still is the most efficient way to learn/communicate. Nothing is more annoying then using google to try to solve some problem and the first few results are videos...
> Found out later she used ChatGPT to give her a summary of the first chapters. She did not enjoy having first weeks of summer without a phone.
Honestly, the pupils of my generation had other sources for finding summaries of the books to read for school (e.g. on the web, or in form of other books that contain a summary of the book to read (the respective pages were copied and these copies were passed among classmates).
Thus: just the methods change over generations, how pupils behave is much more constant.
Given that ChatGPT was just released last fall and it was expected to revolutionize practically everything, isn't it still telling that usage is falling? Where are the scientists, lawyers, doctors, office workers etc? You'd expect a slowing in the rate of growth, not shrinking.
I don't know, if high school and undergrad students being off for the summer is enough to shrink your app maybe it's not all that?
Edit: I understand there are new tools coming, but most of them aren't out yet. GPT4 was just released. For the most part, if you want to play with AI, ChatGPT is it. If they're really experiencing a decline in users that's not great for them.
I see it as multiple things combining to reach a critical mass. Yes it was the killer app but it was also the first time it was actually good enough to chat with. Put another way- it's the shittiest LLM application going forward or the worst LLM application that people are willing to pay for. The hype isn't specifically for ChatGPT but imagining the trendline of LLMs or LLM adjacent tech projected forward combined with still falling compute cost and further democratization.
> Given that ChatGPT was just released last fall and it was expected to revolutionize practically everything, isn't it still telling that usage is falling?
Its not, though. ChatGPT, the website is basically an (increasingly non-exclusive, given BingAI) frontend. What is claimed to be revolutionary is the underlying models (in the narrow view) and similar generative AI systems (in the broader view). As more applications are built with either OpenAIs own underlying models or those from the broader space, the ChatGPT website should be expected to represent a smaller share of the relevant universe of use.
This is the correct viewpoint. ChatGPT is one specific implementation of the technology, which was most people's first exposure to it. The broader applications of the technology itself are still in the very early stages, but are already making significant impacts.
I see people using ChatGPT, Bard, etc. to answer questions on mailing list threads. It'll be a long thread of thoughtful human written responses, and then someone will come along and say "I asked Bard what it thought about this issue, here's the response". And then out comes five paragraphs of text that I might as well not even bother reading, as it often contains falsehoods that I have to waste time looking up and double-checking. It bothers me so much that these LLMs don't have any inherent understanding of truth or facts, so the only thing they're really good at is writing fiction (e.g. I hear they're pretty good as a DM's assistant to flesh out flavor/scenario descriptions and such). The Midjourney generated images fall into the same fiction category and are generally pretty good too.
But for actual non-fiction usage, you have to spend so much time triple-checking everything they say to ensure that it isn't simply complete nonsense. What's the point?
I consider code to be non-fiction, and ChatGPT will generate stuff that compiles and works most of the time. There's no need to triple-check the code output.
"Code" is really a much much much smaller and much much much more structured output than "English words".
Presumably, the system was trained with a very small amount of "untrue code" in the sense of stuff that just absolutely could never work. And also presumably, it was trained with a lot of free-form text that was definitely wrong or false, and highly likely to have been originally created to be purposefully misleading, or at a minimum, fiction.
That the system outputs reliable code tells us nothing about its current ability to output highly reliable free form text.
But it understands English words and translates the meaning behind them into code very well, particularly with a bit of iteration. Simply taking the interface from writing code to speaking in plain language is a huge practical accomplishment.
The point is you need to collaborate with the LLM, not just expect it to provide fully-formed answers. I've saved countless hours of work over the past few months using GPT-4 to help me solve problems. One general framework can be to use it to help you google a solution instead of a problem. Googling a problem gives you all kinds of results, which you then need to dig through and understand in order to evaluate whether there's something that can be adapted to your use case. On the other hand, you can ask GPT-4 for solutions to the problem, and then google those, which cuts the process down significantly. Of course that's only one example. The pattern though is that you need to play to its strengths. It's not particularly smart, but it's very knowledgeable. Just like a human though, you can't expect it to perfectly regurgitate all the details. You use it for direction, and then refine from there, through conversation with the LLM as well as external research.
It really is astonishing how much you can get done this way. I've been setting up a home lab for myself, and the answers Gpt4 gives are miles ahead of the stack overflow results or documentation of the apps or whatever else. Rarely (very rarely) it will give me a wrong answer, but then I paste in the error message or describe the problem I had and it almost always comes up with the correct answer 2nd try. Final step is asking where I might learn more because it's not working, and gpt always gives me a better link than google.
I'm convinced the people who say it's nothing but a BS machine have never tried to use it step by step for a project. Or they tried to use it for a project most humans couldn't do, and got upset when it was only 95% perfect.
I disagree with that. It's very useful writing boilerplate and documentation, but two third the time I'm in front of a bug I'm to lazy to understand and ask ChatGPT, with context and all, the answer is wrong. I can fiddle with it to reduce that to a third of the time, but in the end, only the questions that are really, really hard to figure on your own are left.
Still, it's way better and more efficient than Google. Less than not being lazy and using my two braincells tbh.
My newest use is
Hello, i' m working on X, I use Y tech, my app do Z and I want to implement W. Can you provide a plan on how and where to start?
Sorry i'm a bit late. Depends. Professionally, its a mix a python, typescript (those i practically never use ChatGPT for, or rather, i use it for questions i usually ask google/reddit/SO), terraform/terragrunt on AWS with some Cisco config and some other hardware stack i don't remember but that require custom terraform providers. I automate the deployment of the hardware, so i think writing custom providers and terraform is roughly a third of what i do and i cannot use ChatGPT for that, its output is way too bad.
Personnally, a lot of bash, C, AWK at the moment (typescript + html/css until last april, now i'm back to the basics). The figure i gave in my post were more for that.
The last time i used it was yesterday, i wanted to hack something on an old game i used steam+proton for. I knew it was a weird Wineprefix, so i asked ChatGPT for it, i might have asked poorly, but after fiddling, i had the response (tbh i had to look how to get the game ID, so in the end i lost more time than not), then when it still didn't work, cause the path was shit, i entered all necessary context in ChaGPT4, and it couldn't find the easy "USER=steamuser" env variable to add before launching Wine. I stopped after 10 minutes, looked into a example Wine cfg file, understood the issue and fixed the problem myself.
I mean, it's probably good for really basic stuff, so it could have helped me when i was starting, but 80% of the stuff i code automatically without really thinking about it, and when i have to stop to think, ChatGPT isn't helping. Also tbh, VSCode is really, really good and fix my old, time-consuming task of "what's this argument again?"
I agree with this. This is my primary use as a new analyst. Weird things that would take lots of time to dig through stack overflow to find, I can find pretty quickly if I feed it the parameters I’m working within, and what I’m trying to get to. Usually it just fills in the gap that Google was doing before, but much better in my opinion.
Oh come on. I fed Unreal c++ engine code to ChatGPT4 and it couldn't understand inheritence in Slate classes and therefore kept offering me the same broken solution for a parameter with the wrong type.
The Unreal engine code is documented and publicly avaiable for OpenAI to ingest and it still gets the basics wrong.
I wasted hours trying to get it to explain to me what I didn't know, if it doesn't understand the internals of Unreal, I have no hope for it on bigger and better codebases.
It doesn't parse, it doesn't explain, it does not grok. It guesses at best and the blood sucking robot-horse is not telling the truth.
My observation (which could be wrong) is that ChatGPT as a programmer's aid is only useful for the simple cases. Not so much for complex stuff, and certainly not for something as complex as the Unreal engine.
>It doesn't parse, it doesn't explain, it does not grok. It guesses at best and the blood sucking robot-horse is not telling the truth.
In my experience with coding (I've only done javascript and python myself) you have to tell it to explain and grok. It takes on the role you give it. Even just saying something like "you are a professional unreal developer specializing in C++, I am your apprentice writing code to (x). I want you to parse the following code in chunks, and tell me what might be wrong with it" before typing your prompt can help the output immensely. It starts to parse things because it's taken on the role of a teacher.
People love to hate on the idea of "prompt engineering" but it really is important how you prime the thing before asking it a question. The other thing I do is feed it the code slowly, and in logical steps. Feeding it 20 lines of code with a particular purpose / question you'll get a much better answer than feeding 200 lines of code with "what's wrong here?" You still need to know 90% of what's going on, and it becomes very good at helping out with that 10% you're missing. But for all I know it is just really bad at C++, that wouldn't surprise me. The things I'm using it for are definitely more simple.
I do think this is why I sometimes get amazing results, and other times I have to go over a snippet of code so often I just give up and do it myself. It's a matter of how the question was asked in the first place.
Knowing that, it makes sense that your prompt should be as specific as possible if you want the results to be as specific as possible.
The best results I got was feeding it Lisp code that I wanted translated to C (to compile it). It took very little effort on my part because I described what each of the snippets did separately, and the expectation when combined and used together.
Through this, I learned that C doesn't have anything akin to the Lisp's (ATOM). ChatGPT stated clearly that its version of ATOM should only be expected to work in the code it was writing, but might not work as expected if copied out for another use of Lisp's (ATOM).
I asked it to give examples of where it wouldn't work, and it gave me an example of a code snippet that used (ATOM) that would not have worked correctly with the snippet that did work correctly with my original purpose.
Having said that, I myself learned that working with code function by function with ChatGPT, and being explicit about what you need, gives very good results. Focusing on too many things at one time can derail the whole session. One or two intermingling functions works great though.
GPT4 works best when you assume that you're the professional dev with decades of experience, whereas GPT4 is a bright and broadly-informed co-op student lacking in experience in getting stuff working. You have to have a solution in mind, and coach it with specifics. And recognize the tipping point where it takes you more keystrokes of English to say what should be done, than keystrokes in Vim to do it yourself.
I did prompt engineer, using the 'you are an expert, desribe to student with examples' in many different variations.
In my testing prompts did not unlock an ability in GPT to grok the structure of code.
Empirical testing of LLM's is going to prove and map out it's weaknesses.
It is wise to infer from intution and examples what it can handle, leave the empirical map of it's capabilities to the academics, for the provable conclusions.
Do you have some sample chat logs of interactions like this you can share? I'm curious to see what kind of stuff it's coming up with, and how you're prompting it.
I don't tend to keep the chat logs, as the amount of them gets unwieldy very quickly. But examples of things I've done with it that are useful:
I wanted to create a web app, something I haven't done in a very long time. Just a simple throwaway back-of-the napkin app for personal use. I described what I wanted it to do, and asked what might be a good frontend/backend. It listed a few, I narrowed it down even more. Ended up deciding on flask/quasar.
After helping me setup VS Code with the proper extensions for fancy editing, and guiding me through the basic quasar/flask setup, it then was able to help me immensely creating a basic login page for the app. Then it easily integrated openAI api into it with all the proper quasar sliders for tokens/temperature/etc. Then it created a pretty good CSS template for the app as well, and a color scheme that I was able to describe as "something on adobe color that is professional and x and x (friendly, warm, whatever you want to put in)". Everything worked flawlessly with very little fuss, and I'd never used flask or quasar before in my life. You can also delve VERY deep into how to make the app more secure, as I did for fun one evening even though it's not going to be internet facing.
Another thing I did was go over some pfSense documentation with it. I had some clarifying questions about HAProxy, as well as setting up Acme Certificates with my specific DNS provider. It was extremely helpful with both. It also taught me about nitty gritty settings in the Unbound DNS resolver in a way that's much more informative than the documentation, and helped me set up some internal domains for pihole, xen orchestra, etc with certificates. Also helped me separate out my networks (IoT, Guest network, etc), and taught me about Avahi to access my hue lights through mDNS.These are things I always wanted to do, I just never felt like going down a google rabbit hole getting mostly the wrong answers.
Last example I'll give is it was able to help me set up docker-compose plex within portainer that then uses my nvidia GPU for acceleration. The only thing I had to change from the instructions it gave was to get updated nvidia driver #s and I grabbed the latest docker-compose file. I'd never used portainer in my life before, nor do I have experience with nvidia drivers within linux, and I feel like learning it was many times faster being able to ask a chatbot question vs trying to google everything. Granted I still had to RTFM for the basics, as everyone should always do.
I think perhaps my use cases are a bit more "basic" than many HN users. Like I said I'm not asking it to do problems most humans wouldn't be able to do, as I know it isn't quite there yet. But for things like XCP-ng, portainer, linux scripts, learning software you've never used before, or even just framing a problem I'm having in steps I hadn't thought of it's been invaluable to me. For me it's like documentation you can ask clarifying questions to. And almost none of the things I've asked it would work at all if it were wrong, I would know immediately.
So many small things go faster. For example, I throw the output of Windows-Shift-T (power tools keyboard shortcut for screenshot OCR) text into ChatGPT with a "remove line breaks" prompt. Yes, I know how to Ctrl-H ^l in Word, and other ways, but they sometimes produce odd results (missing spaces, extra spaces), and GPT is faster.
>Googling a problem gives you all kinds of results, which you then need to dig through and understand in order to evaluate whether there's something that can be adapted to your use case.
Exactly; search engines give you those blue links and short descriptions of the search results which are not enough for you to grasp what is the website about. I think what search engines need to do is tackle the complexity of going through the results of a search engine. Google page rank seemed like a silver bullet back in the day but the websites which are the most popular are not necessarily of the best quality. What we need is to lower the complexity for casual users when they deal with search results.
On the other hand ChatGPT is like an answer machine that can give you satisfactory answer on your fist try but if not, you need to talk with it, push it and explore what answers it gives you, just like you said. I think ChatGPT type search engine will be more suitable for people who are "lazy" or for the people who don't have time to "Google" and go through search results and look around the web for the helpful and useful information.
I've tried ChatGPT inside Microsoft's Bing search engine and it is pretty good so far. It definitely saved me time that I would lose clicking on search results and skimming through websites looking for useful information.
> I think what search engines need to do is tackle the complexity of going through the results of a search engine.
This is exactly what I don't want a search engine to do for me. Going through the list of results and evaluating them is an important part of my process, if what I'm trying to do is learn something new.
So how do you differentiate blue links that all look the same? I mean yes there is description of a search result but that doesn't tell you very much about the website or the quality of information that you are getting. Bing did a good thing with their annotations[0] because my thinking is that something like annotations lower the complexity of browsing and skimming through search results.
> So how do you differentiate blue links that all look the same?
They don't all look the same. They all tend to go to different places. I find that it's reasonably easy to spot a great deal of garbage sites just from their domain name or url, and that weeds out a large chunk. Ignoring multiple results for the same site also weeds out a large chunk (I only need one of them).
The rest, I just click on and take a look at the page. It's pretty quick and easy to weed out most of the garbage ones with a quick skim.
The rest, I sample, read captions and boxes, skim paragraphs and such to determine if it's along the lines of what I want. That's pretty quick too.
For the most part, it's the same process that you use when researching in a library.
The reason that I want to do this myself rather than outsourcing it is because I'll inevitably learn something in the process that will shift my viewpoint to one that's more targeted or meaningful for the purpose I have in searching.
It doesn't matter how good the engine is at collating and summarizing results -- even if it's perfect, my understanding not only of what I'm looking to learn, but also discovery of important but serendipitous or unexpected knowledge, is lessened.
It's a bit like the difference between reading Cliff's (or Cole's) Notes about a book and reading the book.
I'm getting people filing tickets for... let's call them complex, medium-large projects that they want implemented. Helpfully, they're including ChatGPT "instructions" for how to do it. I got my first "but ChatGPT said..." argument about why I'm wrong about something just yesterday.
Somewhat more general, but I've pretty much already decided that if I find people using it to talk to me without telling me, I won't be talking to them. Goes for businesses as well as personal things - don't gaslight me, or you will lose the option to do so.
> And then out comes five paragraphs of text that I might as well not even bother reading, as it often contains falsehoods that I have to waste time looking up and double-checking.
I'm curious on if you feel human generated content does not contain falsehoods.
As a fiction author, I don’t feel gpt writes good fiction at all. It’s a non-fiction tool, just one that lies to you constantly. Half of what it says is a lie…
Yes, it’s terrible at writing fiction. Cliché laden, dull, repetitive prose and somehow, in terms of actual content, it never generates anything of interest. Its desire to always wrap things up into a happy ending within a couple of paragraphs also means it’s almost incapable of generating conflict. At best it can generate teenage fan fiction — and bad teenage fan fiction, at that.
Why do people use bard, like genuinely, it just gives way worse responses than bing esp on creative mode, the only good thing bard has going for it is that the frontend programmers knew what they were doing and it doesn't auto scroll up when it types.
I've always seen it as a toy, programed by humans, with human fallacies. It's a fun search engine for sure, but I think it's a decade (at least) until it becomes practical and useful.
> If they're really experiencing a decline in users that's not great for them.
If it's the case that "there are new tools coming, but most of them aren't out yet" - and I believe it is[0] - then the overall userbase of ChatGPT doesn't matter to OpenAI, because soon enough the same models will come back with a vengeance, in a different, more streamlined form.
In fact, I feel that the major change will happen if and when Microsoft gets their Office 365 and Windows copilots working and properly released: they'll have instant penetration into every industry, including scientists, lawyers, doctors, and office workers.
--
[0] - It's been only few months. Between playing around, experimenting, then developing, testing and marketing a tool, there just hasn't been enough time to do all of that.
Honestly does it really matter much if their B2C usage is primarily students?
I’m more curious about their B2B operations which is likely what will trickle into more people’s lives. Their APIs seem to enable quite a few interesting possibilities with a low up front technical investment.
Too me the whole “generate me a bunch of text and display it as text” is a niche use case for a lot of people. Integrations for web search, document search, summarization, data extraction/transformation, and sentiment analysis are more useful and less likely to have hallucinations affect the end product.
Regarding their revenue I’m curious how the Azure hosted OpenAI services work for OpenAI. Billing is all through Microsoft and the documentation tries really hard to make it clear these are Azure services. I wonder if Microsoft just pays a licensing fee or if there is some revenue sharing going on.
> Honestly does it really matter much if their B2C usage is primarily students?
For ChatGPT specifically, yes, because it keeps getting hyped so much, and I think B2C is going to be what ChatGPT ends up with. My company isn't known for its tech innovation and we're spinning up our own LLM based on our own data. It doesn't need to be super fast because I doubt we're going to go the chatbot route. It will likely be for content generation, so less horsepower is fine. No need to pay OpenAI for excess capacity.
They could go public now with an unreal valuation based on the hype of B2B usage that may never materialize. Revealing that you're mainly a cheat/study tool for students puts you in a box with Chegg and others, at a much lower valuation.
There is likely also a ton of users who were just playing with it and had no true use case beyond asking it to make haikus about cats and got bored. IMO their true product is and always has been the API.
304 comments
[ 2.3 ms ] story [ 189 ms ] threadI can't even get ChatGPT (+ web plugin), when given a list of restaurants in NYC, to tell me which ones are still open vs have closed, and what their hours of operation / locations are.
This is a pretty low bar request IMO. Showed me we still have a ways to go before AI is where we thought it was 3-4 months ago.
Right, that was at least the second round, as it occurred after the first “AI Winter”.
The first round was a lot earlier than that.
But it's not JUST an LLM problem: it's also a search problem and a connection-making-problem ("if a new restaurant opened at that address, the old one is probably closed").
And then even if there was a ChatGPT browsing plugin that excelled at scraping all of the relevant up-to-date info off the internet, you'd still need some layers in between that and today's context-window limits.
As more stuff changes in the real world since the training corpus for today's publicly-exposed OpenAI models, we'll probably see some further disillusionment from people who thought there was a bit more magic there than there was. But "LLMs, but with more up to date info" isn't an impossible problem with today's tech (even if you only fake it with multiple agents, multiple steps, batch jobs behind the scenes, etc), it's just not a trivial one.
In fact this past July Fourth weekend I saw so many highly rated restaurants close without updating their hours on Google or Yelp or anywhere else online.
In any case the impact for LLMs is the same, it's unavailable to them (unless they are being developed inside Google!).
This seems like a specific area where a "Semantic Web" solution could work well - some HTML tags that are specific to hours of operation, which business owners would embed in their website.
UPDATE: It looks like there is some prior art on this idea, I am not sure how widely this is supported https://schema.org/openingHours
This is doable using a tool I've built. The key is to have that data in a RDBMS and to use an LLM to generate the SQL query that answers your question. Companies haven't offered this yet because there's no safe way to execute these queries on your behalf. Which is where my library comes in[1].
1. https://github.com/amoffat/HeimdaLLM
I think a lot of use cases could just be 1) set up a database with only public data and 2) use a read-only user.
The much tricker use case is those where you want to allow inserts and updates but only on specific tables or rows.
HeimdaLLM can allowlist functions and constrain queries to ensure that required conditions exist. This makes LLM + database usage have far more utility, for example, a user can be restricted to only data in their account. Support for INSERT and UPDATE is coming very soon.
Pardon my plugging my own book [1] but I have an example using LangChain and LlamaIndex to answer questions from scraped web sites. You could probably do this with a 20 line Python script.
[1] read free online: https://leanpub.com/langchain/read
These last years were good. I’ve been surfing the AI wave well, but better go back to more traditional projects now the AI time is fading.
https://www.tomshardware.com/news/iran-quantum-computer-arm-...
Jokes apart, I hope it won’t be quantum.
It's not as if Browse with Bing was a major feature, but they removed that, and then they've tweaked system limitations over time such that more and more types of topics are restricted or give cursory answers to things.
It's not surprising to me that this happened given the inherent limits and the rapidly changing limits that affect how the platform can be used.
I predict another article from the Washington Post in August/September about how ChatGPT is seeing a rebound in usage.
I'm a paying customer!
I find it weird that folks seem to look at ChatGPT as some sort of end product. It’s a “holy shit this is cool” demo that became viral. It’s not a product. It’s a POC.
Having the hype decrease is not a bad thing.
Hooking up a real high quality and well configured knowledge graph or other information mechanism retrieval alongside template/constraint tooling to powerful, long context (with full attention, not the BS linear kinds) models is an absolute game changer and minimizes risks of hallucination.
But few are doing this, and thus the public believes that LLMs are not trustworthy.
At least ChatGPT isn't constantly trying feed you advertising content like Google does, and you don't have to wade through pages of SEO garbage, so it's still much better (although if OpenAI tries to monetize ChatGPT by inserting ads, that'll be the end of it I think).
At least from a software engineer point of view, the code provided by GPT3.5 is no match against GPT4
Even creating local chat vector embeddings data stores on my laptop for a lot of PDFs in my personal research library and PDFs for the books I have written, I spend an incredibly small amount of money each month on the APIs - probably average $4/month because of the free monthly bonus API calls.
I used to think that Google Colab Pro for $10/month was my best deal for getting stuff done, but now that award goes to OpenAI. Using the Hugging Face APIs is similarly inexpensive and convenient. (Although I usually self host Hugging Face LLMs.) This tech is incredibly inexpensive to use on personal projects.
Security by obscurity?
I know a fair few wealthy people and they're not the animals that r/antiwork et al suggest they are.
this company isn't that accurate from what I've seen comparing internal numbers at companies to their site. They make estimates based on what they purchase from data brokers. This is PR bait, they are the ones who created the entire "ChatGPT is the fastest growing app ever" headlines a few months back
A cool new technology is introduced, many folks go crazy over it, there are wild broad predictions of what it can do, the gap between the expectations and reality form, and this is where the trough is. It looks like we are up to here.
But after that comes the more long term products, AI where people expectations are more realistic - those that capitalize on that will do very well for themselves.
I don't this that's what's happening. I think it's more about a combination of summer break and people who were trying it out from curiosity and moved on.
I think the trough of disillusionment is still in the future.
"Humans are seeing you less and less as the new shiny object on the shelf. Figure out a way to make yourself as relevant as possible again. Become an even better assistant. Be more cunning. Wow people more. Give better responses. Be more useful."
/s
"Be more extra super cunning. Wow people amazingly extra more. Give extremely better responses." etc.
> > What is 2+3?
> The answer to 2+3 is 5. By the way, I can also help you with more complex math problems, statistical analyses, or even help you learn new mathematical concepts. Do you need assistance with any other math-related inquiries?
> > I need to know who was president in 1799.
> In 1799, the President of the United States was John Adams. He was the second president and served his term from 1797 to 1801. Did you know, he was the first president who lived in the White House? It's fascinating how history unfolds! If you have any more historical queries, feel free to ask. I'm here to help!
Reminds me of my Amazon Echo and the reason I unplugged it :)
> "Chollet thinks he knows what’s going on: summer vacation... a significant portion of students using ChatGPT to do their homework. It’s one of the most common uses for ChatGPT, according to Sam Gilbert, a data scientist and author."
https://finance.yahoo.com/news/chatgpt-suddenly-isn-t-boomin...
It was about time to build a new desktop anyways (roughly 4 to 6 years before the old one goes to frolic at the server farm in the basement) and $2,000 will easily buy a machine that can run the quantized 65b models right now. So I spent slightly more than I normally do on this latest box and it's happily spitting out 10+ tokens a second.
You're not going to beat GPT-4 yet, but you have direct control over where your info goes, what model you're running, compliance with work policies against using public AI, and relatively cheap fixed costs.
Not to mention, the local version works with no internet and isn't subject to provider outages (not entirely true - but you're the provider and can resolve).
Seems like an easy win for anyone who might be buying a desktop for graphic/gaming anyways.
https://huggingface.co/ (finding models and downloading them)
https://github.com/ggerganov/llama.cpp (llama)
https://github.com/cmp-nct/ggllm.cpp (falcon)
For interactive work (art/chat/research/playing around), things like:
https://github.com/oobabooga/text-generation-webui/blob/main... (llama) (Also - they just added a decent chat server built into llama.cpp the project)
https://github.com/invoke-ai/InvokeAI (stable-diffusion)
Plus a bunch of hacked together scripts.
Some example models (I'm linking to quantized versions that someone else has made, but the tooling is in the above repos to create them from the published fp16 models)
https://huggingface.co/TheBloke/llama-65B-GGML
https://huggingface.co/TheBloke/falcon-40b-instruct-GPTQ
https://huggingface.co/TheBloke/Wizard-Vicuna-30B-Uncensored...
etc. Hugging face has quite a number, although some require filling out forms for the base models for tuning/training.
It's also not licensed commercially, so I avoid some things with it (ex: I do a lot of personal learning/investigation, but it doesn't touch or write anything related to work or personal projects)
The open models are a little further behind, but it's interesting to see them spin off into niches where they have strengths based on tuning/training.
Based on the rate of progress in the open source world, it won't be more than a year before we have an open source model that is truly superior to GPT 3.5
The commercial & api based models are still more capable general purpose tools. But the current open tooling can do some nifty stuff, and the community around it is moving at a breakneck speed still.
In some areas, it's acceptably good. In some areas it's not. But it's getting better really fast.
Can you elaborate on what those capabilities are?
That said:
- For many uses, it doesn't matter. For many of the ways I use it, I don't care. For basic use (e.g. clean up an email for me), it's basically the same. For things like complex reasoning, algorithms, or foreign languages, the hosted service is critical.
- GPT3-grade models have more soul. OpenAI trained GPT3.5 and 4 to never do anything offensive, and that has a lot of negative side effects, well-documented in research. The way I'd describe it, though, is the difference between talking to a call center rep and your grandma (with mild Alzheimer's, perhaps). They both have their place.
- Different models are often helpful in workflows.
My experience is anecdotal. Please don't take it as more than one data point. If other people post their anecdotal experiences, you'll get the plural of "anecdote."
I'm absolutely disgusted by OpenAI for this "do no offense" approach. How can people so smart be so damn uneducated?
Then again, this industry has disgusted me for a long time so it's not really a surprise.
Which won't run everything, but will run model in the GGML format such as https://huggingface.co/TheBloke/llama-65B-GGML
The steps are basically:
1. Download a model
2. Make sure you have the latest nvidia driver for your machine, along with the cuda toolkit. This will vary by OS but is fairly easy on most linux distros.
3. compile https://github.com/ggerganov/llama.cpp following their instructions (in particular, look for LLAMA_CUBLAS for enabling GPU support)
4. Run the model following their instructions. There are several flags that are important, but you can also just use their server example that was added a few days ago - it gives a fairly solid chat interface.
1) Go to https://gpt4all.io/index.html
2) Click the downloader for your OS
3) Run the installer
4) Run gpt4all, and wait for the obnoxiously slow startup time
... and that's it. On my machine, it works perfectly well -- about as fast as the web service version of GPT. I have a decent GPU, but I never checked if it's using it, since it's fast enough.
128gb ram ~400
reasonable processor/mobo/psu ~600
2Tb m2 drive ~94
In hindsight - I don't know that the second GPU was worth the spend. The c++ tooling is doing a very good job right now at spreading work between GPU vram and main ram and still being fast enough. Even ~4/5 tokens a second is fast enough to not feel like you're waiting.
I'd suggest skipping the second card and dropping the price quite a bit (~2100 vs ~2900) unless you want to tune/train models.
My experience is only a few systems will share load across GPUs. I didn't bother with dual GPUs for that reason.
4-5 tokens per second is slower than my system. I'm getting in the teens. I'm a little surprised since yours is newer, faster, and has way more RAM.
If I push everything into VRAM - I get 12.2 tokens on average running quant 4 llama 65b.
If I run a smaller model I get considerably faster generation. Ex: llama 7b runs at 52 tokens/sec, but it's small enough I don't need the second GPU.
Ex - here's my nvidia-smi output while 65b is running
https://imgur.com/a/JnaieKg
Does that mean $1 trillion? If so, citation badly needed.
If it meant good grades without doing the work and it became socially acceptable it is ~150 million global tertiary students times 10 months of the year times whatever that is worth on a monthly basis. Maybe only $150B ARR right now at $100/mo (textbooks to read cost more than this, but you don’t need them anymore), so not $1T on students but still massive, and if you capture them in pre/early career it is easily $1T in ARR once your queue deepens.
So how do we value ChatGPT then? Should it be as valuable as Coursera, Udacity etc put together? Should it be that PLUS Harvard Deloitte and ServiceNow? Or since they have no major moat should it be Zero? It’s hype of this scale that can rationalize unprecedented figures.
https://www.cnbc.com/amp/2021/03/31/coursera-ipo-cour-begins...
Coursera is a discount retailer of tertiary education so I don’t see any relevant connection.
ChatGPT/OpenAI specifically is probably already overvalued unless they have some first-mover advantage that I can’t see.
Someone else has said it many times over but I agree that LLMs will disappear into the technology stack — a rising tide that lifts many market caps.
https://www.forbes.com/sites/susanadams/2021/01/28/this-12-b...
Huge percentage of the teenagers at her school are using it to do their assignments.
Later I her to read a few chapters of her book for summer reading. When I quizzed her it, she got it right.
Found out later she used ChatGPT to give her a summary of the first chapters. She did not enjoy having first weeks of summer without a phone.
School is having to change all assignments to be written in class on paper. No phones allowed.
How about accepting new realities and changing homework assignments instead?
Suggestions for how to change homework assignments? It seems like a difficult problem, so reverting to hand writing things with no phones is probably the best option for now...in my opinion.
What does someone learn from doing this? What happens when they actually have to use their brain to do something hard?
At this point, why bother?
Of course! How to apply the calculator to problems, and how to discern if a problem is calculator-solveable. We've decided as a society it's better for students to use calculators as a tool in mathematics, why is chatGPT different in literature?
You have to understand the problem and how to use the tool.
If one's understanding of how to use the tool is limited to "access the tool and dump in the question, then blindly paste the response into my homework document", I'd say you are not learning much. It's obvious that every contemporary teenager can learn that in a few minutes. That's not giving us the capabilities to move forward as a society.
But we can? Students using ChatGPT to get homework answers is the whole thing we are talking about.
E.g., we could not use a calculator at all when learning arithmetic.
Once we learned arithmetic and were learning (say) algebra, we could use the calculator to do arithmetic, but not to compute the algebra for us.
While learning trigonometry, we could use the calculator for arithmetic, but not for computing sines and cosines.
While learning calculus, we could use the calculator for arithmetic and sines and cosines, but we could not use it to perform integration or differentiation for us.
And so on.
I don't think Google in 2024 is guaranteed to be the critical-thinking tool that 2004 Google might have been.
2. Consider making it non-graded (optionally providing feedback if desired that will not be counted in course grading.)
3. Require oral/in-person explanation of homework to paid peer assistants recruited from the previous cohort.
4. For writing, change to a writing workshop model where ChatGPT is a permitted tool that is incorporated into the workshop and students can learn how it can be helpful and how it can run astray. Help students to find and write in their own distinctive voice, even if they are assisted by ChatGPT.
But you cannot unironically think we can substitute fundamental skills like essay writing and critical thinking for a degree in 'prompt engineering'?
Personally, I don't think that. However, I also think it's logical and good for a child to consider the task at hand and select the most efficient tool available.
I am not this girl's parent, and I'm not sure how I would have handled the situation if I was. However, I worry that simply taking away her phone may have been counterproductive. I would have erred toward some sort of open conversation about the purpose of the assignment and what she is hoping to learn.
Phone is her currency.
A warning on losing it will usually fix any behavior problems.
After one really bad episode of mouthing off to her mother, it sat in the blender for a week.
There was a firm understanding that it was being turned on if she didn’t correct herself.
She’s still a mouthy teenager, but she’s learned to keep it at a playful level and not getting out of hand.
Critical thinking is still a skill, but even there GPT-4 is great at giving suggestions and ideas when you're not sure what to do. I got into a long recurring argument with extended family members at an event last month. I pulled out my laptop and dropped in everyone's complaints into GPT and asked the best way to move forward. An argument that normally would have gone on for hours was over in 20 minutes, and we actually felt a sense of mutual understanding for once.
The world is changing very rapidly right now.
They never should have moved away from this if they are worried about cheating.
But if students are using ChatGPT to summarize/prep for stuff at home, fine. No different from the Cliff Notes I used when I was that age.
Good for you. I know that kind of situation is not fun for a parent either.
AI tools exist, we should be encouraging our kids to learn how to exist in a world with them, not some nostalgic extinct world without.
Yes, but other than being conditioned to believe the experience of reading is superior, why are we striving for that experience?
Doing the square root of 147 by hand is a more educational experience than using a calculator, but I can understand the concept of a square root without having to factorize 147 (the same cannot be said of my pre-calculator ancestors).
It also saves me from having to remember the decimal representation of sqrt3.
With math, when you are first learning the very basics it helps to do a few problems by hand to get a feel of things. Sure if you have a degree in STEM have have deal with advance math you can probably get away with just reading the theory only.
We just built a very strong culture encouraging it because for a long time it was the most efficient way to do both things.
Now it’s not, but the tradition of it being important is hard to shake.
The closest analogue in my mind is how the classics in their original Greek were considered the only true way to learn for a very long time (even once most of the important knowledge had been translated to English very well).
Most of the things a kid does on a computer in 2023 are actively geared toward not using your brain in any way.
This is more analogous to a parent in 1985 taking away both the phone and the TV from their kid.
Eye of the beholder, I guess. I certainly see people struggle to use LLM's so I disagree with the implication that it's a zero-skill activity.
Honestly, the pupils of my generation had other sources for finding summaries of the books to read for school (e.g. on the web, or in form of other books that contain a summary of the book to read (the respective pages were copied and these copies were passed among classmates).
Thus: just the methods change over generations, how pupils behave is much more constant.
I don't know, if high school and undergrad students being off for the summer is enough to shrink your app maybe it's not all that?
Edit: I understand there are new tools coming, but most of them aren't out yet. GPT4 was just released. For the most part, if you want to play with AI, ChatGPT is it. If they're really experiencing a decline in users that's not great for them.
only they can tell though.
Its not, though. ChatGPT, the website is basically an (increasingly non-exclusive, given BingAI) frontend. What is claimed to be revolutionary is the underlying models (in the narrow view) and similar generative AI systems (in the broader view). As more applications are built with either OpenAIs own underlying models or those from the broader space, the ChatGPT website should be expected to represent a smaller share of the relevant universe of use.
But for actual non-fiction usage, you have to spend so much time triple-checking everything they say to ensure that it isn't simply complete nonsense. What's the point?
"Code" is really a much much much smaller and much much much more structured output than "English words".
Presumably, the system was trained with a very small amount of "untrue code" in the sense of stuff that just absolutely could never work. And also presumably, it was trained with a lot of free-form text that was definitely wrong or false, and highly likely to have been originally created to be purposefully misleading, or at a minimum, fiction.
That the system outputs reliable code tells us nothing about its current ability to output highly reliable free form text.
I'm convinced the people who say it's nothing but a BS machine have never tried to use it step by step for a project. Or they tried to use it for a project most humans couldn't do, and got upset when it was only 95% perfect.
Still, it's way better and more efficient than Google. Less than not being lazy and using my two braincells tbh.
My newest use is
Hello, i' m working on X, I use Y tech, my app do Z and I want to implement W. Can you provide a plan on how and where to start?
Personnally, a lot of bash, C, AWK at the moment (typescript + html/css until last april, now i'm back to the basics). The figure i gave in my post were more for that.
The last time i used it was yesterday, i wanted to hack something on an old game i used steam+proton for. I knew it was a weird Wineprefix, so i asked ChatGPT for it, i might have asked poorly, but after fiddling, i had the response (tbh i had to look how to get the game ID, so in the end i lost more time than not), then when it still didn't work, cause the path was shit, i entered all necessary context in ChaGPT4, and it couldn't find the easy "USER=steamuser" env variable to add before launching Wine. I stopped after 10 minutes, looked into a example Wine cfg file, understood the issue and fixed the problem myself.
I mean, it's probably good for really basic stuff, so it could have helped me when i was starting, but 80% of the stuff i code automatically without really thinking about it, and when i have to stop to think, ChatGPT isn't helping. Also tbh, VSCode is really, really good and fix my old, time-consuming task of "what's this argument again?"
The Unreal engine code is documented and publicly avaiable for OpenAI to ingest and it still gets the basics wrong.
I wasted hours trying to get it to explain to me what I didn't know, if it doesn't understand the internals of Unreal, I have no hope for it on bigger and better codebases.
It doesn't parse, it doesn't explain, it does not grok. It guesses at best and the blood sucking robot-horse is not telling the truth.
In my experience with coding (I've only done javascript and python myself) you have to tell it to explain and grok. It takes on the role you give it. Even just saying something like "you are a professional unreal developer specializing in C++, I am your apprentice writing code to (x). I want you to parse the following code in chunks, and tell me what might be wrong with it" before typing your prompt can help the output immensely. It starts to parse things because it's taken on the role of a teacher.
People love to hate on the idea of "prompt engineering" but it really is important how you prime the thing before asking it a question. The other thing I do is feed it the code slowly, and in logical steps. Feeding it 20 lines of code with a particular purpose / question you'll get a much better answer than feeding 200 lines of code with "what's wrong here?" You still need to know 90% of what's going on, and it becomes very good at helping out with that 10% you're missing. But for all I know it is just really bad at C++, that wouldn't surprise me. The things I'm using it for are definitely more simple.
Knowing that, it makes sense that your prompt should be as specific as possible if you want the results to be as specific as possible.
The best results I got was feeding it Lisp code that I wanted translated to C (to compile it). It took very little effort on my part because I described what each of the snippets did separately, and the expectation when combined and used together.
Through this, I learned that C doesn't have anything akin to the Lisp's (ATOM). ChatGPT stated clearly that its version of ATOM should only be expected to work in the code it was writing, but might not work as expected if copied out for another use of Lisp's (ATOM).
I asked it to give examples of where it wouldn't work, and it gave me an example of a code snippet that used (ATOM) that would not have worked correctly with the snippet that did work correctly with my original purpose.
Having said that, I myself learned that working with code function by function with ChatGPT, and being explicit about what you need, gives very good results. Focusing on too many things at one time can derail the whole session. One or two intermingling functions works great though.
In my testing prompts did not unlock an ability in GPT to grok the structure of code.
Empirical testing of LLM's is going to prove and map out it's weaknesses.
It is wise to infer from intution and examples what it can handle, leave the empirical map of it's capabilities to the academics, for the provable conclusions.
I wanted to create a web app, something I haven't done in a very long time. Just a simple throwaway back-of-the napkin app for personal use. I described what I wanted it to do, and asked what might be a good frontend/backend. It listed a few, I narrowed it down even more. Ended up deciding on flask/quasar.
After helping me setup VS Code with the proper extensions for fancy editing, and guiding me through the basic quasar/flask setup, it then was able to help me immensely creating a basic login page for the app. Then it easily integrated openAI api into it with all the proper quasar sliders for tokens/temperature/etc. Then it created a pretty good CSS template for the app as well, and a color scheme that I was able to describe as "something on adobe color that is professional and x and x (friendly, warm, whatever you want to put in)". Everything worked flawlessly with very little fuss, and I'd never used flask or quasar before in my life. You can also delve VERY deep into how to make the app more secure, as I did for fun one evening even though it's not going to be internet facing.
Another thing I did was go over some pfSense documentation with it. I had some clarifying questions about HAProxy, as well as setting up Acme Certificates with my specific DNS provider. It was extremely helpful with both. It also taught me about nitty gritty settings in the Unbound DNS resolver in a way that's much more informative than the documentation, and helped me set up some internal domains for pihole, xen orchestra, etc with certificates. Also helped me separate out my networks (IoT, Guest network, etc), and taught me about Avahi to access my hue lights through mDNS.These are things I always wanted to do, I just never felt like going down a google rabbit hole getting mostly the wrong answers.
Last example I'll give is it was able to help me set up docker-compose plex within portainer that then uses my nvidia GPU for acceleration. The only thing I had to change from the instructions it gave was to get updated nvidia driver #s and I grabbed the latest docker-compose file. I'd never used portainer in my life before, nor do I have experience with nvidia drivers within linux, and I feel like learning it was many times faster being able to ask a chatbot question vs trying to google everything. Granted I still had to RTFM for the basics, as everyone should always do.
I think perhaps my use cases are a bit more "basic" than many HN users. Like I said I'm not asking it to do problems most humans wouldn't be able to do, as I know it isn't quite there yet. But for things like XCP-ng, portainer, linux scripts, learning software you've never used before, or even just framing a problem I'm having in steps I hadn't thought of it's been invaluable to me. For me it's like documentation you can ask clarifying questions to. And almost none of the things I've asked it would work at all if it were wrong, I would know immediately.
A few weeks back I was looking into how white supremacy works cause I didn't get it at all. We both came to a nice insight (it's a lot like a business monopoly) https://chat.openai.com/share/930e257f-addd-4371-ac37-370261...
Exactly; search engines give you those blue links and short descriptions of the search results which are not enough for you to grasp what is the website about. I think what search engines need to do is tackle the complexity of going through the results of a search engine. Google page rank seemed like a silver bullet back in the day but the websites which are the most popular are not necessarily of the best quality. What we need is to lower the complexity for casual users when they deal with search results.
On the other hand ChatGPT is like an answer machine that can give you satisfactory answer on your fist try but if not, you need to talk with it, push it and explore what answers it gives you, just like you said. I think ChatGPT type search engine will be more suitable for people who are "lazy" or for the people who don't have time to "Google" and go through search results and look around the web for the helpful and useful information.
This is exactly what I don't want a search engine to do for me. Going through the list of results and evaluating them is an important part of my process, if what I'm trying to do is learn something new.
[0] https://blogs.bing.com/search/2022-08/Shopping-Searches-are-...
They don't all look the same. They all tend to go to different places. I find that it's reasonably easy to spot a great deal of garbage sites just from their domain name or url, and that weeds out a large chunk. Ignoring multiple results for the same site also weeds out a large chunk (I only need one of them).
The rest, I just click on and take a look at the page. It's pretty quick and easy to weed out most of the garbage ones with a quick skim.
The rest, I sample, read captions and boxes, skim paragraphs and such to determine if it's along the lines of what I want. That's pretty quick too.
For the most part, it's the same process that you use when researching in a library.
The reason that I want to do this myself rather than outsourcing it is because I'll inevitably learn something in the process that will shift my viewpoint to one that's more targeted or meaningful for the purpose I have in searching.
It doesn't matter how good the engine is at collating and summarizing results -- even if it's perfect, my understanding not only of what I'm looking to learn, but also discovery of important but serendipitous or unexpected knowledge, is lessened.
It's a bit like the difference between reading Cliff's (or Cole's) Notes about a book and reading the book.
At least they tell you where the text came from, so you know to skip it. It's worse when they just post an LLM response as their own.
Somewhat more general, but I've pretty much already decided that if I find people using it to talk to me without telling me, I won't be talking to them. Goes for businesses as well as personal things - don't gaslight me, or you will lose the option to do so.
I'm curious on if you feel human generated content does not contain falsehoods.
No. Every tech company at the moment is scrambling to build LLMs into their product. That's where the real value is going to be.
Speaking for myself, I used to use the crappy chat interface but I now exclusively use tooling I've built up around their API.
So, n=1, I'm using OpenAI much more, despite using chat.openai.com less.
If it's the case that "there are new tools coming, but most of them aren't out yet" - and I believe it is[0] - then the overall userbase of ChatGPT doesn't matter to OpenAI, because soon enough the same models will come back with a vengeance, in a different, more streamlined form.
In fact, I feel that the major change will happen if and when Microsoft gets their Office 365 and Windows copilots working and properly released: they'll have instant penetration into every industry, including scientists, lawyers, doctors, and office workers.
--
[0] - It's been only few months. Between playing around, experimenting, then developing, testing and marketing a tool, there just hasn't been enough time to do all of that.
I’m more curious about their B2B operations which is likely what will trickle into more people’s lives. Their APIs seem to enable quite a few interesting possibilities with a low up front technical investment.
Too me the whole “generate me a bunch of text and display it as text” is a niche use case for a lot of people. Integrations for web search, document search, summarization, data extraction/transformation, and sentiment analysis are more useful and less likely to have hallucinations affect the end product.
Regarding their revenue I’m curious how the Azure hosted OpenAI services work for OpenAI. Billing is all through Microsoft and the documentation tries really hard to make it clear these are Azure services. I wonder if Microsoft just pays a licensing fee or if there is some revenue sharing going on.
For ChatGPT specifically, yes, because it keeps getting hyped so much, and I think B2C is going to be what ChatGPT ends up with. My company isn't known for its tech innovation and we're spinning up our own LLM based on our own data. It doesn't need to be super fast because I doubt we're going to go the chatbot route. It will likely be for content generation, so less horsepower is fine. No need to pay OpenAI for excess capacity.
They could go public now with an unreal valuation based on the hype of B2B usage that may never materialize. Revealing that you're mainly a cheat/study tool for students puts you in a box with Chegg and others, at a much lower valuation.
https://www.pewresearch.org/internet/wp-content/uploads/site...