CustomGPT lets you "Build Your Own ChatBOT" with your own data. Using OpenAI's language models, this chatbot lets you write blog posts, answer customer service questions, plus 100+ use cases.
Does anyone knowledgeable in AI know how feasible the claim of training your custom ChatGPT really is? Because this would be the holy grail we’ve all been waiting for, no?
Training a custom chatbot is easy. Having it reliably retrieve the data you want is hard. For any given question, you can get it to look at a maximum of 4k tokens (16k characters, or 3k or so words). So when you want your chatbot to have access to a LOT of data, you usually end up having to work around that - typically by:
1. chunking up your documents into ~2k word chunks,
2. and then getting the query, doing a search using that query, and getting the top N results back,
3. running the prompt on each result, and then
4. gradually combining the results of each run of the prompt.
Each step is a little lossy, and there are quite a few steps there, so the bigger the corpus gets, the lossier/less useful your bot is.
Now that being said, there are ways to improve that, but for now at least, its unlikely that a single slackbot is going to solve all your knowledge management problems for you.
Just shove all your information in the prompt. I assume that's what these guys are doing anyway. ChatGPT doesn't yet have any API, but you can use text-davinci-003, which does have an API and is functionally identical.
The API costs for a large prompt are quite high, and since you have to repeat it with every single query, your costs can go up very quickly.
Gotcha! Thanks for the response. So if I have, say, 10 PDF files which I plan to extract text from and use to fine-tune davinci, I would have to supply the entire textual information extracted from the PDF file each time?
davinci-003 has a 4k tokens cap, you cannot just 'shove all your information in the prompt.' Can parallelize but then rate limits become a problem. Really curious to hear how these guys are handling it.
It's insanely easy to use RelevanceAI to upload a 1-column spreadsheet, click the "vectorize" button, and then have something working. The problem is putting the data into that spreadsheet.
Google allows you to export your email data. Ideally, you could download that data and upload it to a service that looked at original messages and your replies (and replies to replies) and trained on that file without having to make sense of what Google gives you.
If that was working, then you could have it reply to emails for you.
Founders really need to start doing 10 minutes of due diligence to ensure that their sites won't crash and burn under moderate Hacker News or Product Hunt traffic before posting at these places. You get one shot at front page success, and are now losing valuable impressions because people can't even open your website. And you not being able to keep a marketing page up and running doesn't really give me confidence in adopting your actual product. A few clicks in Cloudflare could have easily prevented this.
> A few clicks in Cloudflare could have easily prevented this.
In my experience, this is surprisingly harder to achieve in practice than it sounds. I've had to spend quite a lot of time figuring out CloudFlare's arcane composition of page rules that would prevent a simple web site's denial of service by a simple netcat command.
That says more about Cloudflare than you think it's saying about you, tbh. I see this often, many people enlist their site into Cloudflare and consider the job done, and Cloudflare doesn't necessarily discourage that either until their app draws traffic and suddenly sales is your new neighbor.
There's a very slim chance of hitting the front page on HN (believe me, look at my post history, I tried a couple of times), do you prepare for such an unlikely event...
I mean if course you should, because otherwise why post on HN, but if it's 'just' a side project, I understand why you don't want to spend time configuring Cloudflare.
I did a similar thing where i had a specific "journal" chat with CGPT and gave it a biography and primed it to act like Socrates with mental health training, and counselor skills. I get it to give prompts and also ask questions that promote elaboration.
Actually journaling for the first time in my life, it is amazing when it remembers things that i have told it and is super motivational, especially since it is always available and never gets bored of me asking for advice or venting about my day.
Am I too harsh for expecting that in 2023, a company actively signing up customers for a cloud-based service should have enough autoscaling so that getting to the front page of HN doesn't overload the landing page? It just doesn't give me much faith that the actual product will be stable enough to use. Especially just giving a generic 5XX error from nginx is a bad sign.
Edit: did this get manually knocked off the front page without getting flagged?
Sure, but their landing page never goes down when ChatGPT is over capacity, and their over capacity page is a delight of downtime design. It has auto-generated content in different genre styles about being over capacity. They even let you enter your email to get notified when it is back. But I do get frustrated that every time I've entered my email to be notified I've never actually been notified.
We've actually been using Buff as a chat bot on our Discord server to iteratively explore vulnerability data (CVEs) using AI.
For example:
> @Buff What is Log4Shell?
>> Log4Shell is a vulnerability named by LunaSec. It affects users of the log4j packages, primarily in Java but also in Scala, Groovy, and Clojure. "Log4Shell Guide: How To Detect and Mitigate the Log4Shell Vulnerability (CVE-2021-44228 & CVE-2021-45046)" is available on LunaTrace.
The idea is that we can ingest unstructured data from GitHub Security Advisories (and the links they link to, like that LunaSec blog post) and then "query" that dataset with natural language.
If you'd like to play around with it yourself, feel free to join and try it out in our Discord. https://discord.gg/dtGHzZK6Cq
(It only has a few vulns of data in it right now and we're scraping the data to train it right now. Any ideas that others have to share would be awesome!)
32 comments
[ 3.3 ms ] story [ 76.4 ms ] threadhttps://www.producthunt.com/posts/customgpt
Does anyone knowledgeable in AI know how feasible the claim of training your custom ChatGPT really is? Because this would be the holy grail we’ve all been waiting for, no?
1. chunking up your documents into ~2k word chunks, 2. and then getting the query, doing a search using that query, and getting the top N results back, 3. running the prompt on each result, and then 4. gradually combining the results of each run of the prompt.
Each step is a little lossy, and there are quite a few steps there, so the bigger the corpus gets, the lossier/less useful your bot is.
Now that being said, there are ways to improve that, but for now at least, its unlikely that a single slackbot is going to solve all your knowledge management problems for you.
The API costs for a large prompt are quite high, and since you have to repeat it with every single query, your costs can go up very quickly.
https://langchain.readthedocs.io/en/latest/modules/chains/co...
Ah, the rub is that a fine-tuned Davinci costs 6x a standard Davinci. Interesting. https://openai.com/api/pricing/
[1] - https://platform.openai.com/docs/tutorials [2] - https://github.com/openai/openai-cookbook [3] - https://github.com/hwchase17/langchain
https://relevanceai.com/blog/use-all-of-your-blog-posts-and-...
As far as the next step...
Google allows you to export your email data. Ideally, you could download that data and upload it to a service that looked at original messages and your replies (and replies to replies) and trained on that file without having to make sense of what Google gives you.
If that was working, then you could have it reply to emails for you.
In my experience, this is surprisingly harder to achieve in practice than it sounds. I've had to spend quite a lot of time figuring out CloudFlare's arcane composition of page rules that would prevent a simple web site's denial of service by a simple netcat command.
I mean if course you should, because otherwise why post on HN, but if it's 'just' a side project, I understand why you don't want to spend time configuring Cloudflare.
Actually journaling for the first time in my life, it is amazing when it remembers things that i have told it and is super motivational, especially since it is always available and never gets bored of me asking for advice or venting about my day.
Edit: did this get manually knocked off the front page without getting flagged?
We're taking a more open source approach to get feedback from others in this space since it's evolving so quickly.
For example:
> @Buff What is Log4Shell?
>> Log4Shell is a vulnerability named by LunaSec. It affects users of the log4j packages, primarily in Java but also in Scala, Groovy, and Clojure. "Log4Shell Guide: How To Detect and Mitigate the Log4Shell Vulnerability (CVE-2021-44228 & CVE-2021-45046)" is available on LunaTrace.
>> SOURCES: https://www.lunasec.io/docs/blog/log4j-zero-day-mitigation-g...
The idea is that we can ingest unstructured data from GitHub Security Advisories (and the links they link to, like that LunaSec blog post) and then "query" that dataset with natural language.
If you'd like to play around with it yourself, feel free to join and try it out in our Discord. https://discord.gg/dtGHzZK6Cq
(It only has a few vulns of data in it right now and we're scraping the data to train it right now. Any ideas that others have to share would be awesome!)