Ask HN: Improving LLM Performance

3 points by gregw134 3y ago ↗ HN

I'm trying to figure out whether it's possible to use LLM's to categorize the internet. Using back of napkin math, if it takes a few seconds per web page this would take $XXX,XXX+ to process the common crawl. Does anyone have tips on speeding up LLMs? Is it possible to use LLMs to train a cheaper student model? Thanks!

5 comments

[ 3.2 ms ] story [ 36.0 ms ] thread

PaulHoule 3y ago ↗

Common Crawl is full of real junk, you'd need some kind of classifier just to pick out the stuff that's worth classifying...

gregw134 3y ago ↗

Agreed. To start I'm thinking of cross-referencing this with a list of the top 10M most popular websites and dropping anything not on the list. If you have any insight into dealing with common crawl data (or better data sources) I'd love to hear it.

PaulHoule 3y ago ↗

I’d start with a small number of high quality web sites, possibly even just one. Look up my profile and send me an email.

throwaway888abc 3y ago ↗

You should have look at DMOZ.

https://dmoz-odp.org/

It feels so old to write this, but back in the days this one was important.

Found this on Kagle: https://www.kaggle.com/datasets/shawon10/url-classification-...

gregw134 3y ago ↗

That's been bouncing around my head too. Would be interesting to use LLM's to generate this classification automatically. Right now it's hard to do it efficiently though.