For FAANG engineers this will likely mean moving to Vancouver, Zurich or Singapore with their job, salary, rsus and taxes.
This story plays out so often that here should be a law about it: Supply lags demand, prices soar, everybody hears about it, everybody pours in, supply surges, demand normalizes, supply overshoots demand, prices…
This one’s a bit worse than meta’s usual sins: enabling political ads to manipulate the user is one thing. But enabling naughty people to generate naughty pictures of innocent bystanders because they happened to be in…
> The British introduced electricity, railways, capitalism and a thousand other things we take for granted behind those saffron-tinted glasses. Some people love being a slave, but that is not a requirement for having…
[flagged]
> CV Raman won the Nobel prize in science, Tagore won the Nobel prize in literature, Ramanujan etc, the names are numerous. So golden age of India was when the country with a seventh of the world's population won 2…
Useful idea, but needs a lot more work IMO e.g. - Looks like there's just 7 abstracts right now. - Most of the abstracts are written by the author of the paper, so might not be as unbiased as an actual…
AI rescued Nvidia when nobody was buying their shovels for digging crypto gold. If it wasn’t for ChatGPT, Nvidia story would have been very different right now. He is probably just hoping that this dream never ends.…
> DPO is pretty much strictly better than RLHF + PPO Out of genuine curiosity, do you have any pointers/evidence to support this. I know that some of the industry leading research labs haven't switched over to DPO yet,…
Another problem with the title: the article is about DPO, which doesn’t do reinforcement learning. So not RLHF. I guess RLHF has more of a name recognition than DPO.
Because a salesman’s skills complements those of a researcher. Salesman sells what the researcher built and brings in money to keep the lights on. Researcher gets to do what they love without having to worry about the…
The fine-grained results look like: - 1444x faster for single character prefixes - 252x faster for two character prefixes - 55x faster for three character prefixes - ~20x faster for 4 and 5 character prefixes - <= 5x…
I have been training a natural intelligence model for 3 years now and she still doesn’t get nuance. Things are either good or bad in her book: nothing in between. My plan is to let her train with binary good/bad labels…
In this day and age a motivated 18+ year old adult has more than enough tools to actively pursue education outside a college setting while having a day job that pays. And this learning does not need to end after four…
Brought to you by the same creative minds that conceived CUDA (which means trash in Hindi).
> another victim of the rapid pace of improvement in smartphones It's not just the pace of improvement, but also the marketing spin. I find the strengths of smartphone camera and ILCs pretty complementary. Smartphone…
The problem with over-hiring is not that there's a lot of people doing nothing. The problem is that there's a lot of people who need to do something to write in their annual reviews. Since there's only so many useful…
Thanks for laying out the plan. I was trying to understand the cost of each of these steps below and started wondering about the following: > rough steps: > 1. collect a very large dataset, see:…
LLMs for the most part learn P(someone on internet said A | internet specific context B) given ginormous amounts data. There’s no other type of A, B with that much training data at hand.
+1 if your input and output are both sequences, FSTs can be your best friend…Unless you prefer the company of the flaky cool kids from deep learning (CTC, RNN-T, …)
+1 Awesome introduction to computational geometry. Couldn’t help but mention this fun fact: Voronoi diagrams were used to fight the 1854 Cholera outbreak in London https://plus.maths.org/content/uncovering-cause-cholera
+1 for including the sample size in the title. Good way of distinguishing from click-bait.
Microsoft spent almost the same on nuance: https://news.ycombinator.com/item?id=26778780
For FAANG engineers this will likely mean moving to Vancouver, Zurich or Singapore with their job, salary, rsus and taxes.
This story plays out so often that here should be a law about it: Supply lags demand, prices soar, everybody hears about it, everybody pours in, supply surges, demand normalizes, supply overshoots demand, prices…
This one’s a bit worse than meta’s usual sins: enabling political ads to manipulate the user is one thing. But enabling naughty people to generate naughty pictures of innocent bystanders because they happened to be in…
> The British introduced electricity, railways, capitalism and a thousand other things we take for granted behind those saffron-tinted glasses. Some people love being a slave, but that is not a requirement for having…
[flagged]
> CV Raman won the Nobel prize in science, Tagore won the Nobel prize in literature, Ramanujan etc, the names are numerous. So golden age of India was when the country with a seventh of the world's population won 2…
Useful idea, but needs a lot more work IMO e.g. - Looks like there's just 7 abstracts right now. - Most of the abstracts are written by the author of the paper, so might not be as unbiased as an actual…
AI rescued Nvidia when nobody was buying their shovels for digging crypto gold. If it wasn’t for ChatGPT, Nvidia story would have been very different right now. He is probably just hoping that this dream never ends.…
> DPO is pretty much strictly better than RLHF + PPO Out of genuine curiosity, do you have any pointers/evidence to support this. I know that some of the industry leading research labs haven't switched over to DPO yet,…
Another problem with the title: the article is about DPO, which doesn’t do reinforcement learning. So not RLHF. I guess RLHF has more of a name recognition than DPO.
Because a salesman’s skills complements those of a researcher. Salesman sells what the researcher built and brings in money to keep the lights on. Researcher gets to do what they love without having to worry about the…
The fine-grained results look like: - 1444x faster for single character prefixes - 252x faster for two character prefixes - 55x faster for three character prefixes - ~20x faster for 4 and 5 character prefixes - <= 5x…
I have been training a natural intelligence model for 3 years now and she still doesn’t get nuance. Things are either good or bad in her book: nothing in between. My plan is to let her train with binary good/bad labels…
In this day and age a motivated 18+ year old adult has more than enough tools to actively pursue education outside a college setting while having a day job that pays. And this learning does not need to end after four…
Brought to you by the same creative minds that conceived CUDA (which means trash in Hindi).
> another victim of the rapid pace of improvement in smartphones It's not just the pace of improvement, but also the marketing spin. I find the strengths of smartphone camera and ILCs pretty complementary. Smartphone…
The problem with over-hiring is not that there's a lot of people doing nothing. The problem is that there's a lot of people who need to do something to write in their annual reviews. Since there's only so many useful…
Thanks for laying out the plan. I was trying to understand the cost of each of these steps below and started wondering about the following: > rough steps: > 1. collect a very large dataset, see:…
LLMs for the most part learn P(someone on internet said A | internet specific context B) given ginormous amounts data. There’s no other type of A, B with that much training data at hand.
+1 if your input and output are both sequences, FSTs can be your best friend…Unless you prefer the company of the flaky cool kids from deep learning (CTC, RNN-T, …)
+1 Awesome introduction to computational geometry. Couldn’t help but mention this fun fact: Voronoi diagrams were used to fight the 1854 Cholera outbreak in London https://plus.maths.org/content/uncovering-cause-cholera
+1 for including the sample size in the title. Good way of distinguishing from click-bait.
Microsoft spent almost the same on nuance: https://news.ycombinator.com/item?id=26778780