one man's exfiltration is another man's distillation `¯\_(ツ)_/¯` you could say they're playing by a different set of rules, but distilling from the best available model is the current meta across the industry. only they…
Check out this recent benchmark MTOB (Machine Translation from One Book) -- relevant to your comment, though the book does have parallel passages so not exactly what you have in mind: https://arxiv.org/pdf/2309.16575 In…
Very cool. Got a silly sci-fi question for you. IIUC, with current technology it would take on the order of tens of thousands of years for a vessel to physically travel to the closest known Earth-like planet (correct me…
looks like it's there now
Models like this are experimentally pretrained or tuned hundreds of times over many months to optimize the datamix, hyperparams, architecture, etc. When they say "ran parallel trainings" they are probably referring to…
He is coming from the perspective of a long-running debate on symbolic versus statistical/data-driven approaches to modeling language structure and use. It seems in recent years he has had trouble coming to terms with…
Savor it while you can. As a former academic, for me the lack of intrinsic motivation to "create value for shareholders" is the hardest part of working in industry.
As painful as it can be at times, it is a truly beautiful phase of life during which your main obligations are to become an expert in something that interests you and to make enough money to not starve and have a place…
This is just brilliant. Brings back memories, some fond others less so. Only addition I'd suggest is a subplot involving teaching/TAing duties and/or money problems. Good to be occasionally reminded that slacking off is…
That's an insane story. As much as I hate flying, modern aviation infrastructure is one of mankind's most impressive feats.
For those interested, there is a podcast about this book and some cases it's been relevant in: https://podcasts.apple.com/us/podcast/hit-man/id1449636432
Yeah that is consistent with my experience for sure. Probably true for lots of others too. I'm pretty neurotic about dumb shit on a day-to-day basis, but the few times something actually extremely serious has gone wrong…
That's kinda how the pandemic felt to a degree. A happier than average time in my personal life, despite (or perhaps due in part to?!) all the carnage and pandemonium throughout the world. Feels wrong to say, but I will…
Big shouts to visidata -- very underrated tool imo.
The consistency of descriptions is particularly surprising to me. Like you got a roughly circular collection of seemingly random scribbles, but they can tell you exactly which parts of it correspond to the person's…
I have a very similar plan involving a Raspberry Pi, I agree that it's a great middle ground. IMO screentime that only involves video calls with family is perfectly fine, especially if it is a group activity. In general…
Is migrating domains from Gandi to somewhere else easy/possible? I've had the same domain for years, I used to pay like $20 for five years, but then I forgot to renew, and because I let it expire for a few days, now…
Incredible post. I laughed when I saw the title, snickered at the first paragraph, and then proceeded to be blown away by the rest of it. Thought I was in for a joke and instead I'm thinking about the nature of ML Ops…
Training code won't get you much if you don't have the infra/money to gather a suitable dataset or actually execute training. Plus if your goal is to "steal" or riff on the base model, it's already there in the weights.…
Hopefully so, would really like to know what else is lost by nerfing potentially offensive responses. Can't imagine a project I'd rather work on. I think open-assistant.io has a chance to do exactly this. We'll see what…
This is the best advice on the topic that I've seen.
Yeah I think iPhone is a very apt analogy: certainly not the first product of its kind, but definitely the first wildly successful one, and definitely the one people will point to as the beginning of the smartphone era.…
I'm shocked that I've never heard this argument before. Very interesting and indeed, disturbing as well.
Yeah this is pretty much my line of thinking. Not making any grand claims, just questioning the conventional wisdom. No doubt that the material conditions, health, and conveniences of today are unparalleled in history,…
Yeah that's pretty much where I end up: what it means to "thrive" is subjective and there's just not really a matter of the fact about what lifestyle is "best" for humans. Nevertheless, there are some scenarios that I…
one man's exfiltration is another man's distillation `¯\_(ツ)_/¯` you could say they're playing by a different set of rules, but distilling from the best available model is the current meta across the industry. only they…
Check out this recent benchmark MTOB (Machine Translation from One Book) -- relevant to your comment, though the book does have parallel passages so not exactly what you have in mind: https://arxiv.org/pdf/2309.16575 In…
Very cool. Got a silly sci-fi question for you. IIUC, with current technology it would take on the order of tens of thousands of years for a vessel to physically travel to the closest known Earth-like planet (correct me…
looks like it's there now
Models like this are experimentally pretrained or tuned hundreds of times over many months to optimize the datamix, hyperparams, architecture, etc. When they say "ran parallel trainings" they are probably referring to…
He is coming from the perspective of a long-running debate on symbolic versus statistical/data-driven approaches to modeling language structure and use. It seems in recent years he has had trouble coming to terms with…
Savor it while you can. As a former academic, for me the lack of intrinsic motivation to "create value for shareholders" is the hardest part of working in industry.
As painful as it can be at times, it is a truly beautiful phase of life during which your main obligations are to become an expert in something that interests you and to make enough money to not starve and have a place…
This is just brilliant. Brings back memories, some fond others less so. Only addition I'd suggest is a subplot involving teaching/TAing duties and/or money problems. Good to be occasionally reminded that slacking off is…
That's an insane story. As much as I hate flying, modern aviation infrastructure is one of mankind's most impressive feats.
For those interested, there is a podcast about this book and some cases it's been relevant in: https://podcasts.apple.com/us/podcast/hit-man/id1449636432
Yeah that is consistent with my experience for sure. Probably true for lots of others too. I'm pretty neurotic about dumb shit on a day-to-day basis, but the few times something actually extremely serious has gone wrong…
That's kinda how the pandemic felt to a degree. A happier than average time in my personal life, despite (or perhaps due in part to?!) all the carnage and pandemonium throughout the world. Feels wrong to say, but I will…
Big shouts to visidata -- very underrated tool imo.
The consistency of descriptions is particularly surprising to me. Like you got a roughly circular collection of seemingly random scribbles, but they can tell you exactly which parts of it correspond to the person's…
I have a very similar plan involving a Raspberry Pi, I agree that it's a great middle ground. IMO screentime that only involves video calls with family is perfectly fine, especially if it is a group activity. In general…
Is migrating domains from Gandi to somewhere else easy/possible? I've had the same domain for years, I used to pay like $20 for five years, but then I forgot to renew, and because I let it expire for a few days, now…
Incredible post. I laughed when I saw the title, snickered at the first paragraph, and then proceeded to be blown away by the rest of it. Thought I was in for a joke and instead I'm thinking about the nature of ML Ops…
Training code won't get you much if you don't have the infra/money to gather a suitable dataset or actually execute training. Plus if your goal is to "steal" or riff on the base model, it's already there in the weights.…
Hopefully so, would really like to know what else is lost by nerfing potentially offensive responses. Can't imagine a project I'd rather work on. I think open-assistant.io has a chance to do exactly this. We'll see what…
This is the best advice on the topic that I've seen.
Yeah I think iPhone is a very apt analogy: certainly not the first product of its kind, but definitely the first wildly successful one, and definitely the one people will point to as the beginning of the smartphone era.…
I'm shocked that I've never heard this argument before. Very interesting and indeed, disturbing as well.
Yeah this is pretty much my line of thinking. Not making any grand claims, just questioning the conventional wisdom. No doubt that the material conditions, health, and conveniences of today are unparalleled in history,…
Yeah that's pretty much where I end up: what it means to "thrive" is subjective and there's just not really a matter of the fact about what lifestyle is "best" for humans. Nevertheless, there are some scenarios that I…