Hilarious name for an open source project released by a government lab.
> For me, it was simply a gut feeling. I’ve been talking to founders and doing deep dives into technology companies for decades. It’s been my entire professional life as a writer. And because of that experience, there…
Is it common to have the same number of employees as there are companies funded at a VC? Is each one assigned a company?
Brilliant way to get thousands of tech people to look at your business card. Edit: This was updated, it used to point to his real Github username. Not that there's anything wrong with that! It's clever marketing.
Aside from the project itself, the production value of this video is crazy good.
JAX is a DSL on top of XLA, instead of writing Python. Example: a JAX for loop looks like this: def summ(i, v): return i + v x = jax.lax.fori_loop(0, 100, summ, 5) A for loop in TinyGrad or PyTorch looks like regular…
Previous discussion: https://news.ycombinator.com/item?id=22374825 Wonder if anyone's turned Instagram's version of Blurhash into a library?
I can't think of anything that neural nets can't beat, except small tabular data with boosted decision trees. Can you give some examples?
The code is very easy to read. Doesn't seem like there's data/model parallelism support for training, which will be important for real-world use.
https://github.com/intel/intel-extension-for-pytorch PyTorch version. Looks like it's been supported a lot longer than TensorFlow, the repo dates back over a year.
> And honestly, their hardware is pretty bad for ML given you can pick up a used 3090 for $600. How do you know it's bad? Do you have benchmarks? Just checked, used 3090s are going for $700-900 on eBay.
No, it's meant for taking something like a CSV file and deciding if each row matches a specific category. A common example, have a CSV, columns corresponding to different features of flowers (e.g. number of petals, size…
My mistake, I was thinking of gpt-neox. Thanks for the correction.
> Tensorflow serving is barely controllable, and requires insane tuning to make it perform the same as pytorch What are you using for serving PyTorch models?
PyTorch has been a darling of almost every noteworthy open source model for the past 3-4 years (BLOOM, GPT-J, StyleGAN3, detectron, etc). Personally, I've only seen people use TensorFlow/XLA if they got free TPU credits…
Looks like its just a name collision. It's a tensor used in distributed models, thus Distributed Tensor, or DTensor for short.
The parent comment said it would be "a good start". It's like adding sleep(1000) to a benchmark to purposely make it look worse than your own product.
How would I be able to respond to the post in detail if I didn't read it? What a bizarre, defensive response. To address your points: - Multiple formats were compared Yes, but not a zero-copy or efficient format, like…
"ML" was never monopolized by Python. Boosted decision trees, as the post demonstrates, are commonly done in Matlab, R, or Julia. Deep learning however is 99% Python interface.
Python can be fast if you don't intentionally cripple it. Doing the following will be most likely a lot faster than postgresml: - replace json (storing data as strings? really?) with a binary format like protobuf, or…
> Vivid Natural — A slide of jam, jazzman, hook, band, pipe, and coil. Small contrasting details in natural colors. vivid natural. This one generates the string instruments with the correct colors.
Can you summarize the approach you took on creating the model? Really curious!
(not a hardware engineer) This looks like Google's version of Infiniband, which is basically mandatory for large scale ML-training servers due to issues with Ethernet, as described in the paper. Really cool stuff,…
Microsoft invested $1B in OpenAI in 2019, the valuation is probably somewhere in the neighborhood of $5-20B.
> But then Meta is using those academic non-commercial datasets to train a model, presumably for future commercial use in their products. Weird, right? This is a very strong and likely inaccurate presumption.
Hilarious name for an open source project released by a government lab.
> For me, it was simply a gut feeling. I’ve been talking to founders and doing deep dives into technology companies for decades. It’s been my entire professional life as a writer. And because of that experience, there…
Is it common to have the same number of employees as there are companies funded at a VC? Is each one assigned a company?
Brilliant way to get thousands of tech people to look at your business card. Edit: This was updated, it used to point to his real Github username. Not that there's anything wrong with that! It's clever marketing.
Aside from the project itself, the production value of this video is crazy good.
JAX is a DSL on top of XLA, instead of writing Python. Example: a JAX for loop looks like this: def summ(i, v): return i + v x = jax.lax.fori_loop(0, 100, summ, 5) A for loop in TinyGrad or PyTorch looks like regular…
Previous discussion: https://news.ycombinator.com/item?id=22374825 Wonder if anyone's turned Instagram's version of Blurhash into a library?
I can't think of anything that neural nets can't beat, except small tabular data with boosted decision trees. Can you give some examples?
The code is very easy to read. Doesn't seem like there's data/model parallelism support for training, which will be important for real-world use.
https://github.com/intel/intel-extension-for-pytorch PyTorch version. Looks like it's been supported a lot longer than TensorFlow, the repo dates back over a year.
> And honestly, their hardware is pretty bad for ML given you can pick up a used 3090 for $600. How do you know it's bad? Do you have benchmarks? Just checked, used 3090s are going for $700-900 on eBay.
No, it's meant for taking something like a CSV file and deciding if each row matches a specific category. A common example, have a CSV, columns corresponding to different features of flowers (e.g. number of petals, size…
My mistake, I was thinking of gpt-neox. Thanks for the correction.
> Tensorflow serving is barely controllable, and requires insane tuning to make it perform the same as pytorch What are you using for serving PyTorch models?
PyTorch has been a darling of almost every noteworthy open source model for the past 3-4 years (BLOOM, GPT-J, StyleGAN3, detectron, etc). Personally, I've only seen people use TensorFlow/XLA if they got free TPU credits…
Looks like its just a name collision. It's a tensor used in distributed models, thus Distributed Tensor, or DTensor for short.
The parent comment said it would be "a good start". It's like adding sleep(1000) to a benchmark to purposely make it look worse than your own product.
How would I be able to respond to the post in detail if I didn't read it? What a bizarre, defensive response. To address your points: - Multiple formats were compared Yes, but not a zero-copy or efficient format, like…
"ML" was never monopolized by Python. Boosted decision trees, as the post demonstrates, are commonly done in Matlab, R, or Julia. Deep learning however is 99% Python interface.
Python can be fast if you don't intentionally cripple it. Doing the following will be most likely a lot faster than postgresml: - replace json (storing data as strings? really?) with a binary format like protobuf, or…
> Vivid Natural — A slide of jam, jazzman, hook, band, pipe, and coil. Small contrasting details in natural colors. vivid natural. This one generates the string instruments with the correct colors.
Can you summarize the approach you took on creating the model? Really curious!
(not a hardware engineer) This looks like Google's version of Infiniband, which is basically mandatory for large scale ML-training servers due to issues with Ethernet, as described in the paper. Really cool stuff,…
Microsoft invested $1B in OpenAI in 2019, the valuation is probably somewhere in the neighborhood of $5-20B.
> But then Meta is using those academic non-commercial datasets to train a model, presumably for future commercial use in their products. Weird, right? This is a very strong and likely inaccurate presumption.