For a more informed opinion than folks on the internet, here's some work from Microsoft with early/internal access to gpt4: https://arxiv.org/abs/2303.12712 . I don't think people close by these systems share the same…
These projects are awesome to see, there are similar efforts for everquest. Is anyone aware of anyone trying to create different clients/renderers for these MMOs? A VR client for any of these worlds would instantly be…
Just to further this, it's not just 'big names' that feel this way. Read this paper from a team at Microsoft Research: https://arxiv.org/abs/2303.12712 . These folks spent months studying properties of GPT-4, that paper…
This seems like (clearly?) bad journalism. RFID tags on their own are for sure not taking active sensor readings, that'd be left for something closer to TPMS style systems.
Yes, it did: https://chasethedevil.github.io/post/the_neural_network_in_y...
No, your OP is mistaken. The model weights have to all be accessed for the forward pass. What has happened is that using mmap changes where the memory is consumed (kernel vs process) and so it was being incorrectly…
Ahh, this would do it, thanks :).
That shouldn't be the case. 30B is a number that directly represents the size of the model, not the size of the other components.
I also have this question, yes it should be. The forward pass should require accessing all the weights AFAIK.
Does anyone know how/why this change decreases memory consumption (and isn't a bug in the inference code)? From my understanding of the issue, mmap'ing the file is showing that inference is only accessing a fraction of…
So can we at least take a second to appreciate the orders of magnitude of growth between BBS' and tiktok? With data structures and algorithms we're very willing to accept that orders of magnitudes shifting can change…
This thread is exposing one of the dirty realities of modern SEO. Most of the time, SEO folks will measure their success based on having some way of determining their ranking for a given search query. The problem is…
It's mostly a curiosity. Occasionally things like this occur: https://googleprojectzero.blogspot.com/2021/12/a-deep-dive-i... . TL;DR exploited an accidentally turing complete file format to execute instructions inside…
For big organizations that have the capacity and data, online advertising becomes a ROI optimization game, and one that they perform quite well at. For a random business that wants to advertise online, without the…
Given the usual markers of attractiveness, wouldn't you expect age and weight to be strong predictors usually?
There's nothing 1960s about it, that's just not well reasoned. These computation constructs/tools just don't currently have better abstractions while maintaining the desired computational performance. It's a strange…
What? CUDA is intentionally granular and low level, why do you feel that is a negative thing at this level of abstraction? Are you suggesting the tools should be better so that doesn't have to be the case? I can't…
You're not wrong that the dataset and compute are important, and if you browse the author's previous work, you'll see there are datasets available. The reproduction of DALL-E 2 required a dataset of similar size to the…
To expand a bit for the grandparent, if you check out this authors other repos you'll notice they have a thing for implementing these papers (multiple DALLE-2 implementations for instance). You should expect to see an…
Why does everyone assume this guy has zero business attempting this? If you read his credentials, he should be every bit as qualified as you to attempt this kind of work while understanding the pitfalls. According to…
So that's actually one of the awful implications of this paper. It's probably actually happening at a rate higher than would be noticed by humans. If a given piece of silicon is hosing up a GEMM (matrix multiply), in…
It's boggling to me that everyone is so entirely convinced of this fact. Check out this paper from google where they observe this exact process occurring in their CPUs at scale:…
Yes, but for different reasons. The grandparent is probably incorrect though, there is emerging evidence that silicon is actually changing/failing over time. See this paper from google on their CPU cores where they have…
Actually there's reasonable evidence to believe this isn't true. This paper from google: https://sigops.org/s/conferences/hotos/2021/papers/hotos21-s... outlines specific scenarios where CPUs fail over time. Given the…
Lua definitely caught a stray here. The reference VM is pretty fast, the JIT is just way faster. No idea how Lua ended up wrapped in the conversation with Python...
For a more informed opinion than folks on the internet, here's some work from Microsoft with early/internal access to gpt4: https://arxiv.org/abs/2303.12712 . I don't think people close by these systems share the same…
These projects are awesome to see, there are similar efforts for everquest. Is anyone aware of anyone trying to create different clients/renderers for these MMOs? A VR client for any of these worlds would instantly be…
Just to further this, it's not just 'big names' that feel this way. Read this paper from a team at Microsoft Research: https://arxiv.org/abs/2303.12712 . These folks spent months studying properties of GPT-4, that paper…
This seems like (clearly?) bad journalism. RFID tags on their own are for sure not taking active sensor readings, that'd be left for something closer to TPMS style systems.
Yes, it did: https://chasethedevil.github.io/post/the_neural_network_in_y...
No, your OP is mistaken. The model weights have to all be accessed for the forward pass. What has happened is that using mmap changes where the memory is consumed (kernel vs process) and so it was being incorrectly…
Ahh, this would do it, thanks :).
That shouldn't be the case. 30B is a number that directly represents the size of the model, not the size of the other components.
I also have this question, yes it should be. The forward pass should require accessing all the weights AFAIK.
Does anyone know how/why this change decreases memory consumption (and isn't a bug in the inference code)? From my understanding of the issue, mmap'ing the file is showing that inference is only accessing a fraction of…
So can we at least take a second to appreciate the orders of magnitude of growth between BBS' and tiktok? With data structures and algorithms we're very willing to accept that orders of magnitudes shifting can change…
This thread is exposing one of the dirty realities of modern SEO. Most of the time, SEO folks will measure their success based on having some way of determining their ranking for a given search query. The problem is…
It's mostly a curiosity. Occasionally things like this occur: https://googleprojectzero.blogspot.com/2021/12/a-deep-dive-i... . TL;DR exploited an accidentally turing complete file format to execute instructions inside…
For big organizations that have the capacity and data, online advertising becomes a ROI optimization game, and one that they perform quite well at. For a random business that wants to advertise online, without the…
Given the usual markers of attractiveness, wouldn't you expect age and weight to be strong predictors usually?
There's nothing 1960s about it, that's just not well reasoned. These computation constructs/tools just don't currently have better abstractions while maintaining the desired computational performance. It's a strange…
What? CUDA is intentionally granular and low level, why do you feel that is a negative thing at this level of abstraction? Are you suggesting the tools should be better so that doesn't have to be the case? I can't…
You're not wrong that the dataset and compute are important, and if you browse the author's previous work, you'll see there are datasets available. The reproduction of DALL-E 2 required a dataset of similar size to the…
To expand a bit for the grandparent, if you check out this authors other repos you'll notice they have a thing for implementing these papers (multiple DALLE-2 implementations for instance). You should expect to see an…
Why does everyone assume this guy has zero business attempting this? If you read his credentials, he should be every bit as qualified as you to attempt this kind of work while understanding the pitfalls. According to…
So that's actually one of the awful implications of this paper. It's probably actually happening at a rate higher than would be noticed by humans. If a given piece of silicon is hosing up a GEMM (matrix multiply), in…
It's boggling to me that everyone is so entirely convinced of this fact. Check out this paper from google where they observe this exact process occurring in their CPUs at scale:…
Yes, but for different reasons. The grandparent is probably incorrect though, there is emerging evidence that silicon is actually changing/failing over time. See this paper from google on their CPU cores where they have…
Actually there's reasonable evidence to believe this isn't true. This paper from google: https://sigops.org/s/conferences/hotos/2021/papers/hotos21-s... outlines specific scenarios where CPUs fail over time. Given the…
Lua definitely caught a stray here. The reference VM is pretty fast, the JIT is just way faster. No idea how Lua ended up wrapped in the conversation with Python...