There seem to be two distinct topics: 1) async programming vs. threading 2) infectious async/await syntax Async programming is great. Coroutines are a powerful tool, both for expressing your ideas more clearly and for…
Sure, knowing the basics of LLM math is necessary. But it's also _enough_ to know this math to fully grasp the code. There are only 4 concepts - attention, feed-forward net, RMS-normalization and rotary embeddings -…
It's also worth mentioning that the original implementation by Meta is only 300 lines of very readable code [1]. [1]: https://github.com/meta-llama/llama3/blob/main/llama/model.p...
JAX requires a bit more work to maintain fixed-size buffers as required by XLA, especially in case of caching and rotary embeddings. But yeah, overall the code can be pretty similar [1]. [1]:…
> there's nothing in the language that prevents this from working with the autograd package, except no one's taken the time to implement it (https://github.com/HIPS/autograd/issues/47). I believe it's more complicated…
I have quite limited experience with Cython and tried Numba just a couple of times, but I'm curious how much would it take to rewrite one of my Julia libraries to them. The library is for reverse-mode automatic…
Almost certainly yes. Although it doesn't mean it's the best choice for your specific task. Python had a long history of web server development. I think, aiohttp may be considered state of the art now. Let's measure its…
Some time ago I implemented a library [1] similar to this tool. The tricky part is that derivatives quickly exceed 2 dimensions, e.g. derivative of a vector output w.r.t. input matrix is a 3D tensor (e.g. if `y = f(X)`,…
If I want to export my own computational graph to ONNX, what is the first place I should look at? Do you know about any documentation or reference implementation of the format?
> I don’t understand their vendetta against the graph, which is a powerful abstraction that lets you choose different backend [...] You don't really need a graph to support different backends. One popular approach is to…
I could have not noticed use of swap in the previous test, so I repeated it on a Linux box and 1e8 numbers (instead of 1e9). Julia took 100.583ms while Python 207ms (probably due to double reading of the array). So I…
BLAS really shines when you do matrix multiplication, for element-wise operations the best you can do is to add numbers using SIMD instructions or put the load to GPU, and most numeric libraries already when possible.…
Even better would be to write (see dot vectorization [1]): C .= A .+ B Benchmarks for 3 matrices of size 1000x1000: julia> using BenchmarkTools julia> @benchmark C = A + B BenchmarkTools.Trial: memory estimate: 7.63 MiB…
> The author understands your perspective but he's deliberately using a different one. In this case the way the author shows it isn't the best one: he modifies Python code to be more realistic - that's ok, but doesn't…
The two language problem is more specific to scientific computing where literally every popular library is written in more then one language - fast C/C++/Fortran and convenient high-level R/Matlab/Python. NumPy, SciPy,…
As far as I know, storing personal data - including photo, name, email and sometimes even IP address - without explicit and clear consent is strictly forbidden in most countries, at least in EU.
If we are talking about professional actors trying to trick the tracker, then yes, it should be pretty hard to design software to overcome it. But most people aren't that good, and although they can mislead their…
Well, I definitely do have other applications for it. For example, I know that similar software has been used in labs to estimate people's reaction to videos and game features, in mobile applications to improve…
> I would feel bad about it and I think the person should ask my permission first. But if that person just memorizes customer reactions to understand how people on average react to particular products or actions, that's…
I'm curious what's the boundary between ethical and unethical. People constantly analyze each other's mood and it's perceived positively. But doing the same thing massively using automated tools is often considered…
But isn't A/B testing doing the same thing? If it's different, what's the key difference between analyzing facial expression in a shop and analyzing user interaction on a site (given that both have a warning about data…
I see 2 separate issues here: 1. Accurate collection of facial features. Illumination, occlusions, head rotation, etc. may seriously affect accuracy, but this is exactly our main focus right now. We are at the very…
> I don't like having my emotions manipulated What about collecting statistics to make better decisions? Let's say, you go to your favorite jeans store, but find out that current collection is disgusting. Does it sound…
Thanks for the detailed comment. Couple clarifying questions if you don't mind: - do you know that loyalty cards are often used in stores to collect customer data (a kind of offline cookie)? do you consider it a…
As someone working on a similar project (specifically, emotion recognition) I'm highly interested to hear how such a product should look like to be not considered unethical. So far from the comments I see that: - it…
There seem to be two distinct topics: 1) async programming vs. threading 2) infectious async/await syntax Async programming is great. Coroutines are a powerful tool, both for expressing your ideas more clearly and for…
Sure, knowing the basics of LLM math is necessary. But it's also _enough_ to know this math to fully grasp the code. There are only 4 concepts - attention, feed-forward net, RMS-normalization and rotary embeddings -…
It's also worth mentioning that the original implementation by Meta is only 300 lines of very readable code [1]. [1]: https://github.com/meta-llama/llama3/blob/main/llama/model.p...
JAX requires a bit more work to maintain fixed-size buffers as required by XLA, especially in case of caching and rotary embeddings. But yeah, overall the code can be pretty similar [1]. [1]:…
> there's nothing in the language that prevents this from working with the autograd package, except no one's taken the time to implement it (https://github.com/HIPS/autograd/issues/47). I believe it's more complicated…
I have quite limited experience with Cython and tried Numba just a couple of times, but I'm curious how much would it take to rewrite one of my Julia libraries to them. The library is for reverse-mode automatic…
Almost certainly yes. Although it doesn't mean it's the best choice for your specific task. Python had a long history of web server development. I think, aiohttp may be considered state of the art now. Let's measure its…
Some time ago I implemented a library [1] similar to this tool. The tricky part is that derivatives quickly exceed 2 dimensions, e.g. derivative of a vector output w.r.t. input matrix is a 3D tensor (e.g. if `y = f(X)`,…
If I want to export my own computational graph to ONNX, what is the first place I should look at? Do you know about any documentation or reference implementation of the format?
> I don’t understand their vendetta against the graph, which is a powerful abstraction that lets you choose different backend [...] You don't really need a graph to support different backends. One popular approach is to…
I could have not noticed use of swap in the previous test, so I repeated it on a Linux box and 1e8 numbers (instead of 1e9). Julia took 100.583ms while Python 207ms (probably due to double reading of the array). So I…
BLAS really shines when you do matrix multiplication, for element-wise operations the best you can do is to add numbers using SIMD instructions or put the load to GPU, and most numeric libraries already when possible.…
Even better would be to write (see dot vectorization [1]): C .= A .+ B Benchmarks for 3 matrices of size 1000x1000: julia> using BenchmarkTools julia> @benchmark C = A + B BenchmarkTools.Trial: memory estimate: 7.63 MiB…
> The author understands your perspective but he's deliberately using a different one. In this case the way the author shows it isn't the best one: he modifies Python code to be more realistic - that's ok, but doesn't…
The two language problem is more specific to scientific computing where literally every popular library is written in more then one language - fast C/C++/Fortran and convenient high-level R/Matlab/Python. NumPy, SciPy,…
As far as I know, storing personal data - including photo, name, email and sometimes even IP address - without explicit and clear consent is strictly forbidden in most countries, at least in EU.
If we are talking about professional actors trying to trick the tracker, then yes, it should be pretty hard to design software to overcome it. But most people aren't that good, and although they can mislead their…
Well, I definitely do have other applications for it. For example, I know that similar software has been used in labs to estimate people's reaction to videos and game features, in mobile applications to improve…
> I would feel bad about it and I think the person should ask my permission first. But if that person just memorizes customer reactions to understand how people on average react to particular products or actions, that's…
I'm curious what's the boundary between ethical and unethical. People constantly analyze each other's mood and it's perceived positively. But doing the same thing massively using automated tools is often considered…
But isn't A/B testing doing the same thing? If it's different, what's the key difference between analyzing facial expression in a shop and analyzing user interaction on a site (given that both have a warning about data…
I see 2 separate issues here: 1. Accurate collection of facial features. Illumination, occlusions, head rotation, etc. may seriously affect accuracy, but this is exactly our main focus right now. We are at the very…
> I don't like having my emotions manipulated What about collecting statistics to make better decisions? Let's say, you go to your favorite jeans store, but find out that current collection is disgusting. Does it sound…
Thanks for the detailed comment. Couple clarifying questions if you don't mind: - do you know that loyalty cards are often used in stores to collect customer data (a kind of offline cookie)? do you consider it a…
As someone working on a similar project (specifically, emotion recognition) I'm highly interested to hear how such a product should look like to be not considered unethical. So far from the comments I see that: - it…