Sorry, what? Passing a pointer is a matter of wrapping the value into the CPU register. OTOH passing an offset into a shared memory is a write to main memory so several magnitudes slower.
Thread pools
I'd say it would be most likely impossible.
Obviously, transformers architecture is just one of the ingredients. Otherwise we wouldn't be seeing competing labs in this race. I also read all his interviews as a marketing material.
> Nah, coroutines/async/etc often lives in various threads (ie, the workers can schedule one of them on different threads during the lifetime and they live concurrently), so you still have all the issues of threading (+…
Pickleball pays better than 40€/h
SIMD programmers have to be paid very well then in the China ... Jokes aside, some 2 or 3 years ago I thought that it is becoming inevitable for CPU designs to become an extended versions of their already quite capable…
Yeah, speaking of bias ...
No, they don't force the model to use a json schema, they simply use the model to extract the data, and then they feed that OCR result into the pipeline further to evaluate the OCR results against the ground truth, and…
Quite unlucky CPU to run the experiments. The article doesn't mention it but I hope that the measurement numbers OP got were extracted by re-running the experiment on same type of cores. Intel Core Ultra 7 255H is a mix…
I am not totally following this area but the link from a commenter from above suggests that it is not SOTA but on the lower end (but still good): https://getomni.ai/blog/benchmarking-open-source-models-for-...
That's extremely cheap. I wonder how the companies making a living out of this will survive. They certainly don't charge $0.004 per scan.
I don't think you're wrong. Virtual functions is a two-pointer dereference operation (vptr, vtable[vptr]), and there we can have a d-cache miss but the main cost of using virtual functions is the increased likeliness of…
You're missing the point that when you're building the product, and not using SBC just for fun, the image you are building, and yes even the userspace, is highly customized. Running a full-blown Linux distro is a…
They do but the order of compiling the object files does not matter, at least not in C or C++ compilation model.
I never tackled the reproducible builds topic but this is also something I had in mind while having a thought about it. Building object files for translation units, apart from macros introducing variance, should be…
And what happens when E-core decoder encounters the AVX-512 instruction set? Does this mean that E-cores will have to implement some sort of (downsized) vectorized execution engine too? Otherwise, I don't see how this…
And do you know about Linux? After cursory search I couldn't find anything as well. In this case it seems very expensive, if not impossible without the huge penalty, to implement such a workaround in the hardware.…
Would it not be possible to solve the problem of "thread migration to an incompatible core" on the kernel scheduler level?
I don't think the open source is the differentiator here. I'm sure there's plenty of solid ideas. It would be interesting though if nlnet would open source the contributions they received in which case we would have a…
Lack of funding on the European ground, yes, I do see and NLNet seems to be yet another example of it. If my peers had the idea to develop some PoC, and they don't run the startup, how else would they get some pocket…
I live in Europe btw, and I am surrounded by people being layed off and none of them are the "bootcamp" developers. So, in order to support your view you'd really have to be more concrete, otherwise your response reads…
Very low interest and investment in the AI-related projects. I wonder if it is because there has been no strong contenders or is it because it is not part of the strategy. In any case it looks a bit awkward because I am…
Try being more respectful and inclusive with your request first.
I've used AI but not through Cursor to implement a high-performance serialization library using expression templates. Non-trivial and few kLoC targeting a soft-realtime and critical system. Would you consider that a…
Sorry, what? Passing a pointer is a matter of wrapping the value into the CPU register. OTOH passing an offset into a shared memory is a write to main memory so several magnitudes slower.
Thread pools
I'd say it would be most likely impossible.
Obviously, transformers architecture is just one of the ingredients. Otherwise we wouldn't be seeing competing labs in this race. I also read all his interviews as a marketing material.
> Nah, coroutines/async/etc often lives in various threads (ie, the workers can schedule one of them on different threads during the lifetime and they live concurrently), so you still have all the issues of threading (+…
Pickleball pays better than 40€/h
SIMD programmers have to be paid very well then in the China ... Jokes aside, some 2 or 3 years ago I thought that it is becoming inevitable for CPU designs to become an extended versions of their already quite capable…
Yeah, speaking of bias ...
No, they don't force the model to use a json schema, they simply use the model to extract the data, and then they feed that OCR result into the pipeline further to evaluate the OCR results against the ground truth, and…
Quite unlucky CPU to run the experiments. The article doesn't mention it but I hope that the measurement numbers OP got were extracted by re-running the experiment on same type of cores. Intel Core Ultra 7 255H is a mix…
I am not totally following this area but the link from a commenter from above suggests that it is not SOTA but on the lower end (but still good): https://getomni.ai/blog/benchmarking-open-source-models-for-...
That's extremely cheap. I wonder how the companies making a living out of this will survive. They certainly don't charge $0.004 per scan.
I don't think you're wrong. Virtual functions is a two-pointer dereference operation (vptr, vtable[vptr]), and there we can have a d-cache miss but the main cost of using virtual functions is the increased likeliness of…
You're missing the point that when you're building the product, and not using SBC just for fun, the image you are building, and yes even the userspace, is highly customized. Running a full-blown Linux distro is a…
They do but the order of compiling the object files does not matter, at least not in C or C++ compilation model.
I never tackled the reproducible builds topic but this is also something I had in mind while having a thought about it. Building object files for translation units, apart from macros introducing variance, should be…
And what happens when E-core decoder encounters the AVX-512 instruction set? Does this mean that E-cores will have to implement some sort of (downsized) vectorized execution engine too? Otherwise, I don't see how this…
And do you know about Linux? After cursory search I couldn't find anything as well. In this case it seems very expensive, if not impossible without the huge penalty, to implement such a workaround in the hardware.…
Would it not be possible to solve the problem of "thread migration to an incompatible core" on the kernel scheduler level?
I don't think the open source is the differentiator here. I'm sure there's plenty of solid ideas. It would be interesting though if nlnet would open source the contributions they received in which case we would have a…
Lack of funding on the European ground, yes, I do see and NLNet seems to be yet another example of it. If my peers had the idea to develop some PoC, and they don't run the startup, how else would they get some pocket…
I live in Europe btw, and I am surrounded by people being layed off and none of them are the "bootcamp" developers. So, in order to support your view you'd really have to be more concrete, otherwise your response reads…
Very low interest and investment in the AI-related projects. I wonder if it is because there has been no strong contenders or is it because it is not part of the strategy. In any case it looks a bit awkward because I am…
Try being more respectful and inclusive with your request first.
I've used AI but not through Cursor to implement a high-performance serialization library using expression templates. Non-trivial and few kLoC targeting a soft-realtime and critical system. Would you consider that a…