Did you consider using an B-epsilon-tree [1] ? This is a B-tree, but each node has a staging area for new writes. In practice, incoming writes get added to the staging area of the root node, and once a staging area is…
Any background info/links to this? The Wyze forum comes up empty, only some discussions on WebRTC being complex to add.
This would be interesting to integrate into https://github.com/openmiko/openmiko which is a firmware for the T20 based ip-cameras. Right now, I am not aware of any cheap ip camera that can stream its H264 video to a…
Haxl is actually an example of "the more impressive and more useful" approach: it is an abstraction embedded within a Haskell library, that allows you to express your "intent" in simple, standard Haskell code, decoupled…
This indeed seems to be the main/only point of criticism in this post that is valid: however, it is not that Haskell has no "pragmatic" libraries to get stuff done (e.g. WAI/Warp, Yesod, Servant, ... are top notch,…
The issue is that it hasn't translated into local production fabs. Which means that the semiconductor ecosystems in Belgium is still really small. IMEC's research mainly benefits companies like Intel, Samsung, TSMC. The…
ReScript (formerly called ReasonML), is an interesting option for web apps (especially the front-end part), if you are interested in Ocaml-like languages. Language-wise, it is Ocaml with a different syntax layer, so all…
In your example, there is no relation between anything in Module1Interface and Module2Interface. Probably closer would be (not sure if this is possible in Typescript): class CustomModule<S, T1 extends…
For me, the value of a module is that you can describe a set of multiple types and the functions that operate on these types, all in one place. In OO, there is essentially one type that is "special", the type of the…
Glass platters have been used a lot since the IBM "Deathstar" series. They are used in almost all 2.5 inch laptop drives.
I think what you witnessed is the fact that being an expert in programming language theory or category theory, does not make you a good software engineer. Haskell is surprisingly well suited for back-end services (e.g…
I fail to see how else than "primitive" you can call the Go type system: it is roughly what you would get from a language designed in the 70's, and a lot of advances have since been made. Whether or not these advances…
I have not yet seen a storage system that is actually 100% S3 compatible. Either functionality is missing, and/or there are corner cases where the behavior is just slightly different. It is also a complicated API, that…
My experience with using general purpose languages for things like configuration or build systems has not been great. You always seem to end up with building a bunch of abstractions on top of that language, which…
I worked at a startup (Amplidata) where we build a distributed storage system in OCaml. It got acquired by Western Digital, which later sold it to Quantum. Afaik, there is still development going on using OCaml. Couple…
MIPS is still alive in the IP camera world. There exist very cheap SoC's (e.g. the Ingenic T20 - http://www.ingenic.com.cn/en/?product/id/14.html ), tailored towards making cheap network camera's (~ €20 retail price for…
I hacked together a proof of concept of using Janus to wrap the H264 RTSP stream of the Xiaomi Dafang into a WebRTC stream without transcoding, giving close to real-time streaming of the ip camera to browsers without…
They have been doing erasure coding since at least 2016 (it is mentioned in their original Magic Pocket post), just like almost all distributed storage system, so nothing really new there. This article is about…
The Dropbox technical blog is always a very interesting read, I love that they provide so much detail into the technical background of their solution. Still was left with quite some questions when trying to understand…
Does Backblaze have any SMR drives in use, or do you have plans to start using them? Would be really interesting to see if these exhibit a worse failure rate. Anyway, thanks for sharing this data, I really like the…
CRUSH is an example (and not the first) of a "distributed rebuild" approach: you have an array of N drives (with N large, e.g. 100), and if 1 drive fails, you read in parallel from all (N-1) remaining drives, while…
A full blown filesystem is overkill for an object store. You could use something like libzbc ( https://github.com/hgst/libzbc ) to write directly to the SMR drives on the block level. I believe Ceph now has abstracted…
If you are the target audience for these drives, you likely don't even consider regular (i.e. including random) IO as a use-case for modern HDD's . 10+ TB HDD's really only make sense for sequential IO, even if they…
There is also Intel's ISA-L: https://github.com/01org/isa-l It contains optimized Galois Field multiplication, resulting in Reed Solomon (both Vandermonde as well as Cauchy) at multiple GB/s on a modern x86 CPU. Still,…
I believe TokuDB is using a flavor of B epsilon trees, i.e. a B tree with, per node, an associated staging area for writes to that node or its child nodes. Keeps inserts localised at the top of the tree, until the…
Did you consider using an B-epsilon-tree [1] ? This is a B-tree, but each node has a staging area for new writes. In practice, incoming writes get added to the staging area of the root node, and once a staging area is…
Any background info/links to this? The Wyze forum comes up empty, only some discussions on WebRTC being complex to add.
This would be interesting to integrate into https://github.com/openmiko/openmiko which is a firmware for the T20 based ip-cameras. Right now, I am not aware of any cheap ip camera that can stream its H264 video to a…
Haxl is actually an example of "the more impressive and more useful" approach: it is an abstraction embedded within a Haskell library, that allows you to express your "intent" in simple, standard Haskell code, decoupled…
This indeed seems to be the main/only point of criticism in this post that is valid: however, it is not that Haskell has no "pragmatic" libraries to get stuff done (e.g. WAI/Warp, Yesod, Servant, ... are top notch,…
The issue is that it hasn't translated into local production fabs. Which means that the semiconductor ecosystems in Belgium is still really small. IMEC's research mainly benefits companies like Intel, Samsung, TSMC. The…
ReScript (formerly called ReasonML), is an interesting option for web apps (especially the front-end part), if you are interested in Ocaml-like languages. Language-wise, it is Ocaml with a different syntax layer, so all…
In your example, there is no relation between anything in Module1Interface and Module2Interface. Probably closer would be (not sure if this is possible in Typescript): class CustomModule<S, T1 extends…
For me, the value of a module is that you can describe a set of multiple types and the functions that operate on these types, all in one place. In OO, there is essentially one type that is "special", the type of the…
Glass platters have been used a lot since the IBM "Deathstar" series. They are used in almost all 2.5 inch laptop drives.
I think what you witnessed is the fact that being an expert in programming language theory or category theory, does not make you a good software engineer. Haskell is surprisingly well suited for back-end services (e.g…
I fail to see how else than "primitive" you can call the Go type system: it is roughly what you would get from a language designed in the 70's, and a lot of advances have since been made. Whether or not these advances…
I have not yet seen a storage system that is actually 100% S3 compatible. Either functionality is missing, and/or there are corner cases where the behavior is just slightly different. It is also a complicated API, that…
My experience with using general purpose languages for things like configuration or build systems has not been great. You always seem to end up with building a bunch of abstractions on top of that language, which…
I worked at a startup (Amplidata) where we build a distributed storage system in OCaml. It got acquired by Western Digital, which later sold it to Quantum. Afaik, there is still development going on using OCaml. Couple…
MIPS is still alive in the IP camera world. There exist very cheap SoC's (e.g. the Ingenic T20 - http://www.ingenic.com.cn/en/?product/id/14.html ), tailored towards making cheap network camera's (~ €20 retail price for…
I hacked together a proof of concept of using Janus to wrap the H264 RTSP stream of the Xiaomi Dafang into a WebRTC stream without transcoding, giving close to real-time streaming of the ip camera to browsers without…
They have been doing erasure coding since at least 2016 (it is mentioned in their original Magic Pocket post), just like almost all distributed storage system, so nothing really new there. This article is about…
The Dropbox technical blog is always a very interesting read, I love that they provide so much detail into the technical background of their solution. Still was left with quite some questions when trying to understand…
Does Backblaze have any SMR drives in use, or do you have plans to start using them? Would be really interesting to see if these exhibit a worse failure rate. Anyway, thanks for sharing this data, I really like the…
CRUSH is an example (and not the first) of a "distributed rebuild" approach: you have an array of N drives (with N large, e.g. 100), and if 1 drive fails, you read in parallel from all (N-1) remaining drives, while…
A full blown filesystem is overkill for an object store. You could use something like libzbc ( https://github.com/hgst/libzbc ) to write directly to the SMR drives on the block level. I believe Ceph now has abstracted…
If you are the target audience for these drives, you likely don't even consider regular (i.e. including random) IO as a use-case for modern HDD's . 10+ TB HDD's really only make sense for sequential IO, even if they…
There is also Intel's ISA-L: https://github.com/01org/isa-l It contains optimized Galois Field multiplication, resulting in Reed Solomon (both Vandermonde as well as Cauchy) at multiple GB/s on a modern x86 CPU. Still,…
I believe TokuDB is using a flavor of B epsilon trees, i.e. a B tree with, per node, an associated staging area for writes to that node or its child nodes. Keeps inserts localised at the top of the tree, until the…