> Reading values from Workers KV is designed to have the same reliability as reading static files, making it much less likely to become unavailable than a traditional database. It’s designed to have the same performance as reading a file cached within our network, close to your users, giving it the speed of serving a static file as well.
I wonder how this kind of speed is achieved with an API that has to go over the network? Even if the round trip is short such as between two AWS services, there's always at least some latency.
Reading cached files goes over the network too. CDN servers share the cache load and have just 1 or a few copies per datacenter that are served by every machine there.
It's likely that this KV store is built on top of the existing cache storage layer, which would also explain the eventual consistency and high-reads with low-writes.
The trick likely has to do with their write semantics. The post states that writes in WorkerKV are ratelimited to 1 a second per key. From there, we can interpret a likely scenario: if at each data center requests for a given client usually route to the same server nodes, and that information is discoverable on write, then it is likely possible for commands to those nodes to be issued during the write. The commands could be as simple as "pull latest value from upstream store". All commands could easily be issued in under a second. Given that the blog states global consistency is reached in around 10 seconds, it seems reasonable to me that all nodes operating on behalf of a client would have finished pulling the updated value by then.
That said, the scenario above is just speculation on my part.
I love it and want to use it, but two announcements of a new cloud hosted distributed DB and no mention of how consistency works or how conflicts are handled worries me a bit.
Maybe one of the cloudflare people that read HN can shed some light on this? What happens when I write to the same key simultaneously from multiple workers and/or multiple edge nodes?
> What happens when I write to the same key simultaneously from multiple workers and/or multiple edge nodes?
It's eventually-consistent. Eventually, one of the writes "wins".
If you do a read/modify/write in the context of processing a single request from a Worker, we'll automatically apply optimistic concurrency so that the write fails if another write has occurred on the same key concurrently.
But otherwise, yes, the ability to handle transactions is limited. For now, Workers KV can be used for a variety of use cases but is not a full replacement for a database.
Workers KV is only the first of multiple storage-related projects we're working on. Stay tuned. :)
As we see how initial developers make use of the storage, we are planning to add additional synchronization and consistency functionality. For example the ETags model using If-None-Match and If-Match would be appealing to some.
That's rather vague definition. And is also wrong. They obviously can't guarantee global consistency within 10 sec, as that would require consensus which is impossible to guarantee in bounded time and they probably don't even use any consensus algorithm at all, as that would be pointless for a CDN.
Yes, in the presences of a network partition, propagation can take arbitrarily long. But, that would be classified as an outage. Knowing the typical non-outage delay is useful.
Going a bit off-topic, I can't access this CloudFlare blog using a Tor browser (using the "standard" security level) without a CAPTCHA.
Didn't we discuss just last week an article[0] by CloudFlare where they said they'd solved this problem and Tor users wouldn't need to solve CAPTCHAs for CloudFlare-fronted services anymore?
What happened to that? I've actually seen way more CloudFlare CAPTCHAs through Tor since that article came out.
12 comments
[ 3.0 ms ] story [ 40.5 ms ] threadListing keys in a namespace and/or TTLs would be nice with KV.
I wonder how this kind of speed is achieved with an API that has to go over the network? Even if the round trip is short such as between two AWS services, there's always at least some latency.
It's likely that this KV store is built on top of the existing cache storage layer, which would also explain the eventual consistency and high-reads with low-writes.
That said, the scenario above is just speculation on my part.
Maybe one of the cloudflare people that read HN can shed some light on this? What happens when I write to the same key simultaneously from multiple workers and/or multiple edge nodes?
It's eventually-consistent. Eventually, one of the writes "wins".
If you do a read/modify/write in the context of processing a single request from a Worker, we'll automatically apply optimistic concurrency so that the write fails if another write has occurred on the same key concurrently.
But otherwise, yes, the ability to handle transactions is limited. For now, Workers KV can be used for a variety of use cases but is not a full replacement for a database.
Workers KV is only the first of multiple storage-related projects we're working on. Stay tuned. :)
Last write wins, for some definition of last (https://en.wikipedia.org/wiki/Eventual_consistency).
As we see how initial developers make use of the storage, we are planning to add additional synchronization and consistency functionality. For example the ETags model using If-None-Match and If-Match would be appealing to some.
Didn't we discuss just last week an article[0] by CloudFlare where they said they'd solved this problem and Tor users wouldn't need to solve CAPTCHAs for CloudFlare-fronted services anymore?
What happened to that? I've actually seen way more CloudFlare CAPTCHAs through Tor since that article came out.
[0] https://news.ycombinator.com/item?id=18031649