The MIT course for Raft (with the lectures on YouTube) was great to learn for me: http://nil.lcs.mit.edu/6.824/2020/labs/lab-raft.html
The durability and transformation reasons are definitely more compelling, but the article doesn’t mention those reasons. It’s mainly focused on the insert batching which is why I was drawing attention to async_insert. I…
Sure, but the article doesn’t talk about that, it seemed to be focused on CH alone, in which case async insert is much fewer technical tokens. If you need to ensure that you have super durable writes, you can consider,…
Why add RedPanda/Kafka over using async insert? https://clickhouse.com/docs/optimize/asynchronous-inserts It’s recommended in the docs over the Buffer table, and is pretty much invisible to the end user. At ClickHouse…
Yeah, handles all the OTel signals
Not OP, but to me, this reads fairly similar to how ClickHouse can be set up, with Bloom filters, MinMax indexes, etc. A way to “handle” partial substrings is to break up your input data into tokens (like substrings…
Quite like Cloudprober for this tbh: https://cloudprober.org/docs/how-to/alerting/ Easy to configure, easy to extend with Go, and slots in to alerting.
If you choose not to travel, then you are eligible for a refund, rather than Delay Repay: https://www.nationalrail.co.uk/help-and-assistance/compensat... There’s differences in consumer rights effectively between a…
Claiming delay compensation if you don’t have intent to travel is the fraud part. Easiest example is if you have a season ticket, but you have the day off. You weren’t going to take the train to work that day, so no…
I don’t think it’s fraud on the DR side if you actually take the trains and intend to travel. If you didn’t actually intend to travel, then claiming DR is fraud.
The DR system doesn’t look at ticket scans alone. It also builds a profile per customer based on a number data points. It will flag up quite quickly if you are “sniping” delayed trains at different times.
A small question on the schema, I noticed that you have only “_now” as the Order By (so should just use that for the primary key). Do you expect any cross tenant queries? Just my feeling would be that I’d add the tenant…
Thoughts on stuff like ClickHouse with JSON column support? Less upfront knowledge of columns needed.
https://docs.aws.amazon.com/organizations/latest/APIReferenc... Now has an API (with some caveats)
I think some of the best technical writing I've enjoyed is: https://aws.amazon.com/builders-library/ Clear and concise articles that really dig into some of the hard technical problems with working at scale. Has…
TBH, as the Dockerfile based Lambda layers have a 20GB file size limit, I'm not sure the size matters as much. It's certainly easier to figure out what's going on in a smaller container though! I've had to debug some…
Apparently Python rules the roost for Lambda: https://mobile.twitter.com/julian_wood/status/14427755423742... But NodeJS is second!
I used BPF (using _both_ of Brendan Gregg's recent books!) on our Jenkins builds recently to figure out why `TRUNCATE` statements were taking longer than I expected. The underlying cause was that `ext4` filesystem was…
The MIT course for Raft (with the lectures on YouTube) was great to learn for me: http://nil.lcs.mit.edu/6.824/2020/labs/lab-raft.html
The durability and transformation reasons are definitely more compelling, but the article doesn’t mention those reasons. It’s mainly focused on the insert batching which is why I was drawing attention to async_insert. I…
Sure, but the article doesn’t talk about that, it seemed to be focused on CH alone, in which case async insert is much fewer technical tokens. If you need to ensure that you have super durable writes, you can consider,…
Why add RedPanda/Kafka over using async insert? https://clickhouse.com/docs/optimize/asynchronous-inserts It’s recommended in the docs over the Buffer table, and is pretty much invisible to the end user. At ClickHouse…
Yeah, handles all the OTel signals
Not OP, but to me, this reads fairly similar to how ClickHouse can be set up, with Bloom filters, MinMax indexes, etc. A way to “handle” partial substrings is to break up your input data into tokens (like substrings…
Quite like Cloudprober for this tbh: https://cloudprober.org/docs/how-to/alerting/ Easy to configure, easy to extend with Go, and slots in to alerting.
If you choose not to travel, then you are eligible for a refund, rather than Delay Repay: https://www.nationalrail.co.uk/help-and-assistance/compensat... There’s differences in consumer rights effectively between a…
Claiming delay compensation if you don’t have intent to travel is the fraud part. Easiest example is if you have a season ticket, but you have the day off. You weren’t going to take the train to work that day, so no…
I don’t think it’s fraud on the DR side if you actually take the trains and intend to travel. If you didn’t actually intend to travel, then claiming DR is fraud.
The DR system doesn’t look at ticket scans alone. It also builds a profile per customer based on a number data points. It will flag up quite quickly if you are “sniping” delayed trains at different times.
A small question on the schema, I noticed that you have only “_now” as the Order By (so should just use that for the primary key). Do you expect any cross tenant queries? Just my feeling would be that I’d add the tenant…
Thoughts on stuff like ClickHouse with JSON column support? Less upfront knowledge of columns needed.
https://docs.aws.amazon.com/organizations/latest/APIReferenc... Now has an API (with some caveats)
I think some of the best technical writing I've enjoyed is: https://aws.amazon.com/builders-library/ Clear and concise articles that really dig into some of the hard technical problems with working at scale. Has…
TBH, as the Dockerfile based Lambda layers have a 20GB file size limit, I'm not sure the size matters as much. It's certainly easier to figure out what's going on in a smaller container though! I've had to debug some…
Apparently Python rules the roost for Lambda: https://mobile.twitter.com/julian_wood/status/14427755423742... But NodeJS is second!
I used BPF (using _both_ of Brendan Gregg's recent books!) on our Jenkins builds recently to figure out why `TRUNCATE` statements were taking longer than I expected. The underlying cause was that `ext4` filesystem was…