ClickHouse is pretty good at compressing the wide events, so it's not that dramatic compared to the benefits of having high-cardinality telemetry. check this out:…
> Do wide events really have to take up this much space? 100PB is the total volume of the raw, uncompressed data for the full retention period (180 days). compression is what makes it cost-efficient. on this dataset, we…
we use k8s + otel filelog receiver. in this case you don't have to connect to the clickhouse instance to collect what it's writing to stdout/stderr, just tail /var/log/pods/*/*/*.log.
we keep it for 180 days.
> Anecdotally I know robust tracing data can help me find problems in 5-15 minutes that would have taken hours or days with manual probing and scouring logs. exactly. high-cardinality, wide structured events are the way.
hey there! I work at ClickHouse. to clarify: the vast majority of this 100PB is structured events. in our case logs are supplementary.
ClickHouse is pretty good at compressing the wide events, so it's not that dramatic compared to the benefits of having high-cardinality telemetry. check this out:…
> Do wide events really have to take up this much space? 100PB is the total volume of the raw, uncompressed data for the full retention period (180 days). compression is what makes it cost-efficient. on this dataset, we…
we use k8s + otel filelog receiver. in this case you don't have to connect to the clickhouse instance to collect what it's writing to stdout/stderr, just tail /var/log/pods/*/*/*.log.
we keep it for 180 days.
> Anecdotally I know robust tracing data can help me find problems in 5-15 minutes that would have taken hours or days with manual probing and scouring logs. exactly. high-cardinality, wide structured events are the way.
hey there! I work at ClickHouse. to clarify: the vast majority of this 100PB is structured events. in our case logs are supplementary.