High cardinality on Promscale is very efficient. I have been testing prom-migrator, migrating and ingesting 7 million+ samples a second, and till now, I have seen Promscale's memory usage to be the most efficient, along with a better CPU performance. I have tested Promscale, VictoriaMetrics, Thanos, M3, Cortex, and Promscale continues to lead in efficiency and performance. Also, the added advantage of native SQL support is a big bonus.
We went from Prometheus (bursting at the seams and begging to be scaled) -> a failed Cortex implementation -> a rock solid Victoria Metrics implementation.
We found Cortex to be extremely over-engineered, extremely hard to tune (because of multiple configuration/argument refactors with incomplete doc cleanups) and found that it would just fall over under load. (Though to be fair, there wasn't much in the way of first-party deployment tooling at the time, so it was a hand-rolled Helm chart and at least 50% of the issues were likely my config).
In comparison, the first-party Victoria Metrics Helm chart worked straight out of the box, the maintainers have fixed multiple small issues within hours of me reporting them and we've thrown an extremely large amount of metrics at it with zero problems.
(Promscale Team Lead Here) Promscale, which connects Prometheus to TimescaleDB also handles high-cardinality well and allows you to use both PromQL and SQL for data analysis. I'd humbly suggest taking a look. Plus our PromQL implementation is 100% standards compliant.
* It is based on real-world data instead of synthetic data.
* It measures resources usage (RAM, CPU, disk IO, disk space) during production-like load instead of measuring peak performance under the maximum synthetic load.
I would love to have some kind of integration between Victoria Metrics and Promscale, using VM for ingest from things like InfluxDB and Promscale for SQL queries on stable data.
This is what is the problem with such compatibility tests... they tend to test full power of the language while you really may use quite small subset in your application, as such even solution which is "20% compatible" may well meet all your application needs.
I remember in its early days MySQL had pretty poor SQL support (if you think about full standard) which did not prevent it from having huge success.
Or more recent example ClickHouse which I think similar to VictoriaMetrics as it does not fully implement SQL, but also adds many convenient extensions which are not part of the standard.
Chances are if you chose VictoriaMetrics you will find a lot more utility in advanced features of MetricsQL than you loose from exact compatibility with PromQL
The majority of failed tests for VictoriaMetrics cannot be "fixed" due to deliberate choice made when designing MetricsQL: to rethink and to fix the most annoying and confusing parts of PromQL, while providing drop-in PromQL replacement for the majority of practical cases. See more details at https://victoriametrics.github.io/MetricsQL.html .
14 comments
[ 4.2 ms ] story [ 34.4 ms ] threadThis summary makes it seem like Victoria Metrics is barely compatible with Prometheus, but that can't be further from the truth in practice.
We're currently running Prometheus + Thanos, and high cardinality timeseries are a real issue, which Victoria claims to be good at.
We found Cortex to be extremely over-engineered, extremely hard to tune (because of multiple configuration/argument refactors with incomplete doc cleanups) and found that it would just fall over under load. (Though to be fair, there wasn't much in the way of first-party deployment tooling at the time, so it was a hand-rolled Helm chart and at least 50% of the issues were likely my config).
In comparison, the first-party Victoria Metrics Helm chart worked straight out of the box, the maintainers have fixed multiple small issues within hours of me reporting them and we've thrown an extremely large amount of metrics at it with zero problems.
* It is based on real-world data instead of synthetic data.
* It measures resources usage (RAM, CPU, disk IO, disk space) during production-like load instead of measuring peak performance under the maximum synthetic load.
[1] https://victoriametrics.github.io/vmagent.html
I remember in its early days MySQL had pretty poor SQL support (if you think about full standard) which did not prevent it from having huge success.
Or more recent example ClickHouse which I think similar to VictoriaMetrics as it does not fully implement SQL, but also adds many convenient extensions which are not part of the standard.
Chances are if you chose VictoriaMetrics you will find a lot more utility in advanced features of MetricsQL than you loose from exact compatibility with PromQL
Disclaimer: I work on Promscale!
These tests are great, because they spotted a few minor bugs in our PromQL implementation and these bugs were fixed quickly. See https://victoriametrics.github.io/CHANGELOG.html .
The majority of failed tests for VictoriaMetrics cannot be "fixed" due to deliberate choice made when designing MetricsQL: to rethink and to fix the most annoying and confusing parts of PromQL, while providing drop-in PromQL replacement for the majority of practical cases. See more details at https://victoriametrics.github.io/MetricsQL.html .
How do you include that in a CI/CD environment with regression testing?