I think it's great that projects like this exist where people are building middleware in different ways than others. Still, as someone who routinely uses shared memory queues, the idea of considering a queue built inside a database to be "zero bloat" leaves me scratching my head a bit. I can see why someone would want that, but once person's feature is bloat to someone else.
Postgres durability without having to run Kafka or RabbitMQ clusters seems pretty enticing. May reach for it when I next need an outbox pattern or small fan out.
The vacuum pressure is real. Using a system with the skip locked technique + polling caused massive DB perf issues as the queue depth grew. The query to see the current jobs in the queue ended up being the main performance bottleneck, which cause slower throughput, which caused a larger queue depth, which etc.
Scaling the workers sometimes exacerbates the problem because you run into connection limits or polling hammering the DB.
I love the idea of pg as a queue, but I'm a more skeptical of it after dealing with it in production
Is your comment referring to this project specifically?
Because the docs say:
PgQue avoids that whole class of problems. It uses snapshot-based batching and TRUNCATE-based table rotation instead of per-row deletion.
Would be great if you could specify if you had problems with the exact implementation linked by op or if you did write about a different thing, thanks!
> Bulk loads and deletes can be accomplished by adding or removing partitions, if the usage pattern is accounted for in the partitioning design. Dropping an individual partition using DROP TABLE, or doing ALTER TABLE DETACH PARTITION, is far faster than a bulk operation. These commands also entirely avoid the VACUUM overhead caused by a bulk DELETE.
It was a lot more annoying earlier then pg 13 though, maybe you're just reminiscing things from the 2010s?
> Scaling the workers sometimes exacerbates the problem because you run into connection limits or polling hammering the DB
Design question here (not familiar enough with this approach with Pg)
Would an alternative be to have a small pool of pollers that would "distribute" the records to a later pool of workers instead of having workers directly poll?
I got Claude to analyze the code and it's not really comparable to SKIP LOCKED queues. It's more like Kafka. There's no job queue semantics with acks, workers taking from same job pool.
It's Kafka like one event stream and multiple independent worker cursors.
It's more SNS than SQS or Kafka than Rabbitmq/Nats
Why insist on calling this a queue when it doesn't really have queue semantics? Queues do the job of load balancing between different workers. When workers acknowledge tasks, they get deleted, and there are visibility timeouts.
This is a log.
It's not really solving the problems you claim it solves. It's not, for instance, a replacement for SKIP LOCKED based queues.
18 comments
[ 4.7 ms ] story [ 49.2 ms ] threadThen in the latency tradeof section it says end to end latency is between 1-2 seconds.
Is this under heavy load or always? How does this compare to pgmq end to end latency?
1. SKIP LOCKED family
2. Partition-based + DROP old partitions (no VACUUM required)
3. TRUNCATE family (PgQue’s approach)
And the benefit of PgQue is the failure mode, when a worker gets stuck:
- Table grows indefinitely, instead of
- VACUUM-starved death spiral
And a table growing is easier to reason about operationally?
Scaling the workers sometimes exacerbates the problem because you run into connection limits or polling hammering the DB.
I love the idea of pg as a queue, but I'm a more skeptical of it after dealing with it in production
Because the docs say:
Would be great if you could specify if you had problems with the exact implementation linked by op or if you did write about a different thing, thanks!Were you not using partitions like this?
CREATE TABLE events_2026_04 PARTITION OF events FOR VALUES FROM ('2026-04-01') TO ('2026-05-01');
CREATE TABLE events_2026_05 PARTITION OF events FOR VALUES FROM ('2026-05-01') TO ('2026-06-01');
https://www.postgresql.org/docs/current/ddl-partitioning.htm...
> Bulk loads and deletes can be accomplished by adding or removing partitions, if the usage pattern is accounted for in the partitioning design. Dropping an individual partition using DROP TABLE, or doing ALTER TABLE DETACH PARTITION, is far faster than a bulk operation. These commands also entirely avoid the VACUUM overhead caused by a bulk DELETE.
It was a lot more annoying earlier then pg 13 though, maybe you're just reminiscing things from the 2010s?
Felt like llm for a second.
Would an alternative be to have a small pool of pollers that would "distribute" the records to a later pool of workers instead of having workers directly poll?
It's Kafka like one event stream and multiple independent worker cursors.
It's more SNS than SQS or Kafka than Rabbitmq/Nats
Any database that supports SKIP LOCKED is fine including MySQL, MSSQL, Oracle etc.
Even SQLite makes a fine queue not via skip locked but because writes are atomic.
This is a log.
It's not really solving the problems you claim it solves. It's not, for instance, a replacement for SKIP LOCKED based queues.