> FriendlyId v3.x prevented this by keeping the slug sequence in a separate place in the database, and operating upon it within a transaction, thus guaranteeing atomic access to reading, incrementing, and updating the slug sequence.
So he created a 'solution' where no problem exists. At least, TIL an new word: slug.
>So, the problem is since friendly_id v4, the gem no longer guarantees two really important principles:
>No guarantee that we're the only person reading this value, right now
>No guarantee that the ID we'll choose will be unique a few µseconds from now.
The problem exists in the newer version. If anything, the developers created a problem where none existed, not him. They traded consistency for simplicity without a clear mention of the possible race conditions.
>> No guarantee that the ID we'll choose will be unique a few µseconds from now.
That's why you let the database atomically create the next id for you. There is no problem. You either get the next unique id or you have to try again.
> Sorry, but slugs serve a real purpose, so your solution is only a solution in the sense that walking is a solution for a broken down car.
This doesn't replace slugs, at all - the slugs are still generated, this replaces the SQL unique sequencer. The SQL sequencer which isn't universally supported, and has a different API on every platform.. many modern applications have Redis or a similar atomic structure store which could provide an interim solution, short of building a purpose built ticket provisioning system such as that employed by Flickr.
This problem can also be solved with SQL. An after trigger on the table and an indexed computed column would all work. If you're using Postgres probably just a regular INSERT with a call to curval.
This seems like an awful lot of work to avoid a stored proc / trigger, computed columns, or custom INSERT statement. Even in MySQL a stored proc would work, or two INSERT statements in a transaction.
Minor correction: You call nextval in PostgreSQL to increment a sequence and get the number. curval is for getting the last value returned by a nextval call within the current transaction. Since PostgreSQL implemented RETURNING for INSERTs and UPDATEs curval has become more of a utility function for use with psql and similar clients.
I would definitely contest that you have to lock the whole database table for readers. Modern SQL systems will not have readers waiting upon writers. And if you use a SEQUENCE, like you probably should in this case, then you only need a sequence-lock once in a while.
There are also fully independent schemes for this, but they cannot, usually give you ordering properties of your counter sequence. And those who can will have rather large keys (large in size of bytes)
Of course I wanted sequential, small IDs, within a small margin of acceptable loss (if a save fails that ID is never used), that were't taking up a massive amount of space.
To solve this problem is why many databases support sequences. Sequences are number generators which work outside the transactional system (e.g. sequence incrementations cannot be rolled back) so the locks are very short.
To use sequences in PostgreSQL use simply call nextval('name_of_sequence') in the database. Rails uses the SERIAL type for the primary key column which is implemented with a sequence.
I'm weighing in here on the comments about "Solving a problem that doesn't exist" - the database is quite loaded, and we have Redis in the stack anyway. The solution to this problem is any backend which creates unique numbers atomically; I happened to choose Redis because I'm interested in it technologically and I like the API.
It's worth mentioning that Rails doesn't encourage access to the underlying database features - and that stepping outside of the Rails/ActiveRecord results in features which are difficult to test and more difficult to maintain in the face of an ever changing API.
I don't care where the next atomic ID comes from, as long as it comes from a datastore that is fast, and that I can trust.
> I'm weighing in here on the comments about "Solving a problem that doesn't exist" - the database is quite loaded
Can you give a ballpark estimate of your write load? And in any case, a server-side trigger executed on insert could handle the process much closer to the data, with fewer round trips and much greater performance.
> I happened to choose Redis because I'm interested in it technologically and I like the API.
With due respect, you've gotta get over that. I'm all about experimentation, but there's a time to break new ground and a time to use the tools that others have already developed and tested.
> It's worth mentioning that Rails doesn't encourage access to the underlying database features
That may be true, but if you're pushing it near the edge of breaking, it may be time to bite the bullet and move small amounts of logic into the database. It's only very slightly harder to test - the general principals are the same - and I'd dare say far easier to maintain.
It is a very isolated change so my view is that relying on your database is worth it. Furthermore sequences are very cheap in most databases, I have never heard anyone complaining about sequences in PostgreSQL being to contention point.
17 comments
[ 4.6 ms ] story [ 51.7 ms ] threadSo he created a 'solution' where no problem exists. At least, TIL an new word: slug.
>No guarantee that we're the only person reading this value, right now
>No guarantee that the ID we'll choose will be unique a few µseconds from now.
The problem exists in the newer version. If anything, the developers created a problem where none existed, not him. They traded consistency for simplicity without a clear mention of the possible race conditions.
That's why you let the database atomically create the next id for you. There is no problem. You either get the next unique id or you have to try again.
Sorry, but slugs serve a real purpose, so your solution is only a solution in the sense that walking is a solution for a broken down car.
This doesn't replace slugs, at all - the slugs are still generated, this replaces the SQL unique sequencer. The SQL sequencer which isn't universally supported, and has a different API on every platform.. many modern applications have Redis or a similar atomic structure store which could provide an interim solution, short of building a purpose built ticket provisioning system such as that employed by Flickr.
This seems like an awful lot of work to avoid a stored proc / trigger, computed columns, or custom INSERT statement. Even in MySQL a stored proc would work, or two INSERT statements in a transaction.
There are also fully independent schemes for this, but they cannot, usually give you ordering properties of your counter sequence. And those who can will have rather large keys (large in size of bytes)
He's invented this ridiculous Rube Goldberg contraption because his knowledge of RDBMSs is 20 years or more out of date...
Simple, really.
Of course Twitter have snowflake: https://github.com/twitter/snowflake and many other tools such as UUID[4r] - see an old discussion on HN: http://news.ycombinator.com/item?id=1396667
Of course I wanted sequential, small IDs, within a small margin of acceptable loss (if a save fails that ID is never used), that were't taking up a massive amount of space.
To use sequences in PostgreSQL use simply call nextval('name_of_sequence') in the database. Rails uses the SERIAL type for the primary key column which is implemented with a sequence.
It's worth mentioning that Rails doesn't encourage access to the underlying database features - and that stepping outside of the Rails/ActiveRecord results in features which are difficult to test and more difficult to maintain in the face of an ever changing API.
I don't care where the next atomic ID comes from, as long as it comes from a datastore that is fast, and that I can trust.
Can you give a ballpark estimate of your write load? And in any case, a server-side trigger executed on insert could handle the process much closer to the data, with fewer round trips and much greater performance.
> I happened to choose Redis because I'm interested in it technologically and I like the API.
With due respect, you've gotta get over that. I'm all about experimentation, but there's a time to break new ground and a time to use the tools that others have already developed and tested.
> It's worth mentioning that Rails doesn't encourage access to the underlying database features
That may be true, but if you're pushing it near the edge of breaking, it may be time to bite the bullet and move small amounts of logic into the database. It's only very slightly harder to test - the general principals are the same - and I'd dare say far easier to maintain.
Something like an infinite with Fiber.yield returning a unique ID or something.