Hey all, I made this post to show a technique that I'm using in my keyboard-centric flowchart editor [0]. I really love urls, and the power that they hold, and would love to see more apps implement little hacks like this.
Also, another shout out to mermaidjs and flowchart.fun for also implementing similar url-based sharing.
This is the right answer here. The solution works until it doesn’t. Browser URLs are limited in real life. So when the state grows it will break sooner or later. Of course it depends on the usecase.
(maker of flowchart.fun here) This is technically true but it's a little more nuanced. The 2048 character limit comes from Internet Explorer. There's a great answer on SO https://stackoverflow.com/a/417184/903980
Still on FF I opted to use LZ Compression (pretty sure many other sites do this as well) to get the number of characters down
I have done this myself and various tools e.g. slack, Twitter will truncate long urls. Eventually it will break somewhere, not just in the browser. If your tool has a hard limit on information content it's ok, but that does not seem to be the case here.
Store the content in ipfs and just put the hash in the URL?
I haven't used https://github.com/ipfs/js-ipfs in this capacity but I'm under the impression that that's moving bits around like that is more or less its purpose.
Although I suppose this puts a burden on the URL-creator to pin the content until the URL-clicker doesn't need it anymore, which is not how URLs are supposed to work.
I'm mixed on it. It means it's mostly useless for having your own list of creations because you have to manually store the URLs (PITA) and you need to store them somewhere accessible if you want to repo the UX of the site actually storing them where you can more easily access them from anywhere.
Suppose that you have a hierarchy that was opened. Do you want to store all the hierarchy breadcrumbs in case you hit refresh, and reload all of them?
Furthermore, what about URLs to initiate an action from a particular context? If you refresh what should it show? It should show a dialog / lightbox above the page, showing the form / interface for filling out before taking this action.
Routes are the fundamental building blocks of a webpage, so the hierarchy will be represented by the routes in the url. Proper frontend framework, like Ember.js, built on top of routes and building a UX focused web app is much easier with them.
Your link and the explanation there is not enough. It only shows the current resource you are looking at. As I said above - you should also consider representing the breadcrumbs of how you got there in the hierarchy (you could have taken multiple paths) and also on top of that use URLs to represent actions that can be taken in dialogs !
Here is a question for you: what happens when you have a url for a New Issue in github? By itself. It renders a form for a new issue full-page. But now what if I want it as an overlay over a hierarchical state that U navigated to? I should have a longer URL and it wouldnt be full screen anymore.
Overlay is useful sometimes, however not always the best UX.
The main route of the url (example.com/page/subpage) represents the background page of the overlay.
You have two options. 1. Using further subroutes to represent the stage of the overlay if your framework supports it (Ember does) (example.com/page/subpage/form/step1). 2. Using query parans to store the state of the overlay (without the actual data of course): … subpage?overlay=form&step=1
Also breadcrumbs should always mirror the routes. Frameworks, libraries can automatically do it for you.
If you don’t want to store anything on the server, why serve the static JS and HTML? You can embed them in the URL also.
In that cass the URL just becomes a local file.
You should store the file locally, along with all the data it embeds.
Or you can store the files on a CDN at the edge. The problem with the latter is that people can abuse the storage.
Really, the Web needs to be overhauled to have a topology more like a DHT, where you only have peering agreements with a few peers, and they cache the content temporarily when it is requested a lot. So no centralized servers at all.
A better idea is to use the URI fragment (the string after the # sign). It has the advantage of not having a size limit, and not needing to be sent all to the backend during the request. `window.location.hash` is your friend.
It seems to vary greatly by browser. Based on this SO answer [1] MS edge only supported 2000 characters in 2017, while Chrome currently handles 40 million characters (of compressed, encoded json).
Weirdly, with IEs it was exactly 2083 characters (1) not some base 2 number and MS never increased this number over all these years. This upper limit even included fragments. We tried to do something similar as described in the article and were surprised to learn about IEs limitation. In the end, we stored states on a JS object instead using their hash sum as keys and put that inside the fragment. Then fragment based navigation worked like charm across browsers.
I don't quite understand and it's on me, trust me :)
My reading is you were worried about length of encoding one state, so you moved to encapsulating states in a dictionary with keys of hash of State and objects of State
And this led to a decrease in size of the URL?
My guess at my misunderstanding: you kept the state dictionary server-side or at least some other storage method than the URL, and just used the URL to identify which state to use from the dictionary. I e. The bit you add to the URL is just the hash of the state, or dictionary key
Yes, in the final solution we just stored a hash sum inside the URL fragment (I think it was an MD5 sum) and the actual state inside a JS object in main memory. With a page reload you lost all states which was fine for us but you could use session storage to avoid that.
I don't think React has anything to do with it. Its just structuring URLs in the browser an appropriate way.
You'd need to read state off the url fragment (location.hash). You can feed that into your appropriate state management system, I suppose. Some (like react router) I think support hash routing, but I don't know that they support both has routing and history routing at the same time, so that may not be quite up to task if thats the case.
This is a great point, but also depends on what browsers you want to support. In my experience some browsers a) will truncate the url when you try to copy it from the browser bar (safari), and b) will not support any url including the hash past a certain cut off (like the SO comment states).
Be careful with this. This is only supposed to be used on the client side. Many HTTP server implementations (and infrastructure you run on) will not let you use the URI fragment even if it does hit the wire, or things like caching will break - you almost certainly want use query parameters for anything backend related
A decade-plus ago, this was common in single-page apps. It was the only way to do it before the history API existed, and enough people were doing it that Google was pushing a fragment-url thing [0] that allowed server-side rendering. You'd make such links in a certain way, then the crawler would hope a simple modification of the URL could be rendered by the server and load that.
(Per a comment on that source, it's not been a thing since 2015, since the google crawler can now handle javascript)
If you need to show a flowchart that doesn't fit in a URL, the idea that you should be able to do this with a URL is kinda bonkers. The solution there would be www.example.com/?data=url_for_the_flowchart_definition, because what you're talking about isn't page state, it's a secondary resource.
I just updated knotend so that it now uses the hashmark. I think this is a great idea, and it just loses the ability to get the urls server side, which in my case is totally fine. I also updated the blog post and acknowledged your comment, so thank you!
Mega.co.nz uses that to hide the decryption key for almost decade. You share links but Mega never knows your key because fragment hash is never sent to server or in referrer. Encrypted files are stored on Mega servers but the decryption occurs in JS on the client side, files are decrypted into temporary JS Blob objects in browser then you save them aka "download" locally... a lot of clever tech there. https://help.mega.io/security/data-protection/end-to-end-enc...
Please don't. The web lives and dies by links, the more specific the better, so #to-an-id or #:~:text=to-search are so much more useful than bad state management that probably only works for that one logged-in user anyway. Put that state in the part of the URL you unambiguously control (query params).
As usual when I see this sort of thing, I have questions around the security needs of the application. Storing all of this information on the client means that the client has to be trusted to not mess with all of this state. If it's important, it needs to be stored and checked server-side.
Validating data from the client should always be done on the server in that context, regardless of if state is being saved client side or not. I think this is just an extension of what you said though.
I can think of many apps that don't need server side data storage or security that would benefit from this type of state saving however. For example, this app that I was using today - https://wheelofnames.com/ . Right now to save I need to create an account, presumably to tie my account to the saved state on the server side. That's pretty heavy/intrusive when all I want to do is keep the names list and preferences populated for my next visit.
Sure, there are reasonable use cases. It's worth pointing out where those use cases are not reasonable though, because those devs that don't know better will be inclined to treat obfuscated inputs as if they're trustworthy.
yep. A coworker introduced a CSRF doing this. I had to point out that you can't just take raw input from a URL and throw it back on the page. Even after I pointed it out and gave a proof-of-concept they still didn't get it.
Right, that's usually when I go into story-telling mode, to paint a picture of exactly the sorts of things I'd be doing with the problem as an attacker. Technical vulnerability descriptions provide useful information, but people often need an idea of what it really means, to them.
I tried to do that at first with https://textshader.com because the whole site is client-side and I didn't want to bother with a backend. It didn't really work because the state is unbounded in size, so instead I just chuck it on GitHub gists and then point to the gists. It means I don't have to store any data myself and there's zero recurring costs outside of the domain name... but if the site ever got popular I'd probably have to figure something else out.
I dream of a world where there’s some kind of ipfs like thing for storing state in some sort of distributed commons, all while maintaining user privacy/allowing for state to expire if untouched, and monetized in a balanced way to incentivize hosting without discouraging consumer adoption too much/trending towards gouging.
We’re so close/all the pieces needed seem to exist, but getting something like that off the ground is super difficult.
This is pretty common and has a bunch of advantages, like the fact you can link to and bookmark a particular state.
Also, if you are careful you get undo and redo for free with the browser's back button doing all the work for you.
The disadvantages are that your representation of internal state becomes part of the interface - if you ever change your app you need to deal with versioning the state so your new version can transparently handle the old state format.
If your app has a server component that acts on this state, be super careful about acting on it and treat it as you would any other input under user control.
If you app is completely client side, consider storing the state in the #fragment section of the URL. This never gets sent to the server. An example from my own site [0] - see how the fragment part of the URL changes as you select different topics.
There are also limits on just how much you can cram into a URL but with care you can shove a lot of state.
> If your app has a server component that acts on this state, be super careful about acting on it and treat it as you would any other input under user control.
I would recommend signing it if it's generated by the server component, and checking the signature when the server component is provided this signed state.
For example to do this in Node is quite straightforward.
Key generation:
const crypto = require('crypto');
crypto.generateKeyPair('ed25519', (e, pubkey, privkey) => {
// save pubkey and privkey somewhere
// ...
}
const parts = signeddata.split('.');
const data = Buffer.from(parts[0], 'base64');
const signature = Buffer.from(parts[1], 'base64');
if (crypto.verify(null, data, pubkey, signature)) {
// signature not verified, throw or return
// ...
}
const state = JSON.parse(data);
As the above uses Ed25519 the signatures are quite small too. It needs a bit more error checking, and might need extras like expiry time and such, but should be roughly sufficient.
Good advice. For some apps you might also need to protect against replay attacks to prevent the user from reverting to a previous state in a way that you app should not allow (undo'ing a change of bank balance, etc).
But if your state is so important it is probably better to not uses these techniques at all and just store the state on the server.
This technique is client-side data only, so updating a client-side state would only appear to revert your bank balance on the UI, but wouldn’t trigger any server-side functions to do so.
If you sent the client state to the server for some type of CRUD action or persistence, you’d need to sanitize and validate it first. And at that point like you said, why not just keep the state secure server-side and not trust the client.
Excellent idea for those apps that need that type of protection. For many apps however ability to manually edit the link to get to the different state is actually a free bonus feature. I've built a CRM/Cases management app for the client that stores data filters in the url, and while there's a nice UI to control filters, we observed that lots of users simply go and edit the url directly to quickly get what they need.
I disagree, URLs are supposed to point to a resource - in this case the resource is the client state of the app (aka "deep linking"). It is a useful approach.
There is no such thing as "client state" separate from the notion of resource under REST. The URL is meaningful only to a server. If a server can encode data in the URL so it doesn't need to persist it, that's perfectly fine, so long as the URL conforms to all expectations of resources, ie. caching directives, lifetime, etc.
Modern web apps often use the URL in ways that only matter to the client and can be edited client-side without issuing another request to the server, unless you reload manually, in which case everything is reset except the URL state. This doesn't violate REST principles. Some of that makes sense to put in the URL so users can return to past state, some is very temporary and wouldn't make sense at all to persist across a reload/back.
For example, you may track the scroll position for the purpose of loading more content when the user reaches the bottom of a feed, but you don't want to persist that across reloads. And the user may click on some content and anchor the view to it, which you do persist in the URL for the purpose of history or sharing.
IMHO you should never store the data itself in the url, only the params that you use to fetch the data. So if the page is the list of the latest 10 posts in a category, you'll store the category in the url and the sorting criteria, not the IDs of the posts themselves. On the refresh you always get the fresh data.
If the state cookie is generated by the server, then the best thing to do is to encrypt all application state cookies, and include an expiration time in them.
Also, consider whether you want to allow the user to share the URI or not. If not, then you need to always check that the user is authenticated as whatever you wrote in the state cookie.
The reason I did the compression (using pako as well) was that without it the URL was too long to show a preview on iMessage. I also compress the state manually myself which saved a bunch of bytes too.
I discovered when I hit a similar issue, that just putting a "-" character every 40 or so characters resolved the iMessage parsing issue. There may be a max URL length, but I hit it much sooner without the hyphens.
I love doing this for playgrounds! It's so nice to be able to share a link with the code embedded. Especially if you're giving someone a bug report and they need a minimal reproduction.
I appreciate this feature a lot on sites like https://tio.run and https://mermaid.live/. However the URLs also get very long and I often want to run them through a shortener for e.g. posting to IRC. But there are worse problems to have!
This is quickly becoming a standard in apps and it really shouldnt be handrolled since its such a common requirement and easy to get wrong (between serializing/deserializing/unsetting states). In Svelte it is now as easy as using a store: https://github.com/paoloricciuti/sveltekit-search-params
in general i've been forming a thesis [0] that there is a webapp hierarchy of state that goes something like:
1. component state (ephemeral, lasts component lifecycle)
2. temp app state (in-memory, lasts the session)
3. durable app state (persistent, whether localstorage or indexeddb or whatever)
4. sharable app state (url)
5. individual user data (private)
6. team user data (shared)
7. global data (public)
and CRUD for all these states should be as easy to step down/up as much as possible with minimal api changes as possible (probably a hard boundary between 4/5 for authz). this makes feature development go a lot faster as you lower the cost of figuring out the right level of abstraction for a feature
btw my personal site makes fun use of it -> any 404 page takes the 404'ed slug and offers it as a URL param that fills out the search bar for you to find the link that was broken, see https://www.swyx.io/learn%20in%20public
So you think of a new feature "X". That feature has a bunch of state it needs to manage. Each piece of state has _requirements_ over where it fits within this hierarchy based on how this state relates to the feature being developed. I don't understand where the cost lowering is? I know instantly where the state should live when I design the feature.
requirements evolve over time. and they tend to go up in terms of persistence/auth/sharability needs. it would be nice to make that easy; and conversely when they arent needed anymore it'd be nice to take the state persistence level down with just a few character changes rather than switching state systems entirely
Note on 2: I haven't seen this get a lot of use (or maybe I just haven't noticed it), but the Storage API also provides a `sessionStorage` object [1] with some interesting properties. Not only are values persisted across same-tab refreshes, but if you duplicate the tab, the session values are copied over as well (but they don't remain linked like `localStorage`). An interesting use case for this, IMO, is for tracking dynamic layout values (like if you have resizable panels, scroll areas, etc). If you keep this kind of thing in sessionStorage, you can refresh or duplicate the working tab and that ephemeral state will carry over, but since it's copied (not shared) the tabs won't step on each other.
Unfortunately sessionStorage is not copied over when you ctrl+click a link, and that’s arguably the most common way to “duplicate” a tab. Something about security iirc but i dont grok it.
Still an underappreciated browser feature though, esp for class server-side rendered apps.
I'm actually kinda surprised that's the case for links to the same origin. I suppose you could probably still get this working by using `window.opener` along with `postMessage` to request data from the opener, although you'd have to be very conscientious about checking the origin and verifying that the requested data is actually ok to share.
ive never honestly used sessionStorage as it's not intuitive to me (if i want it clientside i should probably track it serverside, and at that point server is my source of truth)
This makes sense but also seems very annoying to do? E.g let's say you are storing state in the URL, and now the user wants to save it to their profile. You need to upgrade the state from "encoded into the URL" to "stored in my database", which seems annoying.
Could we please stop this "shouldn't be handrolled" or "don't write your own X". It only results in pushing some false idea that you have to be special to do these things, or that innovation is impossible. I can't believe we went from "don't roll your own crypto" to a plain thing like "don't roll your own url state management". What? That is obviously exactly what you should do to get a deeper understanding of the technologies you're working with. I think the only people I hear say this is because they have some agenda in pushing developers to be "tool users" instead of "problem solvers".
It's not that you shouldn't say "there be dragons" or suggest existing solutions, but that's doable without saying you shouldn't do X at all...
Seems like it could be better to store the state in a database and create a unique, short ID that goes in the URL, especially if your doing server rendering (like SSR react or HTMX, or rails/Django)
> I also didn't want to store anything on the backend (at least for the free tier).
You can make the state unique in the DB by a user cookie ID and then upsert the state which will overwrite older state with the latest state. Then you only have 1 record per free user.
Remove record that haven’t been modified in 30 days and you don’t have much overhead. Just off the cuff thinking.
Especially as in the second part of your comment you restrict shareability/usability significantly from what the author wants to achieve (IE I can't as a free user bash out a diagram and share it, then later do another and share with a different person).
In fact the solution is so elegant when you think on it; free account volume is incredibly scalable, zero storage cost, offer immediate value but also a really simple upsell to pro accounts (save to the DB so you don't have to save the URls).
> Especially as in the second part of your comment you restrict shareability/usability significantly
no it doesn't. If my URL is `/thing#a1234` and I share it with you it will load my state for you. If you make a change, it's easy to push a new hash to the url that belongs to you.
> I am genuinely intrigued as to why that's better?
I didn't say it was better. We engineers are supposed to discuss solutions. I ended my last comment with "Just off the cuff thinking".
Isn't this also how old school games (I've seen it as late as the NES I think) did allow you to save progress, you'd get a short string as your "save" and could just enter it to restart the game in the same state later. I love that it offers essentially unlimited "saves"
The downside of a lot of those "password" save systems was that you lost your lives, and sometimes more. The password sometimes just wasn't long enough to store enough information. For example, Driver[0] (at least on the Game Boy) is an icon-based one with eight(?) symbols per field and four fields. That's only 12 bits.
We wrote a library for storing application state in the URL. Its novel feature is using fragment query, which prevents the information from being sent to the server.
The fragment query delimiter is `?` with `&` for between values, just like normal queries. It practice, browsers are not fragment delimiter aware so at the moment I would treat them as mutually exclusive.
There's a larger issue of URL fragment delimiters. For a practical work around on Chrome only, fragment directive/text fragment can be used as a delimiter. (In my opinion, Chrome's delimiter should be re-worked with a new, more comprehensive delimiter.) Example of the workaround:
https://en.wikipedia.org/wiki/URI_fragment#References:~:text...
I recall this technique being used in 1999. It doesn't work well with caching proxies which key off the URL, using a Cookie header is cleaner and simpler.
This shows the performance of a market timing strategy on the S&P500, where you invest $20,000 upfront and $1,000 per month thereafter, while only buying when the market is 25%+ away from the all-time high price (analysis from 1993 to 2023).
These user input variables are all directly encoded in the URL, after the # symbol.
An often unintended side effect of using the URL hash to contain state is that it breaks potentially linking to an anchor element ID somewhere in the page. Obviously this doesn't matter much if the page doesn't use anchor IDs or you don't intend on users doing that.
Interesting. Just to make sure I understand — that url contains all of the IDs and user inputs, and then you parse out of the key/value pairs and feed those inputs into the app?
Before cookies, I remember in middle school (ebay?) using the URLs for sessions.
If they forgot to include the session token on any link on the page, you'd be logged out and have to authenticate again, or you could navigate backwards back to a sessioned address in your history. Once I figured that out, I copy and pasted my session id so if they omitted the session on a link I wouldn't have to log in again.
I always like this when the url is always copyable, but in our app, the state can get quite large. We started with base64 encoded gzip of json.stringified app state but just gets quite long. Arguably it could be reduced a little bit, but would still be hundreds-thousands of chars routinely, so we switched to using a little URL shortening service that we wrote on aws lambda+dynamodb
I built a quick/wip payroll validator [0] that can read state (i.e. payroll details) from the URL.
The purpose in this case was to generate a QR code, the idea is this can be included in physical payslips. Then the numbers behind the calculations (tax deductions) can be validated, broken down, and understood.
I'm developing more tools to these ends, via my overarching project calculang, a language for calculations [1].
I also have a loan/repayment validator [2] but haven't added this QR code feature yet.
Bank letters e.g. "Interest rates are rising and now we want 100e more per month, every month" could use a QR code to an independent validator or to see the workings behind the 100 calculation.
Not using this in the real world now and there are security considerations to keep in mind, but reading state from a URL facilitates the usecase: QR codes that link physical numbers to their calculation and model.
Implementation of payroll calculator is an Observable notebook and thankfully it neatly supports all my strict requirements as demo of this.
This is a handy trick that I've used on multiple occasions. However, it may not work on some corporate networks that filter URLs with more than a certain number of characters (I've encountered limits of 2048 and 4096 characters).
The rationale being that long URLs are often suspicious and could potentially be used for SQL injection or path traversal attacks. Whether or not this is a good heuristic is left as an exercise to the reader.
415 comments
[ 3.4 ms ] story [ 328 ms ] threadAlso, another shout out to mermaidjs and flowchart.fun for also implementing similar url-based sharing.
[0] https://www.knotend.com
Still on FF I opted to use LZ Compression (pretty sure many other sites do this as well) to get the number of characters down
I haven't used https://github.com/ipfs/js-ipfs in this capacity but I'm under the impression that that's moving bits around like that is more or less its purpose.
Although I suppose this puts a burden on the URL-creator to pin the content until the URL-clicker doesn't need it anymore, which is not how URLs are supposed to work.
https://www.typescriptlang.org/play
I'm mixed on it. It means it's mostly useless for having your own list of creations because you have to manually store the URLs (PITA) and you need to store them somewhere accessible if you want to repo the UX of the site actually storing them where you can more easily access them from anywhere.
Suppose that you have a hierarchy that was opened. Do you want to store all the hierarchy breadcrumbs in case you hit refresh, and reload all of them?
Furthermore, what about URLs to initiate an action from a particular context? If you refresh what should it show? It should show a dialog / lightbox above the page, showing the form / interface for filling out before taking this action.
https://guides.emberjs.com/release/routing/
Here is a question for you: what happens when you have a url for a New Issue in github? By itself. It renders a form for a new issue full-page. But now what if I want it as an overlay over a hierarchical state that U navigated to? I should have a longer URL and it wouldnt be full screen anymore.
The main route of the url (example.com/page/subpage) represents the background page of the overlay.
You have two options. 1. Using further subroutes to represent the stage of the overlay if your framework supports it (Ember does) (example.com/page/subpage/form/step1). 2. Using query parans to store the state of the overlay (without the actual data of course): … subpage?overlay=form&step=1
Also breadcrumbs should always mirror the routes. Frameworks, libraries can automatically do it for you.
If you don’t want to store anything on the server, why serve the static JS and HTML? You can embed them in the URL also.
In that cass the URL just becomes a local file.
You should store the file locally, along with all the data it embeds.
Or you can store the files on a CDN at the edge. The problem with the latter is that people can abuse the storage.
Really, the Web needs to be overhauled to have a topology more like a DHT, where you only have peering agreements with a few peers, and they cache the content temporarily when it is requested a lot. So no centralized servers at all.
[1] https://stackoverflow.com/a/44532746
(1) https://support.microsoft.com/en-us/topic/maximum-url-length...
My reading is you were worried about length of encoding one state, so you moved to encapsulating states in a dictionary with keys of hash of State and objects of State
And this led to a decrease in size of the URL?
My guess at my misunderstanding: you kept the state dictionary server-side or at least some other storage method than the URL, and just used the URL to identify which state to use from the dictionary. I e. The bit you add to the URL is just the hash of the state, or dictionary key
IE was all about limitations Nothing surprised me about IE limitations.
https://github.com/Cyphrme/URLFormJS
I use it in a lot of places, like: https://convert.zamicol.com/#?inAlph=Text&in=Hello%20world!&...
https://cyphr.me/ed25519_applet/ed.html#?msg=Hello%20World!&...
https://cyphr.me/coze#?verify&input={%22pay%22:{%22root%22:%...
You'd need to read state off the url fragment (location.hash). You can feed that into your appropriate state management system, I suppose. Some (like react router) I think support hash routing, but I don't know that they support both has routing and history routing at the same time, so that may not be quite up to task if thats the case.
Still though I suppose you could probably gzip it before base64 encoding it for some additional optimization.
Extremely long URLs have other UX issues, e.g. sending them on chat apps and having them eat up multiple scroll pages in one URL, like:
"Check out this event:
http://..... ... ... (500 scroll-screen-lengths of just URL) ... ...
Here's another event on the same day:
http://..... ... ... (500 scroll-screen-lengths of just URL) ... ...
Can you let me know what you think and which one you'd like to go to?"
A decade-plus ago, this was common in single-page apps. It was the only way to do it before the history API existed, and enough people were doing it that Google was pushing a fragment-url thing [0] that allowed server-side rendering. You'd make such links in a certain way, then the crawler would hope a simple modification of the URL could be rendered by the server and load that.
(Per a comment on that source, it's not been a thing since 2015, since the google crawler can now handle javascript)
[0] https://stackoverflow.com/questions/6001433/does-google-igno...
https://web.dev/text-fragments/#textstart
I can think of many apps that don't need server side data storage or security that would benefit from this type of state saving however. For example, this app that I was using today - https://wheelofnames.com/ . Right now to save I need to create an account, presumably to tie my account to the saved state on the server side. That's pretty heavy/intrusive when all I want to do is keep the names list and preferences populated for my next visit.
We’re so close/all the pieces needed seem to exist, but getting something like that off the ground is super difficult.
Also, if you are careful you get undo and redo for free with the browser's back button doing all the work for you.
The disadvantages are that your representation of internal state becomes part of the interface - if you ever change your app you need to deal with versioning the state so your new version can transparently handle the old state format.
If your app has a server component that acts on this state, be super careful about acting on it and treat it as you would any other input under user control.
If you app is completely client side, consider storing the state in the #fragment section of the URL. This never gets sent to the server. An example from my own site [0] - see how the fragment part of the URL changes as you select different topics.
There are also limits on just how much you can cram into a URL but with care you can shove a lot of state.
[0] https://sheep.horse/tagcloud.html#computing
I would recommend signing it if it's generated by the server component, and checking the signature when the server component is provided this signed state.
For example to do this in Node is quite straightforward.
Key generation:
Signing: Verification: As the above uses Ed25519 the signatures are quite small too. It needs a bit more error checking, and might need extras like expiry time and such, but should be roughly sufficient.But if your state is so important it is probably better to not uses these techniques at all and just store the state on the server.
If you sent the client state to the server for some type of CRUD action or persistence, you’d need to sanitize and validate it first. And at that point like you said, why not just keep the state secure server-side and not trust the client.
This is the biggest reason to avoid this. URLs aren't meant to be used this way. I'd only do it if I wanted a quick and dirty solution.
I disagree, URLs are supposed to point to a resource - in this case the resource is the client state of the app (aka "deep linking"). It is a useful approach.
Of course, it can easily be misused.
For example, you may track the scroll position for the purpose of loading more content when the user reaches the bottom of a feed, but you don't want to persist that across reloads. And the user may click on some content and anchor the view to it, which you do persist in the URL for the purpose of history or sharing.
Conversely, you can't link to the newest version.
Also, consider whether you want to allow the user to share the URI or not. If not, then you need to always check that the user is authenticated as whatever you wrote in the state cookie.
Ugh! Please don't do this!
The reason I did the compression (using pako as well) was that without it the URL was too long to show a preview on iMessage. I also compress the state manually myself which saved a bunch of bytes too.
in general i've been forming a thesis [0] that there is a webapp hierarchy of state that goes something like:
1. component state (ephemeral, lasts component lifecycle)
2. temp app state (in-memory, lasts the session)
3. durable app state (persistent, whether localstorage or indexeddb or whatever)
4. sharable app state (url)
5. individual user data (private)
6. team user data (shared)
7. global data (public)
and CRUD for all these states should be as easy to step down/up as much as possible with minimal api changes as possible (probably a hard boundary between 4/5 for authz). this makes feature development go a lot faster as you lower the cost of figuring out the right level of abstraction for a feature
0: relatedly, see The 5 Types of React Application State, another post that did well on HN https://twitter.com/swyx/status/1351028248759726088
btw my personal site makes fun use of it -> any 404 page takes the 404'ed slug and offers it as a URL param that fills out the search bar for you to find the link that was broken, see https://www.swyx.io/learn%20in%20public
1: https://developer.mozilla.org/en-US/docs/Web/API/Window/sess...
Still an underappreciated browser feature though, esp for class server-side rendered apps.
ive never honestly used sessionStorage as it's not intuitive to me (if i want it clientside i should probably track it serverside, and at that point server is my source of truth)
https://tanstack.com/router/v1
It's not that you shouldn't say "there be dragons" or suggest existing solutions, but that's doable without saying you shouldn't do X at all...
> I also didn't want to store anything on the backend (at least for the free tier).
You can make the state unique in the DB by a user cookie ID and then upsert the state which will overwrite older state with the latest state. Then you only have 1 record per free user. Remove record that haven’t been modified in 30 days and you don’t have much overhead. Just off the cuff thinking.
Especially as in the second part of your comment you restrict shareability/usability significantly from what the author wants to achieve (IE I can't as a free user bash out a diagram and share it, then later do another and share with a different person).
In fact the solution is so elegant when you think on it; free account volume is incredibly scalable, zero storage cost, offer immediate value but also a really simple upsell to pro accounts (save to the DB so you don't have to save the URls).
no it doesn't. If my URL is `/thing#a1234` and I share it with you it will load my state for you. If you make a change, it's easy to push a new hash to the url that belongs to you.
> I am genuinely intrigued as to why that's better?
I didn't say it was better. We engineers are supposed to discuss solutions. I ended my last comment with "Just off the cuff thinking".
> Seems like it could be better
Maybe you didn't mean it that way, but it does come across that way. Just a data point for you; I got no beef...
https://gdpr-info.eu/
[0]: https://en.wikipedia.org/wiki/Driver_(video_game)
https://github.com/Cyphrme/URLFormJS
I would love to see it get more use.
Here's a small demo: https://cyphrme.github.io/URLFormJS/#?first_name=Bob&last_na...
See my other comment on this page for some other examples of its use.
There's a larger issue of URL fragment delimiters. For a practical work around on Chrome only, fragment directive/text fragment can be used as a delimiter. (In my opinion, Chrome's delimiter should be re-worked with a new, more comprehensive delimiter.) Example of the workaround: https://en.wikipedia.org/wiki/URI_fragment#References:~:text...
For example: https://themeasureofaplan.com/market-timing/#initialContribu...
This shows the performance of a market timing strategy on the S&P500, where you invest $20,000 upfront and $1,000 per month thereafter, while only buying when the market is 25%+ away from the all-time high price (analysis from 1993 to 2023).
These user input variables are all directly encoded in the URL, after the # symbol.
They both achieve the purposes of allowing custom URLs that bring the user to a specific state of the app.
Edit: better link to explain the concept
Back before websites were "Apps", you could often do exactly this (Including Authentication!) right from the url.
If they forgot to include the session token on any link on the page, you'd be logged out and have to authenticate again, or you could navigate backwards back to a sessioned address in your history. Once I figured that out, I copy and pasted my session id so if they omitted the session on a link I wouldn't have to log in again.
https://www.seaside.st
Wrote this SerDe which can be extended with better algorithms in the future. https://github.com/mermaid-js/mermaid-live-editor/blob/devel...
Eg: https://mermaid.live/edit#pako:eNo9kUGTnCAQhf8K1adNlesoqKiHV...
The purpose in this case was to generate a QR code, the idea is this can be included in physical payslips. Then the numbers behind the calculations (tax deductions) can be validated, broken down, and understood. I'm developing more tools to these ends, via my overarching project calculang, a language for calculations [1].
I also have a loan/repayment validator [2] but haven't added this QR code feature yet.
Bank letters e.g. "Interest rates are rising and now we want 100e more per month, every month" could use a QR code to an independent validator or to see the workings behind the 100 calculation.
Not using this in the real world now and there are security considerations to keep in mind, but reading state from a URL facilitates the usecase: QR codes that link physical numbers to their calculation and model.
Implementation of payroll calculator is an Observable notebook and thankfully it neatly supports all my strict requirements as demo of this.
[0] https://observablehq.com/@declann/payroll-playground-ireland...
+Feature tweet: https://twitter.com/calculang/status/1608183731533107206
[1] https://github.com/calculang/calculang
[2] https://observablehq.com/@declann/loan-validator-dev
The rationale being that long URLs are often suspicious and could potentially be used for SQL injection or path traversal attacks. Whether or not this is a good heuristic is left as an exercise to the reader.