Trurl is the name of a character in the science fiction short story collection "The Cyberiad" by Polish author Stanisław Lem. Trurl is a highly intelligent robot and inventor who, along with his friend Klapaucius, goes on various adventures throughout the universe.
— via CGPT
A great book that prefigured the kinds of interactions people are now having with ChatGPT. For example:
“Have it compose a poem — a poem about a haircut! But lofty, noble, tragic, timeless, full of love, treachery, retribution, quiet heroism in the face of certain doom! Six lines, cleverly rhymed, and every word beginning with the letter S!!”
Stanisław Lem’s "Cyberiad" is a great book that predicted machine learning and various modern technologies, although credit for this particular poem should probably go to Michael Kandel for his brilliant English translation. It's very different in other languages:
Apparently, it requires curl 7.81 to compile, but I'm on ubuntu 20 (7.6x) and wanted to do a quick try, so this patch makes it work (trivially) by removing the couple of new symbols depending on that newer version. Just a quick hack to compile.
Given what it does, and to avoid confusion with perl, "transform url" seems more reasonable. Though I probably would have gone with "turl" so it's easier to pronounce.
Though it hardly gets any use because s/// is so much more flexible, Perl also has a tr/// builtin that replicates the behavior of the command line tool.
Pretty dang cool, but I might have missed a feature: can trurl do multiple manipulations on a single url, as opposed to a single manipulation on multiple urls (which the blog post says is supported)?
For example, you want to apply a sequence of predefined normalization steps, such as removing the user part, converting http to https, etc etc. You put these steps in a file and then invoke trurl with that file and pointed at your url or urls. Very much like what you would do in sed, say. Possible?
Pardone my ignorance, serious question:
Why is this a big deal?
I'm probably underestimting the work needed, but it doesn't look like a hard thing to write.
What am I missing?
> URLs are tricky to parse and there are numerous security problems in software because of this. trurl wants to help soften this problem by taking away the need for script and command line authors everywhere to re-invent the wheel over and over.
Over the years, curl itself has had 9 CVEs relating to handling URLs [0] so this is most definitely not a trivial piece of code to write. The basic case is easy, yes. Getting everything in the spec right and then some is hard.
I had to implement a routine involving URL parsing in a library that is supposed to be behavior-identical across implementations in multiple languages. That was fun.
And, now I'm scared of establishing a TLS connection with untrusted servers after reading CVE-2021-22901 from that page. Remote code execution from an adversarial *server*. I can understand an adversarial client, but that just expanded the things of which I'm wary.
I currently parse out this stuff using a flaky little bit of python I cooked up myself and it gives me no end of grief when scaled. So many awful edge cases.
The author wrote curl, so I know it's going to do what it says it does well.
That's why it's getting love. It's a rock star dev putting out open source code many of us will absolutely be using regularly.
Way classier than yet another "product" built on the OpenAI API that will be gone in a year.
When I was building my CI jobs at $job I needed url manipulation in shell. Had to use python inline, but it was long and ugly... trurl simplified it a bit.
I’m curious about why people dislike this question. This is a specialized library; it should be testing everything including crazy edge cases that most of us haven’t even considered. Also, someone stated that there have been several URL-related security vulnerabilities in curl.
39 comments
[ 5.3 ms ] story [ 82.7 ms ] threadhttps://news.ycombinator.com/item?id=35419573 - 22 points, 2 comments
https://news.ycombinator.com/item?id=35426050 - 5 points, 0 comments
tru-rul? tee-ru-rul?
“Have it compose a poem — a poem about a haircut! But lofty, noble, tragic, timeless, full of love, treachery, retribution, quiet heroism in the face of certain doom! Six lines, cleverly rhymed, and every word beginning with the letter S!!”
“Seduced, shaggy Samson snored.
She scissored short. Sorely shorn,
Soon shackled slave, Samson sighed,
Silently scheming,
Sightlessly seeking
Some savage, spectacular suicide.”
https://mwichary.medium.com/seduced-shaggy-samson-snored-725...
She sat with scissors, ready to snip,
Slicing strands of hair, curl by curl, clip by clip,
Shears shimmering, sharp as swords in sunlight,
Swiftly, skillfully, she shortened the sight,
Saddened by strands strewn on the floor,
She stood, serene, satisfied, the style was no more.
Clearly nowhere as good, and it couldn't get all the words to start with an 'S', even with a few attempts to correct it.
Still, some software should succeed someday, sans skipped S's. Substituting some synonyms, surely.
From https://curl.se/trurl/
Apparently, it requires curl 7.81 to compile, but I'm on ubuntu 20 (7.6x) and wanted to do a quick try, so this patch makes it work (trivially) by removing the couple of new symbols depending on that newer version. Just a quick hack to compile.
> trurl is a tool in a similar spirit of tr but for URLs. Here, tr stands for translate or transpose.
For example, you want to apply a sequence of predefined normalization steps, such as removing the user part, converting http to https, etc etc. You put these steps in a file and then invoke trurl with that file and pointed at your url or urls. Very much like what you would do in sed, say. Possible?
> $ trurl --url https://example.com --set path=/abc/123 --set scheme=ftp --set host=foo.bar
> $ ftp://foo.bar/abc/123
Most of the tools assume opposite default (like xargs for example). Sure, you can specify a different order but why make life harder for no reason?
But you can put options wherever you want, they are identified by the leading '-' char.
> URLs are tricky to parse and there are numerous security problems in software because of this. trurl wants to help soften this problem by taking away the need for script and command line authors everywhere to re-invent the wheel over and over.
[0] https://curl.se/docs/security.html
Or is it just that this is focused on command-line usage
I currently parse out this stuff using a flaky little bit of python I cooked up myself and it gives me no end of grief when scaled. So many awful edge cases.
The author wrote curl, so I know it's going to do what it says it does well.
That's why it's getting love. It's a rock star dev putting out open source code many of us will absolutely be using regularly.
Way classier than yet another "product" built on the OpenAI API that will be gone in a year.