Delighted to see work continue on jq, which I use for altogether way too many things.
I ask this as a fan of both the project and the new pick() feature: are there any other functions in jq whose interpretation of their arguments is "non-regular" in the sense the PR author means [here][1]?
More concretely, the expression `.a, .b.c, .x` means something very different outside a pick call. It represents a uncollected stream of more than one value, rather than a reduction of an object into a subset of itself.
The syntax inside a pick call is (I believe) impossible to implement in jq; it has to be a builtin. There's no way to get "regular jq" to extract the path expressions as strings with which to set an object's keys.
Ultimately I do think like the addition, but I recognize the tradeoff of "pure" consistency for syntactical brevity. I'd be curious to hear from the maintainers A) whether I'm off-base about the pathexp syntax limitation and B) if there's some philosophy or design principle guiding why and when they might decide to add specialized interpretations of pathexps.
When the jq manual say "path expression" like the argument to pick you can think of it as just normal jq expression but instead outputting values when path/1 is used it will instead output the paths to those values. The limitation is that you only can index into the input (actually including the null value) and don't construct new literals etc.
btw an alternative to pick that might be handy at times is the destruction and shorter object construction syntax:
# use {a:{b:1},c:2} as input
# . as {a:{$b}} will bind $b to 1
# {$b} is shorthand for {b:$b}
# {a} is shorthand for {a:.a}
$ jq -n '{a:{b:1},c:2} | (. as {a:{$b}} | {$b}), {a}'
{
"b": 1
}
{
"a": {
"b": 1
}
}
I could see this being useful when you coded a utility, which runs off a huge existing JSON as its configuration. You want to go straight for one or two values, not this huge data structure.
In that case it would be very similar but pick is more general then the shorthand object literal syntax, it can be used with nested index lika .a.b.c or any expression like pick(if .a then .b else .c end) but both has it uses i think.
Great to see the NUL separator work I had a hand in has reached a release. Wish it got to use -0 for easy typing, but with tab completion it shouldn't be too bad.
Is it not an indictment of bash that, in order to make shells useful to modern data formats, we have to create tools with their own scripting languages and pipes? Since bash can't operate on structured data like jq does, jq has to have pipes inside jq expressions.
That's how I'd like it to be. These are domain specific languages, so their own scripting is where I'd see them belong. Command line should stay true to its name: place where you enter your commands (and of course, other things that you can't delegate elsewhere, like variables and control structures).
yeah but JSON has become to de-facto universal format. At the very least, it would be cool if you could convert that into a common runtime format and pipe structured data from one command to another so you don't have to keep parsing things at every stage, or abandoning the shell entirely and working only in jq
ksh93 has "compound variables" that let you have arbitrarily deep trees of data. The syntax for that is ugly. It's hard to use. And I've never had luck with not causing ksh93 to crash when trying to use compound variables.
I think it'd be very nice to have a bash-like shell with a) jq-like JSON values in the shell (really, parsed, jv-like[0]), b) syntax for using jq programs in ${...}, something like ${var@jq:.jq.program.here}.
As for jq having pipes, they're a rather different beast, but I take your point.
[0] `jv` is the internal representation of JSON values in jq.
Shell scripting is beyond salvation. Inconsistent, illogical crap. It's ridiculous that Bash doesn't have first-class JSON support in 2023, but even if it did, that still wouldn't make me want to write Shell.
I could see a shell that doesn't suck for scripting that has first-class JSON support. Something like PowerShell, but not PowerShell, and close enough to bash that one could quickly get used to it.
> 1. Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features".
> 2. Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input.
> 3. Design and build software, even operating systems, to be tried early, ideally within weeks. Don't hesitate to throw away the clumsy parts and rebuild them.
> 4. Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you've finished using them.
Hey, this is a release candidate. It got made mainly to test the new GitHub Actions release workflow, though it's just as well because we're very close to doing a 1.7. Probably a week away.
27 comments
[ 2.7 ms ] story [ 25.9 ms ] threadI’m excited about pick()! Could have used this so many times:
> Adds new builtin pick(stream) to emit a projection of the input object or array. @pkoppstein #2656
I ask this as a fan of both the project and the new pick() feature: are there any other functions in jq whose interpretation of their arguments is "non-regular" in the sense the PR author means [here][1]?
More concretely, the expression `.a, .b.c, .x` means something very different outside a pick call. It represents a uncollected stream of more than one value, rather than a reduction of an object into a subset of itself.
The syntax inside a pick call is (I believe) impossible to implement in jq; it has to be a builtin. There's no way to get "regular jq" to extract the path expressions as strings with which to set an object's keys.
Ultimately I do think like the addition, but I recognize the tradeoff of "pure" consistency for syntactical brevity. I'd be curious to hear from the maintainers A) whether I'm off-base about the pathexp syntax limitation and B) if there's some philosophy or design principle guiding why and when they might decide to add specialized interpretations of pathexps.
[1]: https://github.com/jqlang/jq/pull/2656#issuecomment-16228220...
When the jq manual say "path expression" like the argument to pick you can think of it as just normal jq expression but instead outputting values when path/1 is used it will instead output the paths to those values. The limitation is that you only can index into the input (actually including the null value) and don't construct new literals etc.
btw an alternative to pick that might be handy at times is the destruction and shorter object construction syntax:
I think it'd be very nice to have a bash-like shell with a) jq-like JSON values in the shell (really, parsed, jv-like[0]), b) syntax for using jq programs in ${...}, something like ${var@jq:.jq.program.here}.
As for jq having pipes, they're a rather different beast, but I take your point.
[0] `jv` is the internal representation of JSON values in jq.
Shell scripting is beyond salvation. Inconsistent, illogical crap. It's ridiculous that Bash doesn't have first-class JSON support in 2023, but even if it did, that still wouldn't make me want to write Shell.
> 2. Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input.
> 3. Design and build software, even operating systems, to be tried early, ideally within weeks. Don't hesitate to throw away the clumsy parts and rebuild them.
> 4. Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you've finished using them.
https://en.wikipedia.org/wiki/Unix_philosophy
(No, it is not an indictment of bash. This is how tools are supposed to work together.)