Show HN: Jb / json.bash – Command-line tool (and bash library) that creates JSON (github.com)

What happens if the next program in the pipe is not jb? Does jb also exit with a code?

For example `jb | jq`, where jq or a similar program discards the cancel character.

(Away from pc, unable to check right now.)

    $ jb size:number=oops; echo $?
    json.encode_number(): not all inputs are numbers: 'oops'
    json(): Could not encode the value of argument 'size:number=oops' as a 'number' value. Read from inline value.
    ␘
    1

    $ jb size:number=oops | jq
    json.encode_number(): not all inputs are numbers: 'oops'
    json(): Could not encode the value of argument 'size:number=oops' as a 'number' value. Read from inline value.
    parse error: Invalid numeric literal at line 2, column 0

    $ declare -p PIPESTATUS
    declare -a PIPESTATUS=([0]="1" [1]="4")

Thanks!

It's nice to have compact single-file dependencies like this! I like the look of your assertions (checking out, err & status). I definitely found myself writing my own assertions to get understandable errors.

> I like the syntax to send typed values from the terminal:

Incidentally, this syntax shows a notation that is clearly superior to json (at least for non-nested stuff). If all you need is this, you'd be better off by avoiding json altogether.

[Rant: if json is so unergonomic that people keep inventing alternatives like this syntax and stuff like "gron" to de-jsonise their lives, maybe using json was always a bad idea, after all... I guess in a decade everybody will look at json with the same disdain as we do XML today.]

    a=1 b=2 c=3

   {"a"="1", "b"="2", "c"="3"}

    %{n:1}

Glad to hear, this was something I wanted to make reliable, ergonomic and intuitive. I figured a lot of languages use `: type` to declare types.

The same using jo would be like this, which I find harder to type and remember:

  jo -- -s id=42 -n size=42 -s surname=null data=null
  {"id":"42","size":42,"surname":"","data":null}

Notice that surname comes out as the empty string though, I think this must be a bug in jo!

Kubernetes already got you covered: What if everything was a YAML?

I did find it quite satisfying to coerce bash into doing this while maintaining decent performance. I definitely came to appreciate some aspects of bash more from this, but it's so easy to shoot yourself in the foot!

If I started from scratch now I'd use a compiled language that could produce a single static binary and start with really low latency. I'm pretty sure jo must not be tuned for startup time, if they optimised that they must be able get it way faster than bash can start and parse json.bash. I was pretty surprised that bash can startup faster!

The codebase is basically at the limit of what I'd want to do with bash, but there are features I could add if it was in a proper programming language. e.g. validating :int number types, pretty-printing output, not needing the :raw type to stream JSON input.

Thanks for the heads up on Shellcheck, I'd be happy to take a PR if you'd like to.

This is mentioned.

Thanks for giving it a try and your feedback. I agree, the array splitting is a bit fiddly. It is actually possible to pass JSON directly, you use the :json type on the argument:

    $ jb dependencies:json='["Bash","Grep"]'
    {"dependencies":["Bash","Grep"]}

    $ jb foo=bar bar:json='"this is a well formed string"'
    {"foo":"bar","bar":"this is a well formed string"}

And then you can indeed use command substitution to nest calls:

    $ jb foo:json=$(jb bar=baz)
    {"foo":{"bar":"baz"}}

It works even better to use process substitution, this way the shell gives jb a file path to a file to read, and so you don't need to quote the $() to avoid whitespace breaking things:

    $ jb foo:json@<(jb msg=$'no need\nto quote this!')        
    {"foo":{"msg":"no need\nto quote this!"}}

Another option is to use jb-array to generate arrays. (jb-array is best for tuple-like arrays with varying types):

    $ jb dependencies:json@<(jb-array Bash Grep)
    {"dependencies":["Bash","Grep"]}

And if you use it from bash as a function, you can put values into a bash array and reference it:

    $ source json.bash
    $ dependencies=(Bash Grep)
    $ json @dependencies:[]   
    {"dependencies":["Bash","Grep"]}

Powershell has the upper hand here!

Still, bash can try to keep up using json.bash. :)

    $ source json.bash
    $ declare -A greeting=([Hello]=World)
    $ json ...@greeting:{}
    {"Hello":"World"}

... is splatting the greeting associative array entries into the object created by the json call.

Without the ... the greeting would be a nested object. Probably more clear with multiple entries:

    $ declare -A greeting=([Hello]=World [How]="are you?")
    $ json @greeting:{}   
    {"greeting":{"Hello":"World","How":"are you?"}}

Vs:

    $ json ...@greeting:{}                                
    {"Hello":"World","How":"are you?"}

    $h=@{x=1; y=2}; $h + @{z=3} | ConvertTo-Json

    {        
      "y": 2,
      "z": 3,
      "x": 1 
    }

Not only its built in, but syntax is on another level, i.e. you don't need to learn special syntax if you know PowerShell. This thing alone makes pwsh worth it instead of using number of other tools.

    @{ Hello = 'world'; array = 1..10; object = @{ date = Get-Date } } | ConvertTo-Json
 
    {
      "array": [
        1,
        2,
        3,
        4,
        5,
        6,
        7,
        8,
        9,
        10
      ],
      "object": {
        "date": "2024-07-03T21:07:21.6562053+02:00"
      },
      "Hello": "world"
    }

    $ jb Hello=world array:number[]@<(seq 10) object:json@<(date=$(date -Iseconds) jb @date)
    {"Hello":"world","array":[1,2,3,4,5,6,7,8,9,10],"object":{"date":"2024-07-03T19:26:36+00:00"}}

    jb Hello=world array:number[]@<(seq 10) object:{}=date=$(date -Iseconds)
    {"Hello":"world","array":[1,2,3,4,5,6,7,8,9,10],"object":{"date":"2024-07-03T19:30:26+00:00"}}

If it helps, there's a little example of using the bash API with bash variables/arrays, should give you an idea of how it could be to use: https://github.com/h4l/json.bash/blob/main/examples/notify.s...

This example uses the pattern of setting an out=varname when calling a json function, the encoded JSON goes into $varname variable. This pattern avoids the overhead of forking processes (e.g. subshells) when generating JSON.

Otherwise you can use the more normal approach of jb writing to stdout, and capturing the output stream.

How did you guess my password?!?!

This is just the kind of use case I had in mind. Something I've considered is publishing a mini version with only the json.encode_string function, as that's enough to create an array of JSON-encoded strings and use a hard-coded template with printf to insert the JSON string values.

That would be a fraction of the overall json.bash file size.

RIP bash.org!

I wouldn’t do that to my users; the happy path here is nice, but the unhappy path seems very likely to end in garbled bash errors that are impossible to track down.

As a user, I’m fine with embedding a reasonably small VM to handle the configs; disk space is cheap. Better yet would be a compiled binary that handles it, but that feels like asking a lot of maintainers.

There’s a lot of surface area for someone to mis-quote stuff in their environment and generate unintelligible bash errors.

Or that may be just me; I hate bash in general, so maybe it’s just that bleeding over.

Two main situations I think. The first is just interactive use in any shell to encode ad-hoc JSON. If you have a next-gen shell which can handle structured data directly, then you probably don't need it.

Second is situations where you'd rather not add an additional dependency, but bash is pretty much a given. For example, CI environments, scripts in dev environments, container entrypoints. Or things that area already written in bash.

I don't advocate writing massive programs in bash, for sure it's better to turn to a proper language before things get hairy. But bash is just really ubiquitous, and most people who do any UNIX work will be able to deal with a bit of shell script.

Thank you, that's high praise! I learnt a lot about bash writing this, but I've also not looked at the code in a few months, and it's already starting to look quite intimidating!

I definitely like the idea of a goland/rust implementation, there are certainly things I could improve.

So the argument syntax escapes by repeating a character rather than backslash. I chose this because with backslashes escapes it would be unclear whether a backslash was in the shell syntax or the jb syntax, and users may end up needing to double escape backslashes, which is no fun! Whereas a shell will always ignore two copies of a character like =:@.

The downside of double-escaping is that the syntax can be ambiguous, so sometimes you need to include the middle type marker to disambiguate the key from the value. But the type can be empty, so just : works:

    $ jb ===msg==:==hi=
    {"=msg=":"=hi="}

In the key part, the first = begins the key, the == following are an escaped =. The first = following the : marks the value, and everything after is not parsed, so =hi= is literal.

When you have reserved characters in keys/values (especially if they're dynamic), it's easiest to store the values in variables and reference them with @var syntax:

    $ k='=msg=' v='=hi=' jb @k@v
    {"=msg=":"=hi="}

There is, the earliest version I've tested with is 4.4.19, but ideally a 5.x version. 3 certainly won't work I'm afraid. If you use homebrew on Mac it's a good way to get the latest bash.

Awesome, thanks for packaging it!