This V2 is still pushing forward the retarded behavior from v1 when it comes to handling nil for maps, slices and pointers. I am so sick and tired of this crap. I had to fork the v1 to make it behave properly and they still manage to fuck up completely new version just as well(by pushing omitempty and ignoring omitnil behavior as a standalone case) which means I will be stuck with the snale-pace slow v1 for ever.
It is good to see some partial solutions to this issue. It plagues most languages and introduces a nice little ambiguity that is just trouble waiting to happen.
Ironically, JavaScript with its hilarious `null` and `undefined` does not have this problem.
Most JSON parsers and emitters in most languages should use a special value for "JSON null".
Could somebody give a high level overview of this for me, as not a godev? It looks like Go JSON lib has support to encode native go structures in JSON, which is cool, but maybe it was bad, which is not as cool. Do I have that right?
I still don't get how a common thing like JSON is not solved in go.
How convoluted it is to just get a payload from an api call compared to all languages is baffling
It's not like this is new. Look at Java and .NET collection APIs, for example - both languages have the OG 1.0 versions, and then the more modern ones. In a similar vein, .NET has four different ways to deal with XML, of which three (XmlDocument, XPathDocument, and XDocument) are basically redundant representations of XML trees, each one doing things differently based on lessons learned.
I will say this and I feel it's true. Dealing with JSON in Go is a pain. You should be able to write json and not have to care about the marshalling and the unmarshalling. It's the way that serde rust behaves and more or less every other language I've had to deal with and it makes managing this behavior when there's multiple writers complicated.
Are you referring to the json macro that allows variable interpolation? Doing that will void type safety. Might be useful in dynamic languages like Python but I wouldn’t want to trade type safety for some syntactic sugar in Go
I just ran our full suite of a few thousand unit tests with GOEXPERIMENT=jsonv2 and they all passed. (well, one test failed because an error message was changed, but that's on us)
I'm especially a fan of breaking out the syntactic part into into its own jsontext package. It makes a ton of sense, and I could see us implementing a couple parsers on top of that to get better performance where it really matters.
I wish they would take this chance to ditch omitempty in favor of just the newly-added omitzero (which is customizable with IsZero()), to which we'll be switching all our code over Real Soon Now. The two tags are so similar that it takes effort to decide between them.
I'm coming in a little hot and contrarian. I've been working with the Go JSON library for well over a decade at this point, since before Go 1.0, and I think v1 is basically fine.
I have two complaints. Its decoder is a little slow, PHP's decoder blows it out of the water. I also wish there was an easy "catch all" map you could add to a struct for items you didn't define but were passed.
None of the other things it "solves" have ever been a problem for me - and the "solution" here is a drastically more complicated API.
I frankly feel like doing a v2 is silly. Most of the things people want could be resolved with struct tags varying the behavior of the existing system while maintaining backwards compatibility.
My thoughts are basically as follows
The struct/slice merge issue? I don't think you should be decoding into a dirty struct or slice to begin with. Just declare it unsupported, undefined behavior and move on.
Durations as strings? Why? That's just gross.
Case sensitivity by default? Meh. Just add a case sensitivity struct tag. Easy to fix in v1
Partial decoding? This seems so niche it should just be a third party libraries job.
Basically everything could've been done in a backwards compatible way. I feel like Rob Pike would not be a fan of this at all, and it feels very un-Go.
Without any major changes, struct tags cannot solve the majority of the problems, because they do not propagate. The v1 MarshalJSON/UnmarshalJSON interfaces are too simplistic. At the very least, an alternative pair of interfaces should exist, which take options/flags/tags/whatever in the parameters. However, I agree this doesn't need to be in an entirely new package. It's not even hard to favor one interface over another:
switch ifc := val.(type) {
case MarshalJSONV2: // new interface, always used if present
case MarshalJSON: // old interface, fallback
default: // type isn't self-marshaling
}
The jsontext package is what's really revolutionary and needed here. The poor performance of the existing API is due primarily to the lack of a proper streaming API as the foundation. Using this as the basis for the new interfaces makes sense, and I agree that once this exists, the need for an entirely separate v2 package largely vanishes.
Please do run this on your own workloads! It's fairly easy to set up and run. I tried it a few weeks ago against a large test suite and saw huge perf benefits, but also found a memory allocation regression. In order for this v2 to be a polished release in 1.26, it needs a bit more testing.
23 comments
[ 164 ms ] story [ 131 ms ] threadhttps://github.com/centralci/go-benchmarks/tree/b647c45272c7...
It is good to see some partial solutions to this issue. It plagues most languages and introduces a nice little ambiguity that is just trouble waiting to happen.
Ironically, JavaScript with its hilarious `null` and `undefined` does not have this problem.
Most JSON parsers and emitters in most languages should use a special value for "JSON null".
No, it's an exception. It was badly designed from the start - it's not just that people's json needs (which hardly changed) outgrew it.
How did that make it into the v1 design?
Given that it is not even yet solved in its namesake language, Javascript, that's not saying much.
And then people complain that Rust doesn't have a batteries-included stdlib. It is done to avoid cases like this.
It's still better than the mess that is Node.js.
That does look a lot cleaner. I was just grumbling about this in golang yesterday (yaml, not json, but effectively the same problem).
I just ran our full suite of a few thousand unit tests with GOEXPERIMENT=jsonv2 and they all passed. (well, one test failed because an error message was changed, but that's on us)
I'm especially a fan of breaking out the syntactic part into into its own jsontext package. It makes a ton of sense, and I could see us implementing a couple parsers on top of that to get better performance where it really matters.
I wish they would take this chance to ditch omitempty in favor of just the newly-added omitzero (which is customizable with IsZero()), to which we'll be switching all our code over Real Soon Now. The two tags are so similar that it takes effort to decide between them.
I have two complaints. Its decoder is a little slow, PHP's decoder blows it out of the water. I also wish there was an easy "catch all" map you could add to a struct for items you didn't define but were passed.
None of the other things it "solves" have ever been a problem for me - and the "solution" here is a drastically more complicated API.
I frankly feel like doing a v2 is silly. Most of the things people want could be resolved with struct tags varying the behavior of the existing system while maintaining backwards compatibility.
My thoughts are basically as follows
The struct/slice merge issue? I don't think you should be decoding into a dirty struct or slice to begin with. Just declare it unsupported, undefined behavior and move on.
Durations as strings? Why? That's just gross.
Case sensitivity by default? Meh. Just add a case sensitivity struct tag. Easy to fix in v1
Partial decoding? This seems so niche it should just be a third party libraries job.
Basically everything could've been done in a backwards compatible way. I feel like Rob Pike would not be a fan of this at all, and it feels very un-Go.
It goes against Go's whole worse is better angle.