HN Explorer

Ask HN: A matter of protocols

5 points by maximveksler ↗ HN
Assuming one wished to sent data from his brilliantly developed mobile sdk to his still brilliant but less resource constrained backend, efficiently.

Would he be using capnproto or msgpack or protobuf or flat buffers or thrift or maybe cbor or rust or sbe?

So much awesomeness, so little love for poor old json. oh my :)

Which one will it be? We’re swift / objc on the SDK frontend, go on the backend. Nothing but net in between.

6 comments

[ 0.18 ms ] story [ 54.0 ms ] thread
angersock
Why not just use JSON until you've got a compelling reason not to?
maximveksler
because we will be sending lots of small data bits, in chunks and one of our main features is 0% impact on performance. Now, clearly 0 is impossible but infinity close to 0 is a good goal to have. JSON is textual protocol, which carries the scheme inside every message, not the most effective use of network data traffic.
Blackthorn
Choose what makes sense for the long-term and uses (storage, querying, extensibility, whatnot). I see little reason to use a schemaless thing like JSON anymore. If I was putting together a new project I'd go with capn proto. If I was putting together a new project and wanted a universal RPC stack, I'd go with Thrift (eventually capn proto when it gets more languages w/ RPC support). That's just my opinion though, others will vary. The important thing to remember about your choice is that it's going to be really difficult if not impossible to change later on down the line, so choose wisely.
maximveksler
That’s a good and very viable option to consideration, and is a factor for when you start focusing on the direction but I think it’s unwise to being with narrowing down the options based on such constrains as IMHO this does not lead to the optimal solution but instead to a compromise.

// (btw, totally OT) If you’d like to know where I’ve learned about this reasoning, kindly introduce self to http://hpmor.com (chapter 35-45, can’t remember more specific than that).

vendakka
Transport Layer

I'd use HTTP with SSL on port 443. The ports 443 and 80 are whitelisted by a number of corporate firewalls and the like. I've run into problems in the past caused by blocked ports. In addition there is a large amount of existing software which already understands HTTP for things such as load balancing, caching, etc.

Encoding

I would recommend using JSON to begin with. It is extremely flexible and coupled with the compression abilities you usually find built into HTTP systems it will also be compact. I've used Protobufs (at Google) and Thrift (at my previous job) before, but find that the primary use case ended up being code generation and serialization using their respective description languages. This might seem absurd, but in my experience, since these don't allow you to embed the schema in the message their main advantage was reduced to having a very compact representation with some automatic schema evolution capabilities thrown in.

JSON might lead to some additional initial typing but provides very good performance in most languages. Storage of JSON messages can also be very compact if you store the compressed form. In addition most JSON libraries allow you to be backwards compatible when decoding messages (i.e adding new fields, etc).

When compressing, gzip can be expensive in terms of CPU cycles so consider using Snappy[1] from Google.

I've also heard both good and bad things about Avro but I haven't used it personally.

Most people claim JSON is schema-less but this misses the point a little bit. The JSON schema is the code you write detailing the various structs/objects that the JSON objects map to. Merely because there isn't a specific language used to lay out the schema does not immediately mean you cannot have a JSON schema. There just isn't a standard so use the library that works best for your language.

[1] https://code.google.com/p/snappy/

maximveksler
right, forgot about Avro.

s/“^/or avro”/