HN Explorer

Ask HN: GRPC vs. 0mq/nanomsg?

15 points by baybal2 ↗ HN
My intention is to have few load balanced NGINX servers with DPDK patches to do further request routing within the infrastructure.

With recent addition of gRPC to nginx, I thought of foregoing having an additional service running on the same machine as nginx to translate public API requests to our internal mq based services.

The bad thing is that gRPC is HTTP/TCP based, while our internal services were all zero copy/IP-less/RDMA based and our engineering has grown used to do synchronous processes across the rack.

Is it a worthy undertaking to write an own gRPC transport that can work on top of some RDMA protocol like SDP?

Or am I better staying with double layer routing: nginx --UWSGI--> script translating http requests to mq formats --SDP--> internal services

8 comments

[ 0.35 ms ] story [ 41.5 ms ] thread
gdamore
Latest NNG (nanomsg-next-gen) has the ability to support HTTP and SP protocol in the same application. You have to write your own code to map the two and add business logic, but it should be pretty straight-forward.

NNG doesn't have an RDMA style transport today, although that can be developed. (Staysail Systems offers commercial support of NNG and nanomsg, including custom development.)

Ultimately, there are lots of ways to solve this, but you need to have a better understanding of your particular needs -- the fact that you're using DPDK and RDMA suggests to me that latency and performance are very important to you, and some more assessment would be required to make a good recommendation.

If you want, you can reach out to me -- I'm the author of NNG and now maintainer of nanomsg. I'm also the CTO for Staysail, so if you need commercial help, I can offer that. (See www.staystail.tech or info@staysail.tech for contact info.)

baybal2
>the fact that you're using DPDK and RDMA suggests to me that latency and performance are very important to you, and some more assessment would be required to make a good recommendation.

A company preparing to handle wire speed services for millions of Internet toasters, fridges and such.

Stuff like high frequency logging, handling millions of active TCP connections on a single endpoint, faster than realtime media recoding/recognition, wire speed object storage, distributed search and indexing.

Effectively, the engineering task is to squeeze Amazon DC into a single rack or two that can be shipped to a colo.

We are a Chinese company, so I doubt we will ever be hiring engineering cadres or external contractors at anywhere near American wages.

gdamore
With the tasks you're trying to achieve, you should have both the budget and the ability to hire competent engineers anywhere in the world. Maybe not to hire hundreds of them, or keep them employed indefinitely, but a few dollars (or yen or whatever) invested wisely at the beginning can spell the difference between success and failure, or between having a model that works efficiently and profitably, and one that has to be re-engineered and costs many many times what would have been invested up front.
baybal2
No doubt that, just here an HDL engineer can be hired cheaper than web dev monkey in US. If so, why not?

In between 3 to 4 k usd of monthly income here allows a person to live very very well, better than with double of that in SV.

ddorian43
Curious, why doesn't that developer work remote for US/EU/UK and cash-in but works locally for 4K?
baybal2
>Curious, why doesn't that developer work remote for US/EU/UK and cash-in but works locally for 4K?

Certainly some do. If US/EU/UK companies were more open to hiring remote freelancers with near nil language/social/work culture skills who can run away with money at any moment, Western labour market would've been overrun long ago.

Please guys, no more further derailing of the original topic.