HN Explorer

Ask HN: What work is being done on non-proprietary cloud computing protocols?

16 points by wizofaus ↗ HN
It's occurred to me a few times that I don't mind if our architecture uses cloud-based serverless tech, lambdas, message queues, storage etc. - but we should be talking to such endpoints using non-proprietary protocols that would allow us to switch to another cloud provider if necessary (possibly even our own non-cloud-based provider, esp. while developing/debugging). Is any work being done on this front? Is it likely to be successful? What are the major hurdles?

12 comments

[ 0.24 ms ] story [ 43.1 ms ] thread
nicolaslem
You are basically describing the vision behind Open Stack. It used to have a fair amount of traction in the 2010's but I rarely hear about it anymore.
mikalstill
That's not really true. OpenStack never attempted to create generic APIs other implementations could use, it attempted to build an open cloud implementation you could use.

Ultimately I think OpenStack was too complicated, in much the same way that Kubernetes seems to be embracing. That said it is still widely deployed, mostly by telcos, large enterprises, and governments.

[deleted]
(comment deleted)
wizofaus
From what I can read there are quite a few open message queue protocols already in existence (AMQP, MQTT etc.) and AWS at least claim to support them but I'm not sure how feasible it is to rely on that entirely, has anyone tried? Likewise there are plenty of existing open filesystem storage protocols (sftp etc.) - would it be practical to use such a protocol exclusively to access S3 storage (again, aws claim to support it)? A protocol to upload runnable-code to a cloud-compute instance or schedule it to be run based on various triggers might be asking too much and necessarily require vendor specific messaging though.
orf
If you’re honestly asking if accessing s3 exclusively through SFTP is a good idea (regardless of if it somehow makes you magically cloud agnostic), I really really don’t know what to say. Other than “no”.

I’m reminded of a quote by Babbage

> I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question

wizofaus
Can you give an example of something that wouldn't work well if you did so? I've written code to talk to both sftp and s3 servers, the basics aren't that different...
orf
Listing prefixes, signed URLs, tags/metadata, throughput and of course latency.

The S3 API is pretty standard a number of open source projects and cloud providers use it.

But the whole question smells funny. It’s much better in almost all regards to abstract over a storage interface in code rather than use something like SFTP.

wizofaus
Granted on signed URLs - you'd need a vendor-specific extension to generate one I suppose. Not sure why throughput/latency should be worse with sftp. And yes, I've done the abstraction thing in the past too - unfortunately the code base I'm currently working with doesn't, meaning you can't run it fully without an actual connected aws environment. I'm not suggesting sftp is the right choice, just wondering how feasible it is to make use of existing open protocols to access cloud-based services.
orf
You need to understand that the S3 API is an open protocol. It’s a simple, well documented HTTP API of which there are many open source implementations. Minio is one of the well known ones.

It’s not some closed, custom binary protocol.

wizofaus
Thanks, I actually wasn't aware of that. Do other cloud providers support it? Will look into minio as an option for running s3 locally.
wizofaus
As per info from an HN commentator - S3 is now such a protocol that can be used for accessing cloud-based storage at least. I had assumed it was proprietary (and certainly never used it for anything other than AWS S3).