The ground-up approach includes the entire stack.
Hardware to software. The company has designed
its own servers down to the bend of the metal
and the motherboards used.
Why, oh why, did these guys think they needed to design their own motherboards? I couldn't think of a more commodity product. You want to do that eventually, then sure, do it. But not as a startup. Focus first on what will differentiate you from everyone else. Hint: IT'S NOT THE FUCKING MOTHERBOARDS!!!
The founder spent 22 years working for St. Steven, and yet he didn't learn one of the most important lessons:
“People think focus means saying yes to the thing you've got to focus on. But that's not what it means at all. It means saying no to the hundred other good ideas that there are. You have to pick carefully. I'm actually as proud of the things we haven't done as the things I have done. Innovation is saying no to 1,000 things.”
What if they wanted substantially higher IO on their hardware, for example? I'm not saying it is a good idea, I'm saying there's a possibility it could have been a good idea.
It says they designed the servers rather than the motherboards, so I'm guessing it just means they shopped around for different off-the-shelf motherboards rather than, say, hiring hardware engineers to lay out gates and chips.
This seems to be a recurring theme: Unix (and Windows) use files. Keeping track of files is hard. If we could just use enough Unix (internet, mobile, the cloud, oh my), we could make files go away.
And then pictures, video, music, etc come along and wreck everything.
Maybe there are better machines to rage against than a hippie OS.
Also, I wonder if they've done the math on how much data you can actually cram into the airwaves. You can't instamagically move more data through a finite spectrum at rush hour on a Wednesday.
They're essentially making end user devices into thin clients. Which I'm actually a fan of, if only I get to run the backend on my own server hardware.
We badly need standardized home servers that can run 24/7 and managing storage and processing tasks for our mobile device. It would solve sooo much of the problems derived from unreliability of mobile devices.
I personally love this idea. Its a mini cloud running out of your home and it puts you in complete control of your data. I would even take it a step further and say that as long as you have a home server running 24/7 why not run a mail server as well? It would resemble something like an apple tv. Small, quiet, power efficient. You plug it in, run through a simple setup wizard, pay ten bucks for a domain name, and off you go.
A mix between a basic game console (see Steam machines), router (see OnHub) and a plug computer could work, maybe.
Something that the regular users have an incentive to buy, because the primary advertised features are something they need or really want, and that then also offers a wide variety of additional services, such as mail, VoIP, coalescing proxy for your mobile devices to save their battery, storage, etc...
I think more than 24/7, we need systems that can still work offline, and "do the right thing" transparently when reconnected.
Copying files into your personal cloud while offline should still work. Deleting/renaming/moving files from the cached directory tree should still work offline. Opening cached files should still work offline.
>home servers that can run 24/7 and managing storage and processing tasks for our mobile device
They better be low power - I pay ~34c/kWH for at my house. I recently looked into building a NAS for storage + OwnCloud and I can buy an awful lot of BackBlaze + AWS for the same price.
A little bit of feedback: Quite large parts of the UK still have patchy 2G reception so this would not be practical for here. People on HN have been saying don't worry the network will catch up for years, but where I live the network coverage and speed is crap and hasn't changed at all in the last 5 years in the 80 miles between here and the nearest major city.
I really, really wish startups stop assuming everyone has unlimited data plan and broadband Internet available 24/7. I'm not even begging for the common engineering sense anymore (don't route around half of the globe the data that has to travel only 5 meters in physical space; some data does not belong on the Internet, etc.), or even common decency. For practical reasons and the love of God, stop with the requirement of being constantly on-line to use a goddamn app.
I really need to register bandwidthprersevationsociety or something a bit shorter. I was initially going to say that developing nations are going to be left behind but that's not right, a lot of them have very good high-speed net access. Australia on the other hand - a lot of places are going to have a maximum of 25 mbit / 1 mbit for a long time due to the geniuses running our government.
I'm on what is effectively 8/1 and I'm starting to struggle with the size of Windows Updates (8->8.1 failed multiple times resulting in a repeated 2GB download), XBox One games (20-40GB), streaming and RDP/Citrix/VPN for working from home.
> I'm not even begging for the common engineering sense anymore
OneDrive! It doesn't even implement any kind of LAN Sync - you'd think MS would have bolted on some of the newer SMB stuff like HomeGroup (which is discovery+SMB over IPv6 AFAICT)
>For practical reasons and the love of God, stop with the requirement of being constantly on-line to use a goddamn app.
Ubisoft managed to lock the whole of Australia out of one if its recent betas with this mentality.
Well we certainly have slow internet here. It'd be a good joke/political site to do a "slow internet movement" in the style of the "slow food movement"
The Verge, TechCrunch and their website, and I still have no idea what their product is. They have an impressive team, that has been put across. Other than that, just buzz words.
If, but it is a big if, I have understand correctly it is a simple cloud service but it doesn't save your file in your hard drive, but directly and only on the cloud...
"Upthere says it plans to offer an API in hopes that developers will make it the default storage solution for their own apps"
So... how is that in any way different to using the AWS API to upload stuff to S3? Being a "storage solution provider" (and reselling other companies' cloud offerings at that) has some razor thin margins and a whole lot of competition. I don't understand what makes these guys different.
They talk about running their own hardware so assume something like open compute or backblaze. But without the size or reliability of S3. Can't imagine they'll be able to do it at lower cost than Amazon either. Like you I have no idea why not use S3.
Like you, I can't see how they'll compete on price if they own their own hardware. Microsoft, Amazon, Google, Facebook have all been working on this stuff for years. Even adopting Opencompute or similar, they will have comparatively no buying power, no users and no data.
Agreed. I don't understand this at all. The whole point of syncing was that it got around all the inherent limitations of requiring an always-on network connection in a client-server architecture. And now somehow going back to requiring a network connection to save data is an improvement?
They probably only want to provide B2B services, which prove valuable to offices full of workstations with deep pockets. Not consumer grade, residential drones like us.
They probably only want to provide B2B services, which prove valuable to offices full of workstations and deep pockets. Not consumer grade, residential drones like us.
Syncing everything to the cloud may be great if you have an unlimited plan, but as long as each GB costs around $10USD these kind of initiatives will have a very limited impact.
While the solution going forward is most likely to go back to dumb terminals (thin clients, if you prefer) I think just putting it in "the cloud" as it stands today is not the solution. Granted, it's not very clear what they are offering, but it does appear to be a gated service (think Dropbox) rather than an open technology (think HTTP). It just doesn't seem to solve problems any better than what's already out there, but I'd be happy to be proven wrong.
Personally, what I'd like to see is mainstream software bundled with https://ipfs.io/ technology. If content could be served via a DHT plus all the extra things IPFS offers, we'll be a lot closer to a world where content doesn't get lost. At that point I'd gladly pay a company to store my personal files (by simply giving them a list of hashes which resolve to my computer so they can download them), and trust that it's safe because they were encrypted on my end and the company is just ensuring that the content can be redownloaded should my computer explode.
On a sidenote, IPFS would also solve a lot of problems with public data, where everything can be reliably synced (and even updated with IPNS) for people with no persistent access to the Internet (and people on space stations or the Moon or Mars, should we ever put bases on them).
Pretty much, but large multinationals are the only ones that own the mainframes, and it's only cost-effective to have a few very large providers. That's the way it's all moving.
> "NO ONE DID WHAT WE ENVISIONED. SO WE STARTED FROM SCRATCH."
Yet you were the "senior vice president of software engineering for Apple." You were in a better position than almost anyone else to "do what you envisioned." What stopped you? If you could not do it at Apple, why will you be able to do it at a startup?
The ability to start a startup is a new bargaining chip for high value corporate employees. Why build an innovative feature at your company -- and give them all the money -- when you could just leave and do it yourself, and keep all the money?
There is a moral hazard for corporate employees that poses a large risk to corporate innovation. If an employee at a company stumbles upon an innovative opportunity, it can be economically sensible to for the employee to avoid implementing it, in preparation for a future startup that will attract easy VC money.
The best solution to this is probably just for big companies to grant more and more stock to employees to keep incentives aligned.
Well, because starting a company is a huge risk with a very low percentage of success, even with an awesome idea. Some ideas will require corporate backing to be successful, either because it is more of a feature than a product or because you need to leverage the infrastructure of the corporation.
Way less risk when you are former VP of engineering at apple. That guarantees at least enough investment to hire a team the size of what you had at Apple. Presumably as former VP you also have the experience to increase chance of success.
So no, not a huge risk, and higher percentage chance of success than anyone else starting the same company.
Some of the worst startup founders I have seen are people who come from being execs or management at very large companies, because they have no frame of reference for how things should be done when you have 5 people.
So I don't think you can generalize on that point. And as far as this goes:
> That guarantees at least enough investment to hire a team the size of what you had at Apple.
There's also no way this is a guarantee. Sure, it'll be easier to get money than an 18 year-old walking in off the street will, but again, you can't compare the steady job, steady paycheck, "let's build a feature internally" with "let's go start a startup".
I will definitely agree that having the experience of a vp of engineering is better than not having it, but that's a far cry from saying that quitting your highly paid job as a VP of engineering for the biggest company in the world to start one of many cloud storage companies is "not a huge risk".
Syncing will eventually be a thing of the past for most people, but I imagine what will make syncing obsolete is not cloud storage alone, but rather cloud storage in combination with the emergence of a new class of portable devices that is compact and efficient enough to carry around for personal communications, powerful enough to get real work done quickly, and flexible enough to support large displays and VR for media consumption and gaming: a single device that literally does it all.
Cloud storage will still be important in this new single-device computing model, but only as a means for archival and backup. Sync won't come into play anymore because you only have the cloud and a single device to manage, and changes will tend to only propagate in a single direction from the device to the cloud.
What I'm imagining is a "wearable server", an "anchor device", that power all your screens and input devices. Maybe you've got "smart clothes" with communication and power lines embedded so it can communicate with your smart glasses, or it talks wirelessly with your smartwatch, etc.. Doesn't really matter how exactly it works, just that it manages your connectivity and holds your local data. Maybe you have it in your bag, or on a necklace, it it is your necklace, or on your wrist (maybe it IS your smartwatch) etc...
Reading through the spin.. it's just an object store and pubsub with some demo apps? Like S3 and SNS on Amazon or Google Datastore and Cloud pubsub? That's it? Why would anyone use this instead of AWS or Google?
I'm a bit skeptical of this. The way the CEO brushes off the issue of what happens when the user is offline leads me to think that they don't have a great answer for that. Services like Dropbox and iCloud are great because you don't need constant connectivity to use them - with Dropbox you always have all your files on your computer, and with some exceptions (Photos) the same is true for iCloud. This company might be living in a Silicon Valley bubble of perfect connectivity if they think that people always have a fast and reliable internet connection.
I'm also interested to see if they have a way for you to run your own Upthere server instead of using their service. I'm not eager to have to trust yet another third party with my private data, and if I can't host it myself somehow I'll probably avoid using the service.
The hard part is that you can't expect the user to make choices on how to merge conflicting edits in every circumstance, being online and having collaborative live editing is really the simplest solution.
But a very, very large percentage of the time the reason that conflicts happen in the first place is that one of the devices was offline at the time of an edit. So to say the solution is "tough, you just won't be able to save anyway" - I don't see how that is an improvement.
He talks about caching. As soon as you add caching, you now have a problem that isn't just close to syncing. It's utterly, absolutely identical to syncing. Syncing is just distributed caching where you only ever invalidate a cache entry when someone tells you to by deleting the local file.
You can "skip" the cache consistency problems by "just" always treating "the cloud" (which is internally probably a distributed cache) as the authority, but now you have an unreliable distributed system from a user's point of view since the thing you are skipping is how to handle real-world network weather. You can't eliminate problems by pretending they don't exist.
You also can't skip this problem with "the cloud." I am SO damn sick of "the cloud" being talked about as magic beans. If you're building a global distributed drive, then you are doing a distributed cache -- SYNCING -- in the cloud between cloud nodes. If you can do it there, why not do it on the end-user device too and let me cache my data? If you are using someone else's boxed cloud service to do this, then you are letting them build a distributed system and operate it for you in the cloud and if you ever have problems with it you better hope they have good support/SLA... and you're also doing something trivially duplicated. If that's what you're doing why don't I just use s3fs?
I sincerely hope there's more to it than this. I can trivially do this now with sshfs or Expandrive if I want a pretty GUI, and it's only tolerable with cloud storage that's physically very close to me (about <500mi).
I agree with many things you say. Take AWS S3 as an example: it reduces the problem by making objects immutable. Internally, they can cache aggressively without scaling fine-grained consistency on the objects as such. As an S3 user I can aggressively cache on the client too.
Now we've reduced the problem to consistency on the metadata structure which aggregates objects for the user. There are "well known" ways to do this for traditional trees. Other well known options include doing an all search based approach (i.e. always talk to a server for metadata, perhaps with local result caching), and so on.
The last part you wrote is what I was getting at: you cannot eliminate distributed systems problems from distributed systems. You can only either (a) push them around into different parts of your design or (b) outsource them by using stuff like Amazon's DB-as-a-service stuff. Using immutable objects is (a): now you have a distributed metadata problem.
If they're doing (a) then it's just another Dropbox but with less tolerance for disconnected operation. If they're doing (b) they're just reselling and maybe with a nice client. That's not doomed as a business of course but it's not all that technically interesting.
I strongly agree with you re 'conservation of distributed systems problems'. I'm being vague because I know what they're doing, but I'm limited in what I can comment on.
Their public comments are that they are running their own servers and not reselling other storage.
Personal opinion: no one can do (c) because of (a). That is, any possible (c) must tackle the fundamental hardness in the distributed systems problem, and this is what we agree (a) is doing. Using immutable objects as in S3 just shifts the problem elsewhere, while it reduces it, it doesn't solve it.
Immutability does reduce the problem by reducing the data footprint of stuff that needs to be synchronized.
So let me guess:
They're doing a distributed metadata store where the cloud is the tie-breaker combined with flexible caching of immutable content-addressable (identified with SHA512 or similar) objects.
In that case it sounds like cloud-hosted-only ipfs plus a nice client to access it from a host.
Woah, getting a strong whiff of "we will XML everything" from the late 90s. Lots of big words, grandiose vision, but all reeking of technical inefficiency, which when resolved will yield... a form of an SQL database. 4 years in stealth mode also don't instill much confidence that they will live up to the claims.
54 comments
[ 2.5 ms ] story [ 58.3 ms ] threadThe founder spent 22 years working for St. Steven, and yet he didn't learn one of the most important lessons:
“People think focus means saying yes to the thing you've got to focus on. But that's not what it means at all. It means saying no to the hundred other good ideas that there are. You have to pick carefully. I'm actually as proud of the things we haven't done as the things I have done. Innovation is saying no to 1,000 things.”
And then pictures, video, music, etc come along and wreck everything.
Maybe there are better machines to rage against than a hippie OS.
Also, I wonder if they've done the math on how much data you can actually cram into the airwaves. You can't instamagically move more data through a finite spectrum at rush hour on a Wednesday.
We badly need standardized home servers that can run 24/7 and managing storage and processing tasks for our mobile device. It would solve sooo much of the problems derived from unreliability of mobile devices.
Something that the regular users have an incentive to buy, because the primary advertised features are something they need or really want, and that then also offers a wide variety of additional services, such as mail, VoIP, coalescing proxy for your mobile devices to save their battery, storage, etc...
Copying files into your personal cloud while offline should still work. Deleting/renaming/moving files from the cached directory tree should still work offline. Opening cached files should still work offline.
They better be low power - I pay ~34c/kWH for at my house. I recently looked into building a NAS for storage + OwnCloud and I can buy an awful lot of BackBlaze + AWS for the same price.
I'm on what is effectively 8/1 and I'm starting to struggle with the size of Windows Updates (8->8.1 failed multiple times resulting in a repeated 2GB download), XBox One games (20-40GB), streaming and RDP/Citrix/VPN for working from home.
> I'm not even begging for the common engineering sense anymore
OneDrive! It doesn't even implement any kind of LAN Sync - you'd think MS would have bolted on some of the newer SMB stuff like HomeGroup (which is discovery+SMB over IPv6 AFAICT)
>For practical reasons and the love of God, stop with the requirement of being constantly on-line to use a goddamn app.
Ubisoft managed to lock the whole of Australia out of one if its recent betas with this mentality.
If, but it is a big if, I have understand correctly it is a simple cloud service but it doesn't save your file in your hard drive, but directly and only on the cloud...
"Upthere says it plans to offer an API in hopes that developers will make it the default storage solution for their own apps"
So... how is that in any way different to using the AWS API to upload stuff to S3? Being a "storage solution provider" (and reselling other companies' cloud offerings at that) has some razor thin margins and a whole lot of competition. I don't understand what makes these guys different.
Personally, what I'd like to see is mainstream software bundled with https://ipfs.io/ technology. If content could be served via a DHT plus all the extra things IPFS offers, we'll be a lot closer to a world where content doesn't get lost. At that point I'd gladly pay a company to store my personal files (by simply giving them a list of hashes which resolve to my computer so they can download them), and trust that it's safe because they were encrypted on my end and the company is just ensuring that the content can be redownloaded should my computer explode.
On a sidenote, IPFS would also solve a lot of problems with public data, where everything can be reliably synced (and even updated with IPNS) for people with no persistent access to the Internet (and people on space stations or the Moon or Mars, should we ever put bases on them).
Yet you were the "senior vice president of software engineering for Apple." You were in a better position than almost anyone else to "do what you envisioned." What stopped you? If you could not do it at Apple, why will you be able to do it at a startup?
The ability to start a startup is a new bargaining chip for high value corporate employees. Why build an innovative feature at your company -- and give them all the money -- when you could just leave and do it yourself, and keep all the money?
There is a moral hazard for corporate employees that poses a large risk to corporate innovation. If an employee at a company stumbles upon an innovative opportunity, it can be economically sensible to for the employee to avoid implementing it, in preparation for a future startup that will attract easy VC money.
The best solution to this is probably just for big companies to grant more and more stock to employees to keep incentives aligned.
So no, not a huge risk, and higher percentage chance of success than anyone else starting the same company.
So I don't think you can generalize on that point. And as far as this goes:
> That guarantees at least enough investment to hire a team the size of what you had at Apple.
There's also no way this is a guarantee. Sure, it'll be easier to get money than an 18 year-old walking in off the street will, but again, you can't compare the steady job, steady paycheck, "let's build a feature internally" with "let's go start a startup".
I will definitely agree that having the experience of a vp of engineering is better than not having it, but that's a far cry from saying that quitting your highly paid job as a VP of engineering for the biggest company in the world to start one of many cloud storage companies is "not a huge risk".
Cloud storage will still be important in this new single-device computing model, but only as a means for archival and backup. Sync won't come into play anymore because you only have the cloud and a single device to manage, and changes will tend to only propagate in a single direction from the device to the cloud.
It's hard not to think of this: http://mashable.com/2012/10/17/color-shuts-down/#
High-concept stuff doesn't seem to fair well, the small and growing always seems to do a lot better. That said good luck to them.
I'm also interested to see if they have a way for you to run your own Upthere server instead of using their service. I'm not eager to have to trust yet another third party with my private data, and if I can't host it myself somehow I'll probably avoid using the service.
You can "skip" the cache consistency problems by "just" always treating "the cloud" (which is internally probably a distributed cache) as the authority, but now you have an unreliable distributed system from a user's point of view since the thing you are skipping is how to handle real-world network weather. You can't eliminate problems by pretending they don't exist.
You also can't skip this problem with "the cloud." I am SO damn sick of "the cloud" being talked about as magic beans. If you're building a global distributed drive, then you are doing a distributed cache -- SYNCING -- in the cloud between cloud nodes. If you can do it there, why not do it on the end-user device too and let me cache my data? If you are using someone else's boxed cloud service to do this, then you are letting them build a distributed system and operate it for you in the cloud and if you ever have problems with it you better hope they have good support/SLA... and you're also doing something trivially duplicated. If that's what you're doing why don't I just use s3fs?
I sincerely hope there's more to it than this. I can trivially do this now with sshfs or Expandrive if I want a pretty GUI, and it's only tolerable with cloud storage that's physically very close to me (about <500mi).
Now we've reduced the problem to consistency on the metadata structure which aggregates objects for the user. There are "well known" ways to do this for traditional trees. Other well known options include doing an all search based approach (i.e. always talk to a server for metadata, perhaps with local result caching), and so on.
If they're doing (a) then it's just another Dropbox but with less tolerance for disconnected operation. If they're doing (b) they're just reselling and maybe with a nice client. That's not doomed as a business of course but it's not all that technically interesting.
If there's a (c) it's not in this article.
Their public comments are that they are running their own servers and not reselling other storage.
Personal opinion: no one can do (c) because of (a). That is, any possible (c) must tackle the fundamental hardness in the distributed systems problem, and this is what we agree (a) is doing. Using immutable objects as in S3 just shifts the problem elsewhere, while it reduces it, it doesn't solve it.
So let me guess:
They're doing a distributed metadata store where the cloud is the tie-breaker combined with flexible caching of immutable content-addressable (identified with SHA512 or similar) objects.
In that case it sounds like cloud-hosted-only ipfs plus a nice client to access it from a host.
Cold? Warm? :)
Nice logo though.