Ask HN: Is your company sticking to on-premise servers? Why?
I've been managing servers for quite some time now. At home, on prem, in the cloud...
The more I learn, the more I believe cloud is the only competitive solution today, even for sensitive industries like banking or medical.
I honestly fail to see any good reason not to use the cloud anymore, at least for business. Cost-wise, security-wise, whatever-wise.
What's a good reason to stick to on-prem today for new projects? To be clear, this is not some troll question. I'm curious: am I missing something?
781 comments
[ 5.1 ms ] story [ 443 ms ] threadWe'll use it for as long as its producing useful output. Lets say 5 years? Probably a little longer?
A 60bay 4u western digital with 14tb drives is under us$50k?
definitely got a dell md1280+1u server with 70x10tb 2yrs ago for under us$50k. Fully populated the following year..
A 2u dell with maybe 20 cores n 128gb ram each should cost less than us$10k.
And we just got 4 or 5 dell switches with 48x10g ports for 50-70k? I'm not sure.
What's the equivalent in the cloud?
Similarly, your 70*10tb tells nothing. What's the redundancy on that, how much of that space did you lose to it, where do you store your backups and at what cost?
As for networking, having those switches is nice, but you still need your internet connection if you're serving anything online from this.
Storage is usually a stripe of 10 disks in raidz2 + 2 spares. Then we do a nightly zfs send..
The internet connection for the hosting is 100mbps. The users share 1gbps of internet.
We pay only for hardware. Rent and utilities. And the internet connections. Everything else we can do it ourselves.
I mean. Seriously. Whats the cost of running 100 x 4 x 2080ti in the cloud for a year? Or storing 500tb(1 instance only,no redundancy) ?
That's enough for 3 HP ML10s equivalently configured plus the power to run them for their entire lifetime, which is well over 3 years.
You are paying out the nose for the fancy-pants deployment and administrative features that the cloud gives you, and for a lot of use cases, that isn't needed.
Even for production workloads, the 4x premium over base hardware cost is absurd.
Besides, I'm not exactly trying to win a war here. Simply trying to know under which circumstances it is better to go for on-prem vs public cloud.
1. Scale up/down - you can say turn off servers off season/nights and spin up only when needed.
2. Your business is not big enough to hire a person for Ops.
3. You don’t have the space for a server room/ unwilling to take the burden of dealing with a co-location provider and are willing to pay through the nose for that choice.
4. Your devs are okay with doing Ops (managing server resources on AWS is still Ops. You still need sense of firewalls, DNS, backups, updates, change management etc)
5. Ownership vs Operation : if a business is not sure they will sustain that amount of resource needs long enough to buy hardware. Buying reserved instances on Amazon comes to mind as opportunities that could have been better as a co located servers.
6. Geography - At times you need geographical spread (CDN, or multiple office/customer locations)
IMO, unless you can find yourself one such justification for the extra costs, you should consider self hosted or atleast collocated servers. At least this is what I tell my friends in the business who seek advice on infrastructure.
(1): Available even if internet is down.
----
Also frequently doing most scientific computing tasks are _much_ cheaper on-premise (if you have space for it). Especially given that some have setups like multiple terabyte of RAM.
I'm not sure where to even to start with this. It's not even wrong ...
For hosted and colo servers, you can have "remote hands" unbox, rack and cable your servers. My colo facility, he.net in Calif., does that for free, including IPMI configuration.
AWS does not secure your servers. They have a "shared security responsibility" cutout in all their contractual language. So they have private networks, but you're responsible for iptables, application security, etc.
Their AMIs aren't even encrypted, ffs. That's right, 99% of AWS customers don't do disk encryption, even if they're required to do so for compliance reasons, because you have to do an extra step yourself to encrypt it with your KMS. (GCP does encrypt.)
By any reasonable metric, onprem is much much cheaper than the cloud.
Simply, _cost_
Our compute servers crunch numbers and data at > 80% util.
Our servers are optimized for the work we have.
They run 24/7 picking jobs from queue. Cloud burst is often irrelevant here.
They deal with Terabytes or even Petabytes of moving data. I’d cry paying for bandwidth costs if charged €/GB.
Sysadmin(yours truly) would be needed even if it were to be run in the cloud.
We run our machines beyond 4 years if they are still good at purpose.
We control the infra and data. So, a little more peace and self-reliance.
No surprise bills because some bot pounded on a S3 dataset.
Our heavy users are connected to the machines at a single hop :-) No need to go across WAN for work.
We have approximately 1000 cores across 45 servers.
A few public facing services - web servers, small APIs, a web front end tool for a big genetics database. Nothing big.
Public services are cordoned off from the compute cluster.
What form do these jobs have?
How do you manage workloads?
How do you manage resources? Do users have quota for compute and storage?
Do you use GPUs? If so, how do you deal with malfunction?
Is this a distributed processing? Are the machines heterogeneous? What do you use for that cluster?
What if a job requires dependencies? Do you create a compute environment on the fly or do all jobs have the same dependencies and these don't change much?
How do you do data governance? Is the data read only? Do you have an API to fetch the data from the job code?
https://aravindh.net/post/sysadmin/
> What form do these jobs have?
Mostly batch jobs written as bash scripts. Occasionally, some users run singularity containers. But, all through SLURM.
> How do you manage workloads?
SLURM
> How do you manage resources?
As a sysadmin, my inventory is via Ansible. All activity on servers happen via Ansible only.
> Do users have quota for compute and storage?
Yes, Users typically can run 72 cores, 512 GiB of mem at a time. Rest is queued until resources are released. Disk quota is only for home directories - 400GiB(enforced by ZFS refquota).
> Do you use GPUs? If so, how do you deal with malfunction?
No, weirdly our workloads(genetics and genomics) don’t fit the GPUs very well, as they are sparse matrix walks with wide precision floats. But, we plan to try for some other stuff soon.
> Is this a distributed processing?
No. Jobs run one node at a time.
> Are the machines heterogeneous?
Yes, inteL based servers all the way from Haswell to Cascade lake.
> What do you use for that cluster?
SLURM
> What if a job requires dependencies?
Taken care of by SLURM.
> Do you create a compute environment on the fly or do all jobs have the same dependencies and these don't change much?
I guess you mean the software libraries and tools that jobs use? If so, our central software repo is NFS mounted on all compute nodes. Users can install things they need if admin priv is not needed. If not either Singularity or email to me.
> How do you do data governance?
This is the painful and human oriented task. We lock down data transport to outside world, educate the users about data policies and then spend a lot of time looking at data flows with hope.
> Is the data read only? Do you have an API to fetch the data from the job code?
Sorry, I could not understand this question. Do you mean metadata about A job?
There's a pretty interesting NSF-wide project for managing clusters in a more commoditized way as part of https://www.xsede.org. You might describe it as a "private heterogeneous almost-cloud" in its goals? That might be saying a bit much.
Density, latency, storage throughput, etc. were in favor of DIY (plus the pricey professionals to run it) rather than cloud offerings. When I was there (2016) I did some basic math for being able to use a cloud provider for some of our lighter workloads when the local cluster was loaded down. Astronomical without a contract, which is quite the thing to set up, etc.
Worth noting that while they do alright, NAU (first link) is hardly a top-tier university with bleeding edge technical requirements.
To manage runtime dependencies we use cvmfs.
A read only distributed filesystems, kinda like a very specific NFS for software distribution.
If that is never been a problem, cvmfs it is quite stable and very used in our field.
With respect to the original questions -- I've investigated running a full SLURM cluster on the cloud and it just never seems worth the effort. For larger clusters than mine, maybe, but then when you start to hit the levels you're talking about, I just don't see the point in moving to the cloud. It would be hard to hit that sweet spot where the costs would make sense. Amazon even has a series on scaling a SLURM cluster with AWS EC2 provisioning, but it just seemed like more work than was justified for us.
[^0]: https://news.ycombinator.com/item?id=23129417
Now, we thought of several ways of solving this. We have a branch that uses Kubernetes but this is not the one that is deployed.
We know about SLURM and this is something we will support for a coarse granularity. i.e: notebook level jobs. This is a common scenario and we'll need it for our AppBook (we allow users to turn a notebook into an application with one click and we automatically generate the form fields for the features, and the API endpoint for the model).
However, I'm interested in finer granularity computation; as in: I want a job for the cell. This involves looking into Jupyter kernels to circumvent the front-end<-->kernel disconnection (we'll have to do that for another use case in low internet bandwidth scenario).
>Sorry, I could not understand this question. Do you mean metadata about A job?
How do your users upload, and manage data access and is the data mutable and versioned. What version of the code ran on which version of the data, and who changed that data, etc.
How do users access the data from a Jupyter notebook? Is the data in different object storages? Do you proxy the requests and handle access, things like that.
1. Their use case is kinda different. The servers mostly run heavy CS research related stuff. E.g. they might have heavy CPU load and heavy traffic between they servers but they have less often heavy traffic to the "normal internet" (if they have heavy traffic to the outside it's normally to other research institutes which not seldom have dedicated wire connections).
2. They might run target specific optimized CPU or GPU heavy compute tasks going on for weeks at a time. This is really expansive in the cloud which is mostly focused in thinks like web services.
3. When they don't run such tasks in the research groups they want to allow their juniors to run their research tasks "for free". Which wouldn't work with a payment model as done in the cloud.
4. They don't want to relay on some external company.
Also I'm not sure are there even (affordable) cloud systems with compatible spec? (like with 4+TB of RAM, I'm not kidding this is a requirement for some kind of tasks or they will take way to long and requires additional complexity by using special data structures which support partial offline data in the right way, which can be very costly in dev time)??
[0] https://aws.amazon.com/about-aws/whats-new/2019/10/now-avail... [1] https://aws.amazon.com/ec2/pricing/on-demand/
The choice is not between "on premises" and "cloud", it's between "on premises" and "our glossy faculty brochure says you can do this but really you can't".
I know some organizations handle this really well, but when mine didn't, it sucked to be a developer there.
But, I’ve found that there are two reasons why there is still a lot of on-premise academic compute: 1) legal, 2) cap-ex.
For the first, there are still many agreements for data sharing that require strict security compliance. It is certainly possible for this to work on the cloud, but it is more difficult than setting up a hardened cluster that’s not exposed to the internet. IT is normally better setup to audit and approve these on-premises systems than cloud systems.
For the second, it is difficult to estimate and write in all of the cloud costs for a grant budget. Trying to manage op-ex can be more of a hassle than just budgeting X amount for “servers” (as a cap-ex). Often, you just care about getting the job done, but less about how long it takes. So, a set capital expense has lower risk than underestimating your cloud needs and not being able to finish your project. (Also, as a bonus, once the equipment is paid for, you get to keep it to use on the next project, or unfunded research).
I'm quite sure that 2 months of my time waiting for results (and the impact on colleagues occupying their HPC cluster) was worth the $9k or so it would have cost to rent enough VMs to get my results in about 4 hours or so instead of 2 months.
That's not counting the hours of machine and my time wasted building enough support for the awful queueing and execution environment it used which was almost impossible to mock up locally.
The big question is - how much do you trust these providers, and do you think they are more competent at security than Amazon/Google/Microsoft?
1. Trust a provider which whole existence relies on trust and which you can audit or at last cross-check the audit and security processes (as a Bank you are normally not a small customer).
2. Trust a provider which might 1st be a potential competitions in some business fields. 2nd is so big that it can easily afford losing you. 3nd for the same reason doesn't allow you any insights into there internal processes. etc.
Plus many of the banks having their own hardware also have their own IT team. So it's often about trusting your own people. I mean either you keep your it or you outsource _and_ go into the cloud. Keeping local servers but outsourcing IT at the same time seems kind not very clever tbh.
I'm sure they're contracted not to do anything bad, but there is risk of an insider incident at the cloud provider leading to an incident the customer can't prevent.
Now imagine you're aware that Amazon's retail arm is quite happy to launch competing products based on sales data from marketplace sellers, so it's not like they have strict firewalls protecting competitors who pay Amazon for services.
Now imagine Amazon offers you a bunch of security options that sound suspiciously like snake-oil. Like an encrypted storage service where they store the encryption key and transparently decrypt files when you request them.
Now imagine you know people with hypervisor access can do what they like to your VM and leave no evidence whatsoever. And your on-prem experience is that there will definitely be some people with hypervisor access.
Now imagine you've been through various audit and certification processes yourself, and you don't find yourself reassured by the knowledge Amazon has too.
Now imagine if you were a boss at AWS and you found an insider had violated policies and accessed things they shouldn't have. Would it be better for your career to publicise that widely, or reprimand the guy then hush things up?
Now imagine you've been an ambitious cowboy at certain times in your career, and although you've never accessed a competitor's data to give an edge to something you were working on, you can see how it would be a temptation.
I can understand how a person who believed all that would believe putting their business in their competitors hands would be a naive decision.
That sounds like it's hard to believe all this. But some thinks you state actually had some proves that they do happen...
Like consider how many cases of "help big company randomly blocked our account" popped up on hacker news and twitter the recent years.
Also consider news like: "Amazon spies on seller to get a competitive advantage when selling their own products".
Or e.g. companies short term massively changing their service terms or API's. Deprecating old ones etc. This might be fine for customers which products are always changing anyway. But for banks this isn't attractive at all, they only want changes if they are really needed for them.
Many banks seem to be aiming for the minimum viable external data processing.
It's way to risky that changes in local and/or US regulations will require you to move your whole infrastructure to a new provider like in a month or two (which is completely impossible).
Btw. same applies for any bank relying on the infrastructure of a foreign company.
Like e.g. just think about the whole legal mess around CLOUD act.
AFAIK data that passes through the US may be legally intercepted by the US authorities.
If by cloud you mean a public cloud like Google, Amazon, or Microsoft, then forget about it; not with these companies piping data directly to U.S intelligence.
Does it just mean you let everyone spin up VMs etc rather than requiring them to go through IT?
But besides that it's a about how the infrastructure is manages, in the classical way. Or with somethings which allows easy setup of new "containers" and similar like Kubernets. This doesn't necessarily means VMs. It can be more lightweight containers, e.g. based on Linux namespaces. Nor does it means you can sidestep the IT ;=) But normally it mans you can much easier spin up new services or change scaling between them.
Cloud and on-premises are not mutually exclusive, as many "bits I push to the cloud" get on-premises of a ponytailed guy named John. We focus on what people mean[^1], not what they say. Some even make a living saying "chemical-free"[^2] or "digital marketing" publicly.
> Does it just mean you let everyone spin up VMs etc rather than requiring them to go through IT?
This is the on-demand, self-service aspect. What the users get as a result of their demand depends on the need you're trying to fulfill and defines (SaaS, PaaS, IaaS, etc.). Do you want them to be able to interact with an application, deploy/run applications, or get a VM? Here's a resource that summarizes it[^3] and the Wikipedia page gives an overview[^4].
[^1]: https://en.wikipedia.org/wiki/DWIM
[^2]: https://en.wikipedia.org/wiki/Chemical_free
[^3]: https://www.redhat.com/en/topics/cloud-computing/iaas-vs-paa...
[^4]: https://en.wikipedia.org/wiki/Cloud_computing
1. IaaS - Which I mainly define as the raw programmable resources provided by "hypercloud" providers (AWS, GCP, Azure). Yes, it seems that using an IaaS provider with a VPC can provide many benefits over traditional on-prem data centers (racking & stacking, dual power supply, physical security, elasticity, programmability, locations etc).
2. SaaS - I lump all of the other applications by the hundreds of thousands of vendors into this category. I find it hard to trust these vendors the same way that I trust IaaS providers and am much more cautious of using these applications (vs OSS or "on-prem software" versions of these apps). They just don't have the same level of security controls in place as the largest IaaS providers can & do (plus the data is structured in a way that is more easily analyzed, consumed by prying eyes).
As for the SaaS, I guess your mileage may vary. I trust some of them really make a point of securing your data :)
- unwillingness to seed control of the critical parts of our software infrastructure to a third party.
- given our small team size and our technical debt load we are not currently able to re-architect to make our software cloud-ready/resilient.
- true cost estimates feel daunting to calculate, whereas on-prem costs are fairly easy to calculate.
What about your first point though? Do you not trust a 3rd party to maintain infrastructure properly? In what way?
Typo alert: you mean “cede control”, not “seed control”.
I also have some other APIs hosted in the same way (eg. website thumbnail generation API), for the very low traffic I have and no chance of getting burst traffic I think the use case of a VPS or dedicated server is perfect.
[0]: https://aws.amazon.com/lightsail/
That said, I've stuck with DigitalOcean even though Lightsail tests fine. I've had a great experience over the years with DO and see no reason to leave.
I personally refer to servers as "cloud" only if they have auto-scaling and usually to services such as AWS or GCP instead of VPSs such as DO, Linode, Vultr, etc.
Yes they offer managed databases and kubernetes, but these are newer offerings. I of course don't know how well used they are, but I never see or hear anybody talking about them.
I use them for all my stuff when I'm not on company time, and the reason is because I can just be like "spin me up a Debian box," which I cannot easily do on the things folks typically describe as cloud providers.
> personal data (IP & such) I agree, Still storing IP is a bit strange. I am still working on the privacy policy and option to not store IP at all. All tracking is session-based (no cookie), and no personal information is stored (only the IP, which is currently only partially displayed, and which is AFAIK not considered personal information).
> track the users mouse movements The site has no inputs, so it only tracks the mouse movements/scroll. Do you consider this to be a privacy concern, especially if the visit is anonymized (can not be linked to yourself as a person)? Seeing a replay of the visit is really useful for understanding what visitors do on the site and how to make their experience better.
> Kinda strange these days, isn't it? I agree, but I still think it's a lot better than other platforms, it complies with GDPR regulations, no cookies involved, no data is sent to third parties, no personal information is being stored (only the previously mentioned IP). I am also continously improving it and adding more privacy-focused features.
We don't generally trust cloud providers to meet our requirements for:
* uptime (network and machine - both because we are good at reliability [and we're willing to spend extra on it] and because we have lots of fancy redundant infrastructure that we can't rely on from cloud companies)
* latency (this is a big one)
* security, to some degree
* if something crazy is happening, that's when we need hardware, and that's when hardware is hard to get. Consider how Azure was running out of space during the last few months. It would have cost us an insane amount of money if we couldn't grow our data centers during Corona! We probably have at least 20-30% free hot capacity in our datacenters, so we can grow quickly.
We also have a number of machines with specs that would be hard to get e.g. on AWS.
We have some machines on external cloud services, but probably less than 1% of our deployed boxes.
We move a lot of bandwidth internally (tens of terabytes a day at least, hundreds some days), and I'm not sure we could do that cheaply on AWS (maybe you could).
We do use <insert big cloud provider> for backup, but that's the only thing we've thought it was economical to really use them for.
Also surprised about latency, latency from what to what? Big cloud providers have excellent globally spanning networks. Long distance networking is crazy expensive, though, compared to the peanuts it costs to transfer data within a data center.
Reliability - again, not sure I buy it. Reliability is "solved" at low levels (such as data storage), most failures occur directly at service levels, regardless of whether you have the service in house or in the cloud.
The rest of your points make sense.
How much would it cost to move this across boxes in EC2? I actually don't know, that's not a rhetorical question. A lot of our servers have 10-40gbit links that we saturate for minutes/hours at a time, which I suspect would be expensive without the kind of topology optimization we do in our datacenters.
> Also surprised about latency
We've spent a surprising amount of money reducing latency :) We're not a high frequency trading firm or anything, but an extra 1ms (say) between datacenters is generally bad for us and measurably reduces performance of some systems.
> Reliability is "solved" at low levels
To whatever extent this may be true, it's certainly not true for cloud providers. One obvious example is that EC2 has "scheduled maintenance events" where they force you to reboot your box. This would cost us a lot of money (mostly in dev time, to work around it).
Also, multi-second network dropouts in big cloud datacenters are not uncommon (in my limited experience), but that would be really bad for us. We have millisecond-scale failover with 2x or 3x redundancy on important systems.
I believe I saw a slide that average lifespan of an EC2 instance at Netflix is 14 minutes.
I’m not necessarily saying cloud will work for your setup, but you can’t compare like that.
Nothing. You generally only pay for data going out of cloud providers. Not data going in or data being transferred within the same region.
> One obvious example is that EC2 has "scheduled maintenance events" where they force you to reboot your box. This would cost us a lot of money (mostly in dev time, to work around it).
You're not going to have a successful cloud experience unless you build your applications in a cloud suitable way. This means not all legacy applications are a good fit for the public cloud. Most companies really embracing the cloud are mitigating those risks by distributing workloads across multiple instances so you don't care if any one needs to be restarted, especially within a planned window.
> Also, multi-second network dropouts in big cloud datacenters are not uncommon (in my limited experience), but that would be really bad for us. We have millisecond-scale failover with 2x or 3x redundancy on important systems.
Are these inter-region network dropouts or between the internet and the cloud data center? You're not going to be relying on a public internet connection to the cloud for critical workloads.
All that being said, there are plenty of workloads which I don't think fit well in the cloud operating model. You may very well have one of them.
It's nobody's fault that the billing structure at Amazon is so complicated and confusing.
Except Amazon's.
Our fault, I suppose -- but multi-az is prohibitively expensive if you need to run anything data heavy distributed.
> but multi-az is prohibitively expensive if you need to run anything data heavy distributed.
If you communicate between your AZs via ALBs, multi-az is effectively free. Our bill is so high because within our Kubernetes cluster, our mesh isn't locality aware; it randomly routes to any available pod. 2/3rds of our traffic crosses AZs.
Right, we have not done this. We basically decided it was cheaper to keep doing the “old school” thing and not spend a bunch of dev time trying to do it in a way that supports arbitrary failure of N boxes. We just spent the money to make it unlikely our boxes or networks will fail, and if they do fail we may be sad (but it rarely happens, and has not yet happened in a catastrophic way).
> Are these inter-region network dropouts or between the internet and the cloud data center?
I’ve seen multi-second dropouts within a DC (cloud, not my current company), and multi-hour single-path failures between DCs (usually something like a fire or construction cutting a line). But all of our DCs have at least 2 physically independent routes to the internet, so it’s never taken us fully offline.
This is not true, AWS charges for all traffic cross AZ, so if you want a resilience within a region (which you mentioned in next paragraph) you will do a lot of that talking and this ends up a non trivial cost. You can do some optimization though by having your apps being aware of AZs they are on, but most places don't do that, more common are places that run everything from a single AZ.
Data transfer between instances in the same AZ is free. If the data crosses AZs, you're changed $0.01 per GB in both directions. This is for instances on a VPC. I think the pricing model is different for classic EC2.
There are some exceptions like all traffic between EC2 and ALBs being free.
Edit: Pricing is described at https://aws.amazon.com/ec2/pricing/on-demand/#Data_Transfer
I think everyone does something of this sort nowadays, that's why networking is ~free within data centers :)
> but an extra 1ms (say) between datacenters is generally bad for us and measurably reduces performance of some systems
That speaks to me. You will always be just the n-th client unless you own the cross-datacenter data links (i.e. have full autonomy on deciding the priority of the traffic). It's similar to the covid provisioning problems you had mentioned.
> One obvious example is that EC2 has "scheduled maintenance events" where they force you to reboot your box.
Yeah like others pointed out - that's just what "cloud" is, and is generally a good idea. You're supposed to handle a certain % of your machines going dark without a warning without violating any SLO (or even worse, certain % of your machines "pretending" they're up but actually being ridiculously slow for this or that reason; and don't even get me started on CPU/RAM bitflips).
It sounds to me that you run an extremely highly sensitive service, something for which paying for true ownership of the hardware just makes sense to remove those kinds of risks that most services don't care about. At the end of the day "cloud" is a shared resource, and no resource separation efforts will be 100% effective.
It's interesting that you say this, because I think this just boils down to how you treat failures. Boxes will fail. It's inevitable, even if you run your own hardware.
We treat failure as an every-day event and have set up our systems so it doesn't matter. One box fails, an automated system notices and brings up another to replace it, while the remaining boxes take slightly more load for a few minutes.
Sure, that kind of failure tolerance doesn't come for free. But, again, you're going to have failures, no matter how much work you put into reliability (which also doesn't come for free!).
What specs are those? I was under the impression AWS has everything from extremely tiny to giant terabytes-of-ram-for-SAP instance types.
Amazon is constantly expanding the selection, so it's possible they've added something since we last seriously looked into this.
For most every workload I can think of, as their load increases it's always one resource in particular that's the limiting factor. (RAM/CPU/Storage/etc). So it makes sense to me that AWS instances focus on optimizing one particular resource type.
Would be interesting to hear about types of workloads that break this pattern. (Or maybe it's a few different types of workloads/services that need to be tightly integrated on one machine?)
In my experience, harder to also balance I/O so you’re close to hitting three limits, but update-heavy transactional workloads can manage it for disk-based DBMS’s.
So this is a rediscovery, at least an example, of what could be called the bottleneck principle of system performance analysis and optimization! Sooo, just look for the bottleneck(s), work on those, and f'get about everything else until, say, everything is equally a bottleneck at which time have a well balanced, call it optimized, which can be appropriate, configuration!
At times, e.g., at IBM's Watson lab, there has been a lot of work on applying queueing theory, analysis, and simulation to analyzing and then optimizing such systems, but, more closely to fully true than one might guess, all that really mattered were the bottlenecks(s)!
What industry are you in?
Are you concentrated in 1 geo location or... across USA / across 1 county / global?
Something plumbing-y; the company is not well known
> 1 geo location or...
Global, although probably 50% in the US
Haha. This can't possibly be true.
Consider that building fault-tolerance into their application may be very difficult, or impossible. Cloud would be incompatible with that.
I've seen on-premises, where reliability wasn't even objective machines that easily run for 5+ or more years without any interruption. Now parent said that they actually need reliability and you can achieve it using many technologies. Starting with RAID, dual PSU, or even hot swappable RAM or CPU (I remember SPARC machines allowed this). With full control of networking you can also make a standby node take over nearly instantaneous, when in AWS it might take couple minutes. You can achieve nearly any availability as long as you have enough money. In AWS you don't have any control and your only way is through designing your application in specific ways and that still has limitation. Just take look at RDS when everyone would want to have it instantaneous, but it usually will take few minutes.
Also, sheer cost. Literally everyone I know in my particular part of the industry uses Hetzner boxes. For what I do, it’s orders of magnitude cheaper than AWS.
Lots of us are here to ship stuff; to provide viable products that customers will like.
Moving to AWS from Hetzner would not help me ship stuff. Moving to AWS from Hetzner would not result in a better product.
Never mind the load balancing. And then we need another server for a different client, and another, and a demo server, and test & staging environments. Ugh. I wish this were on-premmable. We could probably hire an employee for the amount we could save. He'd probably be pulling his hair out supporting it tho. lol
My little on-prem HP dev server's got hundred gigs of ram and 8tb of fast storage. We run a bunch of VMs on there. We paid a couple of thousand dollars for it. And I don't even want to think of what it would cost to ship those VMs to the cloud. Nevermind just how poorly they would perform.
Started with a cluster of Raspberry Pis and expanded onto an old desktop. Primarily did this for cost (raspberry pis alone were more powerful than a GCP $35/mo instance). Everything was fine until I needed GPUs/handling more traffic than those Raspberrys could handle. So I expanded by including cloud instances in my Docker Swarm cluster (tidbit: Using Traefik and WireGuard)
So half on-prem half in the cloud. Honestly just scared GCP might one day cancel my account and I'll lose all my data unless I meet their demands (has happened in the past) so that half on-prem stores most of the data.
Unless I meet their demands, my entire infra is gone/down for days, which I can't deal with.
This hasn't been emphasized enough in these discussions.
Once you're in the cloud, you're dealing with computers as a "utility". Or are you?
When you buy electricity from your local utility, you don't just lose it in the middle of some afternoon for some unknown reason. There's a clear and established procedure, supervised by a Public Utilites Commission. You aren't forced to resort to sending emails and receiving some idiotic canned response from a bot. "Sorry we have reviewed your case and our decision is final. By the way, we don't tell you what you did wrong. Fuck you. Ha. Ha. Ha."
I don't know if Amazon or Azure are better than Google. But I wouldn't trust Google with my worst enemy's data.
Incidentally, this is exactly what happened to me on my very first attempt with Digital Ocean this year. Just an autoresponder that kept sending the same email with no way to reach a human. And this was just after creating an account and going through an payment authorization process. No money was lost, but I’m not touching a system where humans refuse to get involved.
Edit: To add a comparison on this aspect, Hetzner wanted some verification for a new account and handled it quite well with humans involved (and responsive).
https://github.com/krisk84/retinanet_for_redaction_with_deep...
I haven't analyzed the TCO yet but the bandwidth costs alone from hosting my curated model of 100GB in the cloud (Azure blob) have greatly exceeded my total operation spend from downloading the entire dataset and running training. By an order of magnitude.
we're not 100% on-prem, but aws, gcloud and azure are the worst examples of 3rd party hosting - unpredictable and with complicated billing. we're considering the alternatives to big 3 for the "cloud hosting"
Bonus points for small stuff like RADIUS for wifi and stuff. Groups charging $5/user for junk like that is absolutely awful with a high number of staff.
With a staff of 100, a single box with a bunch of hard drives is two months worth of cloud and SaaS.
TCO needs to come down by like at least 100x before I consider going server-less.
And there are clients that demand it.
And researchers, in general, like to do totally wacky things, and it's often easier/cheaper to let us if you have physical access.
GPU machines are clearly too expensive in most cloud providers compared to on-prem.
In the cloud, at least the way it's generally used, cost control is reactive: You get a bill from AWS every month, and if you're lucky you'll be able to attribute the costs to different projects.
This is both a strength and a weakness: on-premise assets will end up at much higher utilisation, because people will be keen to share servers and dodge the bureaucracy and costs of adding more. But if you consider isolation a virtue, you might prefer having 100 CPUs spread across 100 SQL DBs instead of 50 CPUs across two mega-databases.
Most dedicated servers come with unmetered bandwidth so not only is it cheap to serve large files but your bandwidth costs won't suddenly explode because of heavy usage or a DDoS attack.
In general, I would say any noncritical system I would host on-prem.
A retail company may decide that the best place to put up a new branch is coincidentally (though there might be a correlation) at the edge of what the available ISPs currently cover. They might have to make a deal to get an ISP to extend their area to where the store is going to be. However, because of lack of competing ISP options on the part of the retailer, and the lack of clients in the retailer's area on the part of the ISP, that service is probably not going to be all that reliable.
Also, that retail company may experience a big rise in sales after a natural disaster occurs, when communications (phone/cell/internet) are mostly down for the area. One tends not to think about stuff like that until it happens at least once.
It's very important for the ERP/POS systems to be as operational as possible even when the internet is down.
Also, my clients aren't software development firms. They are banks and factories. They buy a software based on features and we figure out how to make it work, and most of the vendors in this space are doing on-prem non-saas products. A few do all their stuff in IAAS or colo but a lot of these places are single-rack operations and they really don't care as long as it all works.
A lot of people in small/midsize banks feel like they are being left out. They go to conferences and hear about all the cool stuff in the industry but the established players are not bringing that to them. If you can stomach the regulatory overhead, someone with drive could replace finastra/fiserv/jackhenry. Or get purchased by them and get turned into yet another forever-maintenancemode graveyard app.