Ask HN: What would you do with unmetered 10gig?
I'd like to actually be able to justify 10gig service. Ideally, running something on it that appeals to other potential customers of the municipal broadband, to attract new customers to it.
The easy one is to set up a Linux mirror server. In the past I've run a mirror server for ~a decade, so that shouldn't be a huge deal. I like the idea of running a Tor exit node, but worry about the liability of that. Some sort of a block storage for backup service came to mind while reading that Cloudflare Utah post yesterday.
I run production Linux boxes and networks as my day job. But I don't have machine space at my house, so I'm limited in the number of servers I can put in place.
Clever ideas to attract other locals to the Municipal Broadband service?
184 comments
[ 2.5 ms ] story [ 218 ms ] threadDepending on those ToS, offering services to others could expose you to legal problems.
So most likely their city/county.
Like if I pay for a 300mbps but I need 1gbps for the day you can charge me 5 bucks extra per day that month.
Are the unit-economics of this that bad? I never understood why ISPs have such crappy/unflexible pricing tiers. Mobile carriers are way ahead on this type of flexible offering and the math seems to work.
garanteed bandwith + max bandwith
You buy a 1gb line with 50mb guaranteed. That would allow you to use it in the night to a full extend while your ISP doesn't need to provide 1gb all the time
It's a bit cheaper to prepay for an extra 100 Gb than it is to eat the overage chargers. Nothing for speed, though it would be great to get a boost for big videoconferences.
Wholesale bandwidth is 5 cents per Mbps or less at scale. Not that the ISP pays for all bandwidth, peering and CDN traffic is free.
In other words, even if you max out a 1 Gbps line 24/7, you are only costing the ISP $50 per month.
Add to that that most people are unable to consume even 10 Mbps on average and bandwidth costs become less and less of an issue. An average subscriber will use 2-4 Mbps of bandwidth during primetime, and primetime is all you care about as an ISP.
Nothing in this means that offering bandwidth on demand is impossible or a threat to the ISP business model.
It’s just that the overhead of offering an on-demand product to consumers isn’t worth the time and effort, as you’d be making peanuts while having to deal with a lot of grief.
Bandwidth on demand is a thing for businesses and operators. A good example of this is Megaport, who offers virtual cross connects and transport. Price start from $100 per month plus a varying fee per Mbps, from a few cents to a few dollars per month.
4k Netflix is around 15 Mbps alone, even 1080p is >5 Mbps.
And with games being now > 100 GB downloads these days, I would expect that to contribute meaningfully too to the bandwidth usage even if average person probably doesn't quite download new games daily.
Netflix is only about 30% of all Internet bits, and there's a limit to how much Netflix and chill anybody can do.
3 GB / hour to Mbps = 6.667 Mb/s https://www.wolframalpha.com/input/?i=3+GB+%2F+hour+to+Mbps
7 GB / hour to Mbps = 15.56 Mb/s
In bandwidth sure, but the world isn't wired that way. There's a bottleneck somewhere between the ISP and the customer, that can't support all customers on that link running maxed out. In the gym analogy, even if the gym has 1 treadmill per customer, that doesn't help the shortage of weight machines. An ISP may choose to (or choose not to) invest in adding capacity to remove bottlenecks, but that's additional capital outlay that may not be prioritized or budgeted for.
You can support 5x the bandwidth at 50% of the cost and 20% of the labor than you could in 2005-2010.
And, as I wrote before, in no real life scenario will all 100% of subscriber be running their line at full tilt 24/7.
Engineering a non-blocking 1 Gbps network isn't particularly hard or expensive using FTTH.
My ISP is a "small" provider that focuses on multi-unit urban residential buildings in the midwest. Their business model is to make an agreement with the building management and provide a baseline service to every unit, while offering upgrades and extra services to individual units that want them. They have an incentive to provide far more capacity than the baseline so they have room to offer upgrades.
I pay $25 per month for 250mb symmetrical and am definitely getting those speeds 24/7. I have the option to upgrade to 1G symmetrical for something like $500 per year, but haven't seen the need.
Unless an ISP is serving a very low density customer base, the cost of building out full capacity just isn't prohibitive anymore. My ISP only has around 20,000 customers and they have direct peering with Netflix, Amazon, Apple and others - clearly an ISP doesn't even have to get very large before those cost benefits arrive.
For example, Brazil literally has thousands of tiny ISPs, each with, at most, a few hundred customers. They operate on the narrowest of margins, squeaking out close to no profits. They cut corners everywhere. They definitely do not have spare capacity, and over sell by insane percentages.
Another example, Mongolia, where there's basically just one ISP, the government owned Unitel. They have zero incentive to do anything extravagant, or even provide good service. Its a monopoly.
The cost of building out full capacity is very much still a limiting factor in many parts of the world.
I imagine ISPs do not do this, because of how they sale their service. They want to make it hard for you to get out of the deal you're in, and hopefully in a carrot kind of way where you feel like you're getting a better than normal deal so you don't want to leave.
Giving people the flexibility to change their plan willy-nilly is goes against how most ISPs sell their service. They want to lock you in, not give you free range.
The exception is internet cafés and hotel internet. I could see something like this for cell phone top up plans, but most of those just resell what's available, which means them giving all of the bandwidth they can get.
This seems like a fundamental problem people should be demanding changes to rather than just accepting it as the way it is.
This is the difference between consumer and business Internet in many cases, and is why so many residential services severely limit the outbound bandwidth: it competes with their extremely profitable business service levels.
(Things may have changed, I haven't been involved with ISPs for 5 years)
Also a day of 300 Mbps would not have cost you thousands of dollars as you get 36 hours for “free” when using 95th percentile billing.
As far as $0.1/Mbps, not common around here. Maybe in CA at HE or something, but I can't touch that price around here. Correct me if I wrong, but Amazon's price for a megabit/mo is $55-ish...
At scale means purchasing bandwidth in quantities of 100G. You'd be surprised how straightforward it is to get reasonable wholesale bandwidth rates almost anywhere once you cross a certain monthly spend threshold.
As to the 95th percentile thing, sure, if you've already used up your "free" burst capacity, pushing a further 300 Mbps for a day will push you over the limit.
And if you are a residential ISP you can probably handle 80% of your traffic by peering at a few internet exchanges (google,youtube, netflix, etc.)..
So the wired ISP doesn't save too much money when you don't use your BW.
Could boost to 100 anytime with one click, but it was ~1 Euro per hour. Used it once lol
They gave up on it, probably because no one used it. Maybe if the price was right.
But technically, it seems easy to implement, at least on some network configs.
But the fixed rate contract for high speeds is likely more profitable for the ISP.
It can sometimes be forgiving, but other times it won't find it unless you name the file exactly correct. With a library of that size, you might have to leave it for a day or two to do the initial scrape, as it sometimes does a couple passes to find everything (why, I don't know).
Symmetric 10Gb/s are still total overkill, symmetric 1Gb/s would already be pretty good for that usage.
But it's specialized: You need at least some knowledge for the setup and on how to get your BR and DVDs into the server - and then of course a collection of DVD & BR to start with. And, in the age of streaming, this means the person running the PMS is either pretty retro, a home theatre nerd or a pirate. And you probably don't want to taint the offering with "it's mostly useful for piratez" ;-)
I've considered using something like PIA that claims they don't log, but not sure if that would help.
Technically: Pairing this with unmetered VPS (Scaleway) for VPN could be good solution
Hotspot Client -> Hotspot -> Wireguard VPN -> Scaleway VPS
That way the client always has hidden your ip and traffic encrypted.
$3/month for 200/Mb https://www.scaleway.com/en/virtual-instances/development/
Still, high latency internet for free is better than no internet access at all.
Another solution can be registration form and tracking end user private IPs assigned (Ipam) for such any legal issues.
My old company, everyone worked remote and we set up a VPN to our data center as the default route. One of my employees did some benchmarking comparing the VPN to our data center vs. without VPN.
He found that it was faster to use Comcast to our data center, and then using their InterNAP optimized transit. I want to say it was something like 30% faster on average, but I don't have specific numbers and they'd be ~8 years old now anyway.
Even Comcast will be able to beat that.
Set em up, turn on filtering, log stuff, turn over any records if/when law enforcement asks.
You could probably hack something together that costs less, but that's adds overhead and wastes time; you need just enough to prove you're doing diligence and keep out of the low hanging fruit. Log everything, and if the fuzz asks for records be able to hand them over.
I've run an open network to try to normalize it for the past decade, and I hope others are willing to do the same.
Mom and Pop shops are a different story, but most of the places around me are using Meraki boxes -- I recognize the log in prompt.
And broadly speaking, the cops realize that coffeeshops are going to have wifi and aren't explicitly putting up Tor nodes or trying to be an ISP, so they'll cut them slack if they can provide what deets they can. Usually they have insurance and lawyers to CYA on top of this.
Now, if I was taking away from the incumbents... :-)
I'd have to check into the terms of the service though.
Related: Anyone used those Ubituiti Mesh things? Not the Amplifi, the other mesh. I've toyed with the idea of putting some of those around the neighborhood to provide some sort of wide area wireless, but never pursued it.
Only do this if you're prepared to talk with law enforcement about child pornography being shared from your IP address.
See USA:
- https://www.mjpetro.com/search-and-seizure/search-warrant-at...
vs Canada:
> Receiving a notice does not necessarily mean that you have in fact infringed copyright or that you will be sued for copyright infringement.
> The Notice and Notice regime does not impose any obligations on a subscriber who receives a notice and it does not require the subscriber to contact the copyright owner or the intermediary.
- https://ic.gc.ca/eic/site/oca-bc.nsf/eng/ca02920.html
- https://www.mccarthy.ca/en/insights/blogs/snipits/you-can-st...
- https://apanewslaw.wordpress.com/2014/06/17/the-supreme-cour...
But IANAL, please correct me if I'm wrong here.
Notice and notice is about copyright infringement only. It's just a pretty toothless way to intimidate. If they were actually going to do something then they would use notice and notice to serve you a summons to court. In that case it's either small claims, where you don't need a lawyer and the judge will help you if the other side uses one, or it's full court where the loser pays the all the winner's costs. So as long as you're on the right side of the law, it actually doesn't cost anything if you get sued (no filing fees as the defendant).
Whether or not you're actually liable in this situation is going to have to be answered by a real lawyer.
reddit.com/r/vainglorygame
If that's the aim, imho it's really difficult. Assuming your neighbourhood isn't made up 99% nerds&geeks, I'd assume most private users are pretty happy if they can watch movies in 4k, download their games on Steam and/or update their gaming console "fast enough". Downloading 50GB already takes only ~30m on my 400MBit/s, and wouldn't feel much different to 1Gb/s. Latency for gaming/video conferencing also isn't an issue these days. Maybe for big families or shared housing (student dorm) this would make sense.
If you wanted to sell me, you'd get me with (1.) reliable and (2.) affordable. Maybe a no-BS pricing scheme instead of "25 US$/month the 24 months and after that 70US$/month" (which virtually every ISP in Germany does, which means I need to waste my time with stupid ISP hopping every other year if I don't want to feel like being ripped off). I just know that - for my usage - there is no practical difference between 1Gb/s and 10Gb/s. At least none that's worth paying more. Sorry, not that useful, but maybe some valuable feedback? :)
On the business end, this is much more interesting. No need to worry about hitting bandwidth limits when video conferencing with customers is pretty nice (we sometimes have that issue and work around it with multiple connections). Also a perk: Faster access to cloud-stored data, or the ability to put data on-premise and still allowing sufficiently fast external access. Of course it must be reliable. But then, your day job is networking, I think you know that kind of business better than I do :)
//edit: Ooooh, post a "Ask HN" and bask in the envy of the internet people =) /s No, seriously, I think promoting offerings like this is a good thing to do, and asking for advice on HN is a good idea. I'd love to see (or maybe even run) a local, no-BS ISP in our municipality.
5G wireless everywhere approaching does not feel practical to me. I do not subscribe to conspiracies about direct health detriments (indirect more concerning - people lusting for 4k videos of everything while sitting in a bathroom stall, or in a public place alone instead of natural resolution of the eyes gazing upon real life).
Perhaps high resolution IoT data upload could occupy your infinite bandwidth somehow? This would only be local.
Red flag to me seems to be constant upgrading of "data throughput". Surely data transmission is good, but at some point is it a waste of effort when we can all only take in so much? Youtube audio and 480p video satisfies my curosity...
Anything for specific providers like FB, Youtube, Netflix, Prime,Play Store, Apple , Disney + etc or general CDN providers like CloudFlare, MaxCDN etc can change end user experience enough for them to buy in.
It also has the added benefit of saving municipality upstream peering requirements and perhaps money.
Alternatively you can provide application services like OwnCloud, Jitsi for equivalent cloud products, it will be cheaper/faster etc. and target schools/community services, it may help them save on their IT bill as well.
Final idea discord, game servers, streaming servers etc can generate network effect driven interest. i.e. If i want to play with my school friends and few of them are on your servers on the municipal broadband already it is strong incentive for me to get one.
You might be able to self host an Apple cache.
You would be surprised on how small some PoPs around the world are. Providers usually come with their own boxes and have specific requirements on space/power/legal ownership etc.
CDN operators and OTT providers have fairly high traffic requirements before they will qualify you for an on-net appliance.
Netflix, for example, requires a minimum of 5 Gbps of traffic until they will even talk to you.
Netflix does have a program where you can submit to get them to ship you a CDN box. I might submit that if I do this, but I doubt they want those in a residence, even if it is 10gig connected. Maybe for a year or two until the municipal provider gets their "data center" space up.
A more realistic plan is to have the muni ISP peer with Netflix at the closest IX and take it from there after traffic levels rise over 5 Gbps.
While they don't have a data center, I presume they do have a place they could put a Netflix box. But that was the closest I know of for running a CDN node. As I said, seems unlikely...
Not that you are particularly hard up for connectivity. Fort Collins is on the old Level3 network so you should have no problem getting transport, waves or dark fiber to the Denver IX. Or you could just buy IP transit at the local PoP.
There are also guard nodes and intermediary nodes, which don't cause third party traffic to exit from your house.
> I run production Linux boxes and networks as my day job. But I don't have machine space at my house, so I'm limited in the number of servers I can put in place.
A single virtual machine host can saturate 10G and allows an arbitrary number of virtual machines, limited primarily by memory and storage. Linux KVM is good enough for Google and AWS and it's free. Get some Socket G34 or LGA2011 workstation with 8+ memory slots, fill it with cheap registered DDR3 and you'll be all set.
But realistically, the biggest benefit of 10G to ordinary people isn't being able to transfer 2500TB a month, it's being able to transfer 25GB in 25 seconds instead of hours.
There are people who upload 5GB to YouTube every day. With garbage upload speeds like 10Mbps, that takes more than an hour and a half. With 10Gbps, not only would it only take five seconds, you don't even need to upload it to YouTube because you can host it yourself.
One of the main value props for YouTube has to be discoverability right? Depends on why you're uploading I suppose.
If more of them were self-hosted then the default search method would be one that finds them everywhere. At this point there is more work to be done than only getting people more bandwidth in order to get there from here, but one step at a time.
Having a Linux mirror within 100 miles has been handy for me in the past though, so if you can spare the storage that would be a good idea.
What I really like is that everyone in the house can go crazy and it doesn't affect me working.
Game servers Voice servers IRC Mirrors for software or datasets Custom download to usb service for those in rural areas.
https://en.wikipedia.org/wiki/USB_dead_drop
Most people want conferencing that doesn't suck, so that they can Zoom/FaceTime/LTE-over-WiFi/Landline-over-VoIP without having Netflix interrupt their call. But I have that solved on a 50mbit down / 5mbit up connection by using an anti-bufferbloat NAT shaper (Eero SQM, or IQrouter, or homebuilt can all do this).
So if you said "this fiber internet ensures that your calls will stop being interrupted by netflix", that might be a great selling point for people who've experienced this problem on their current Internet, without having to discuss bufferbloat in any significant detail.
(Note that you can still have fairness issues with the wireless router, independent of bufferbloat upstream of it — the above-listed devices apply their fairness stuff not only to the WAN uplink but also to the WiFi downlinks — so 10gbit fiber isn't a complete solution, but it's certainly 95% of the way there.)
If the answer is "yes, we introduce significant latency to artificially boost throughput when network capacity is exceeded in either direction", then their 1mbps service is no better than Comcast and is not attractive as a solution to the latency issues of classical providers, and I may as well stick with my 50/5 shaped connection instead of their 1000/1000 connection.
To quote: Bufferbloat is the undesirable latency that comes from a router or other network equipment buffering too much data. It is a huge drag on Internet performance created, ironically, by previous attempts to make it work better. The one-sentence summary is “Bloated buffers lead to network-crippling latency spikes.”
Consider adding torrents.
You could also check out running a folding@home node. Though that seems to require a fairly beefy server.
https://www.youtube.com/watch?v=HaMjPs66cTs