On a few forum sites I'm on, people are just giving up. Looking forward to the post-mortem on how they weren't ready for this (with just a tiny bit of schadenfreude because they've interviewed and rejected me twice).
It’s not everyone. Works fine for me though I did have to reload the page when I skipped past the woman match to the Barrios Ramos fight and it was stuck buffering at 99%.
AB84 streamed it live from a box at the arena to ~5M viewers on Twitter. I was watching it on Netflix, I didn't have any problems, but I also put his live stream up for the hell of it. He didn't have any issues that I saw.
Comments on forums do not provide that data. And if you want to extrapolate self-reports, it's obviously fine (to varying degrees) for the vast majority of people, but that's not the "issue."
These kind of reports are the equivalent of saying "I have power" when you're hundreds of miles away from where a hurricane landed. It's uninteresting, it's likely you have power, and it does literally nothing for people that do not have power.
It doesn't advance the topic anywhere. There are other places to report these issues directly and in aggregate with other people -- HN (and many other forums) are not that place.
You butted into a conversation to tell someone their contribution added no value without adding anything constructive. A comment of “your comment is useless” is pure aggression and is ironically even less useful than the one it’s deriding.
> Tone however, matters in any context.
You are getting upset because someone used a swear word. You’ll find that is just deep seated classism and working on that will let you have much more fulfilling interactions.
Tone policing never works. It’s a waste of calories and everyone’s time.
For the same reason that pointing out the sun rose in the east today would be ridiculous but if it happened to rise in the west, or you perceived it to rise in the west, that would be worth sharing.
Being able to livestream a sporting event is the default now and has been for at least over a decade since HBO’s original effort to stream a Game of Thrones season opener failed because of the MSFT guy they hired, and they fixed it by handing it over to MLBAM.
Maybe that’s what Netflix should do. Buy Disney so they can get access to the MLBAM people (now Disney Streaming because they bought it from MLB).
To be fair, a lot of people pay their ISP for a modem/router combo and connect to something like "Xfinity" at their house. So to them, there is no difference.
I wonder if there will be any long term reputational repercussions for Netflix because of this. Amongst SWEs, Netflix is known for hiring the best people and their streaming service normally seems very solid. Other streaming services have definitely caught up a bit and are much more reliable then in the early days, but my impression still has always been that Netflix is a step above the rest technically.
This sure doesn't help with that impression, and it hasn't just been a momentary glitch but hours of instability. And the Netflix status page saying "Netflix is up! We are not currently experiencing an interruption to our streaming service." doesn't help either...
So the issue is that Netflix gets its performance from colocating caches of movies in ISP datacenters, and a live broadcast doesn't work with that. It's not just about the sheer numbers of viewers, it's that a live model totally undermines their entire infrastructure advantage.
Damn that sucks. I wonder if they could have intentionally streamed it 5 min late? I don’t have all the context around the fight though — maybe a competing service would win if Netflix intentionally induced delay?
they were introducing 5 minute delays on some of the clients. I noticed my ipad was always live and the smart tv had a 5 minute delay but you could fast forward to live.
Correct, this is not Netflix’ regular cup of tea, and it’s a very different problem to solve. They can probably use their edge caches, but it’s challenging.
Not sure how Netflix does it. But this is not very time sensitive, and I would have delayed the stream by 15 to 30 seconds to cache it and then deliver to everyone.
My wild assed guess is the differences in the edge nodes.
Netflix's edge nodes are optimized for streaming already encoded videos to end users. They have to transcode some number of formats from the source and send them all to the edge nodes to flow out. It's harder to manage a ton of different streams flowing out to the edge nodes cleanly.
I would guess YouTube, being built on google's infrastructure , has powerful enough edge nodes that they stream one video stream to each edge location and the edges transcode for the clients. Only one stream from source to edge to worry about and is much simpler to support and reason about.
> I would guess YouTube, being built on google's infrastructure , has powerful enough edge nodes that they stream one video stream to each edge location and the edges transcode for the clients.
Ha, no, our edge nodes don't have anywhere near enough spare CPU to do transcoding on the fly.
We have our own issues with livestreaming, but our system's developed differently over the past 15 years compared to Netflix's. While they've historically focused on intelligent pre-placement of data (which of course doesn't work for livestreaming), such an approach was never feasible for YT with the sheer size of our catalog (thanks to user-generated content).
Netflix is still new to the space, and there isn't a good substitute for real-world experience for understanding how your systems behave under wildly different traffic patterns. Give them some time.
It also helps that youtube serves shit tier quality videos more gracefully. Everyone is used to the step down to pixel-world on youtube to the point where they don’t complain much.
And decent part of these users are on free tier, so they are not paying for it. That alone gives you some level of forgiveness. At least I am not paying anything for this experience.
Live streams have different buffering logic to video on demand. Customers watching sports will get very upset if there is a long buffer, but for a VOD playback you don't care how big the buffer is. Segment sizes are short for live and long for VOD because you need to adapt faster and keep buffers small for Live, but longer download segments are better for buffering.
In my experience even YouTubeTV has problems sometimes. I'll have the 1080p (and enhanced mode also I think) quality set and still deal with a lot of compression artifacts.
Or, hear me out here, it's a wild concept, just work.
You know, like every other broadcaster, streaming platform, and company that does live content has been able to do.
Acting like this is a novel, hard problem that needs to be solved and we need to "upsell" it in tiers because Netflix is incompetent and live broadcasting hasn't been around for 80+ years is so fucking stupid.
This is kind of silly because the delay between actual event happening to showing up on OTA TV or cable TV to showing up on satellite TV can already be tens of seconds.
Yep. Having actually worked on this sort of stuff I can confirm.
Your ISP doesn't have enough bandwidth to the Internet (generally speaking) for all users to get their feed directly from a central location. And that central location doesn't have enough bandwidth to serve all users even if the ISP could. That said, the delay can be pretty small, e.g. the first user to hit the cache goes upstream, the others basically get the stream as it comes in to the cache. This doesn't make things worse, it makes them better.
I don't bet so I have no clue, but why is that? Are people able to place bets in the middle of the match or something? I would have assumed bets get locked in when the fight starts
Not sure I fully buy that. The “live” stream is rarely “live”. It’s often a highly cached buffer that’s a few mins from latest. Those in isp caches can still help here.
That model still works for streaming. You have a central source stream only to the distributed edge locations, then have clients only stream from their local edge location. Even if one region is overwhelmed, the rest can still work. Load on the central source is bounded.
I don't think that live doesn't work with caches. No one watching live would care about a O(s) delay, which is highly amenable to caching at ISPs and streaming changes from there to downstream clients. Offhand I'd say that would support O(ms) delay but no less.
Likely these devices use different media formats and/or quality levels. And yes, it's possible one device buffers more than the other. Infinite freezes sounds like some routing issues or bugs.
When I was watching the behavior on the tv, was wondering if buffering sends some separate, non-business-as-usual requests, and that part of Netflix's delivery architecture was being overloaded.
E.g. "give me this previous chunk" vs "send me the current stream"
Buffering typically just consumes the same live stream until there's enough in the buffer. No difference other than the request rate being potentially higher. At least I can confidently say that for the standard players/video platforms. NetFlix could be doing something different. I'm not sure if they have their own protocols. But I'd be very surprised if the buffering used a completely different mechanism.
Yeah, the funny part is that Hulu, Amazon Prime, and Peacock have all demonstrated the ability to handle an event of this caliber with no issue. Netflix now may never get another opportunity like this again.
I don't spend much time streaming, but I got a glimpse of the Amazon Prime catalog yesterday, and was surprised at how many titles on the front page were movies I'd actually watch. Reminded me of Netflix a dozen years ago.
There's also the "FreeVee" items, which have ads regardless of whether you're a prime subscriber or not. And it feels like a lot of their catalog has been transferred over to FreeVee.
Prime Video has to be the worst of all major streaming services. The video quality is horrible, its crippled with ads (3 not skippable ads for an episode of 45 minutes, lastly), and a lot of interesting titles are behind a "partner paywall".
I was watching the rings of power and it started with a "Commercial free experience provided by so and so" with a long ad at the start of the episode, and then a third of the way into the episode, at a critical action part, it broke in the middle of the actor's sentence to a 6 minute ad block.
I exited playback and haven't gone back to finish it. I'll wait for it eventually to make it to a Blu-ray release someday.
I have prime and my shopping experience is crippled with ads too.
I think it got worse for sellers recently too. If I search for something, like a specific item using its description, sometimes the only result for it shows "sponsored".
It used to show up as sponsored and also unsponsored below.
If this changed, I assume it is bad for the seller. Either they pay for all search results, or their metrics are skewed because all searches were helped by "sponsorship" (and there are no longer unsponsored hits)
It's been pretty rough the last few years. So many great films and series, not to mention kids programming, removed to make way for mediocre NeTfLiX oRiGiNaLs and Bollywood trash.
Amazon Prime isn't so great. Lots of for rent/purchase content or content with ads these days. And they end up repeating slots of content in all the rows in their UI, so I end up seeing the same suggestions everywhere rather than much that's new (other than first party productions).
To me they're basically padding their front page.
But honestly that's most of the major streaming platforms these days. I recently cancelled Disney Plus for similar reasons. The only reasons I don't cancel prime or Netflix are because I have family members I split the memberships with to share.
I recently found a lil dvd rental place in my city. It’s a non-profit, they also do archivals and stuff.
It’s pretty much a two-story townhouse packed head to toe with DVDs (lots of blu rays!)
You don’t realize how limited the streaming collection is until you’re back in a movie store, looking through thousands and thousands of movies you would never find otherwise.
Since I found it, I’ve started doing movie night every week with my friends. It’s such an absolute blast. We go to the store, each pick out a random movie that looks good (or bad, in a good way) or just different.
That's an excellent option. I think it'd be remiss not to mention local libraries. Of course, your mileage may vary, but the ones I've gone to do seem to have adequate selections. I just don't often make time to go there and browse like I would have at traditional video rental places back in the day.
Heck, mine even have some video games; though from when I've checked they're usually pretty back-reserved.
I was in high school in the early 00s, and going to the movies was such a big part of my life. Now, I never even know what's out.
I suspect life stage is a factor, but it does feel like there are many classes of entertainment (cinema and standup come to mind) that don't resonate like they used to.
Back in the day everyone was watching the same thing.
The choices for entertainment were limited to whatever was showing in movie theatres, whatever was on TV and whatever record stores were selling.
I've given Netflix a lot more money than I've gotten value out of. I've had an account for ~15y and only really use it for airplanes unless there's a specific thing I'm excited to watch.
I'm in the same boat where as soon as they make it too hard to share, I'll probably cancel it. I think the main reason their sharing crackdown hasn't been a problem so far is that I use it so seldomly, it thinks the "main" address is my parents, which makes it easy for me to pass the "are you traveling" 2FA on my own phone when I do want to watch something.
> And they end up repeating slots of content in all the rows in their UI, so I end up seeing the same suggestions everywhere rather than much that's new
All of the streaming services do this and I hate it. Netflix is the worst of the bunch, in my experience. I already scrolled past a movie, I don't want to watch it, don't show it to me six more times.
Imagine walking through a Blockbuster where every aisle was the same movies over and over again.
Is it really that big a deal if you are watching a few minutes behind?
I've watched ball games on streaming networks where I can also hear a local radio broadcast, and the stream is always delayed compared to the radio, sometimes by quite a lot. But you'd never know it if you were just watching the stream.
I remember a few years ago reading about a scam at the Australian Open Tennis where there were people inside the stadium who were betting on individual points as they happened.
I guess they could bet before the betting streams caught up.
It seems ridiculous to me that you can bet on individual points, but here we are.
Internet streams are not real-time even in the best case. There is always a few seconds of delay, often quite a bit more than that depending on number of hops and link speeds, congestion, etc.
The issue is that most people are trying to watch live which is what it's advertised as. And until they figure out that they need to watch X minutes behind, it is unwatchable. Many will not figure that out.
So for the first hour it was just total frustration until I stopped trying to go back to live mode.
Not the same demographic but their last large attempt at live was through a Love is blind reunion. It was the same thing, millions of people logging in, epic failure, nothing worked.
They never tried to do a live reunion again. I suppose they should have to get the experience. Because they are hitting the same problems with a much bigger stake event.
yup wanted to say that live stream stuttering has happened before on Netflix - I don't think the reputation is deserved.
From a livestreaming standpoint, netflix is 0/x - for many large events such as love is blind, etc.
From a livestreaming standpoint, look to broadcast news, sports / Olympics broadcasters, etc and you'll see technology, equipment, bandwidth, planning, and professionalism at 1000x of netflix.
Heck, for publicly traded quarterly earnings livestream meetings, they book direct satellite time in addition to fiber to make sure they don't rely only on terrestrial networks which can fail. From a business standpoint, failure during a quarterly meeting stream can mean the destruction of a company (by making shareholders mad that they can't see and vote during the meeting making them push for internal change) - so the stakes are much higher than live entertainment streaming.
Netflix is good at many things, livestreaming is not one of those things.
for livestreams, individual events like the Olympics probably has a surge audience of 10x of netflix events.
Netflix events is small potatoes compared to other livestream stalwarts.
Imagine having to stream a cricket match internationally to UK / India / Australia with combined audience that crushes the Superbowl or a football match to all of Europe, or even something like livestreaming F1 racing that has multiple magnitudes of audience than a boxing match and also has 10x the number of cameras (at 8K+ resolution) across a large physical staging arena (the size of the track/course) in realtime, in addition to streaming directly from the cockpit of cars that are racing 200mph++.
Livestream focused outfits do this all day, everyday.
Netflix doesn't even come close to scratching the "beginner" level of these kinds of live events.
It's a matter of competencies. We wouldn't expect Netflix to be able to serve burgers like McDonald's does - Livestreaming is a completely different discipline and it's hubris on Netflix's part to assume just because they're good at sending video across the internet they can competently do livestreaming.
the point i’m making is that the netflix live streaming timeline didn’t go
chris rock -> love is blind -> mike tyson
they have had other, successful executions in between. the comment i was replying to had cherry picked failures and i’m trying to git rebase them onto main.
I don't think Netflix is even designed to handle very extreme multi-region live-streaming at scale as evidenced in this event with hundreds of millions simultaneously watching.
YouTube, Twitch, Amazon Prime, Hulu, etc have all demonstrated to stream simultaneously to hundreds of millions live without any issues. This was Netflix's chance to do this and they have largely failed at this.
There are no excuses or juniors to blame this time. Quite the inexperience from the 'senior' engineers at Netflix not being able to handle the scale of live-streaming which they may lose contracts for this given the downtime across the world over this high impact event.
Very embarrassing for a multi-billion dollar publicly traded company.
The assumption that it was related to insufficient investment isn’t supported by any evidence. Flawed technical decisions can be made by the most expensive engineers too.
Other potential and future entertainment partners Netflix will be working with e.g. WWE, will certainly see my view as they will be questioning Netflix's capability after that major streaming issue we both saw.
This isn't Netflix's first time they had this live streaming problem.
People will see this as an underinvestment from Netflix's part and they will reconsider going to a different streaming partner.
Most third-party internet-based streaming solutions are overlaid on top of a point-to-point network, while broadcast is one-to-many, and even cable tends to use multicast within the cable provider's network.
You have potentially different problems, e.g. limited bandwidth / spectrum. If, say, there are multiple games going on at the same time, you can only watch whichever feed the broadcaster decides to air. And, of course, regardless of the technology in use, there are matters of acquiring rights for various events. One benefit of internet-based streaming is that one service can acquire the rights and be able to reach everyone, whereas an individual cable provider might only reach its direct subscribers.
On cable(terrestrial is entirely different) even the bandwidth or spectrum is less limiting for broadcasting multiple games. Hard thing is the other parts of production, like cameras, live-directing and live commentary. Adding new channels is less challenging than actual producing content at expected level there.
Not really a joke, though? VOD has obvious methods to cheat a bit. Redundancy abounds and you can even network shape for costs. Could probably get even better compression for clear reasons.
Live, not so much. One source that you have to fanout from and absolutely no way to get cheap redundancy. Right?
Static files have been pretty much the standard streaming protocols for both VOD and live for the last 15 years ago. Before, it was Adobe Flash (RTMP).
With the way that they are designed, you can even use a regular CDN.
You can push these files to all the edges before you release the content which will protect your origin. Livestream all your edge servers are grabbing content from the origin unless you have another tier of regional servers to alleviate load.
Sure but that’s why your edge servers do request collapsing. And there are full blown CDN companies that will write an enterprise contract with you that can do this stuff with ease. Akamai is like 25 years old now.
Scale has increased but the techniques were figured out 20 years ago. There is not much left to invent in this space at the current moment so screwing up more than once is a bit unacceptable.
If Netflix still interviews on hacker rank puzzles I think this should be a wake up call. Interviewing on irrelevant logic puzzles is no match for systems engineering.
For systems design and engineering, absolutely this. I expected the very highest standards and upmost uptime from Netflix, similar to Google and Amazon.
Tells you the uselessness of their engineering blogs.
I did a round of netflix interviews, didn't get an offer (but passed the technical coding rounds) they absolutely had the best interview process of any company I've interviewed at my entire career.
They do make you code but the questions were
1. Not on hacker rank or leetcode
2. Pratical coding questions that didn't require anything more than basic hashmaps/lists/loops/recursion if you want. Some string parsing, etc.
They were still hard, you had to code a fast, but no tricky algorithms required. It also felt very collaborative, it felt like you were driving pair programming. Highly recommended even though didn't get an offer!
I don't think it'll be long-term. Most people will forget about this really quickly. It's not like there will be many people saying "Oh, you don't want to sign up for Netflix, the Tyson fight wasn't well streamed" in even 6 months nevermind 10 years.
Two games actually, both on Christmas Day. A day when most people are at home or the home of family or friends, and they are both pretty good late-season matchups (Chiefs-Steelers and Ravens-Texans) so I imagine viewership will be high.
If they botch the NFL games, it will surely hurt their reputation.
From what I've heard, Netflix has really diluted the culture that people know of from the Patty McCord days.
In particular, they have been revising their compensation structure to issue RSUs, add in a bunch of annoying review process, add in a bunch of leveling and titles, begin hiring down market (e.g. non-sr employees), etc.
In addition to doing this, shuffling headcount, budgets, and title quotas around has in general made the company a lot more bureaucratic.
I think, as streaming matured as a solution space, this (what is equivalent to cost-cutting) was inevitable.
If Netflix was running the same team/culture as it was 10 years ago, I'd like to say that they would have been able to pull of streaming.
Combination of 2 and 3. The business changed. Streaming was more or less a solved problem for Netflix. They needed money for content, not expensive engineers. Ted is co-ceo… you can see where the priority is.
Has Netflix ever live streamed something before? People on reddit are reporting that if you back up the play marker by about 3 minutes the lag goes away. They've got a handle on streaming things when they have a day in advance to encode it into different formats and push it to regional CDNs. But I can't recall them ever live streaming something. Definitely nothing this hyped.
I think Netflix will have even more sw engineers looking to work there once they notice even for average quality of work they can get paid 3 times more than their current pay.
Yea, it’s a bad look. But I switched to watching some other Netflix video and it seemed fine. Just this event had some early issues. Looks fine now though.
> ut my impression still has always been that Netflix is a step above the rest technically.
I always assumed youtube was top dog for performance and stability. I can’t remember the last time I had issues with them and don’t they handle basically more traffic than any other video service?
Maybe a client issue, but i've got a low-end smart tv which handles netflix fine, but youtube is unwatchable due to buffering and failed cuts to adverts
Streamed glitch free for me both on my phone and Xbox. The fight wasn’t so great though, but still a fun event. Jake Paul is a money machine right now.
Based on this I'm wondering whether it was straight up they did not expect it to be this popular?
> Some Cricket graphs of our #Netflix cache for the #PaulVsTyson fight. It has a 40 Gbps connection and it held steady almost 100% saturated the entire time.
I think why I will remember about this fight is not the (small) streaming issue I encountered as much as the poor quality of the fight itself. For me that was the reputational loss. Netflix was touting “NFL is coming to Netflix”. This fight did not really make me want to watch that.
I don't care about boxing or UFC or the grade-A douchebags that are the Paul brothers, but I tuned in just because I had the time and a Netflix subscription.
It was actually great that the fight itself was so boring because it justifies never having to spend time / money on that kind of bullshit. It was a farce. A very bright, loud, sparkly, and expensive (for some people) farce.
The value I got from it was the knowledge that missing out on that kind of thing isn't really missing out on anything at all.
If places like Paramount+ can figure it out, Netflix, given their 10+ year head start on streaming and strong engineering culture, should also have been able to. And if you don't like my example, literally every other streaming service has streamed live sports without issue. YT TV, Hulu, Paramount+, Amazon Prime, Peacock, even Apple TV streams live sports.
It may be "new" to them, but they should have been ready.
I won’t argue that they shouldn’t have done better, I’m only pointing out that this is fairly different from their usual product. Amazon, YouTube, and Hulu all have a ton of experience with live streaming by now. Apple has live streamed wwdc for several years.
I did expect that Netflix would have appropriately accounted for demand and scale, though, especially given the hype for this particular event.
Most people pay Netflix to watch movies and tv shows, not sports. If I hadn't checked Hacker News today, I wouldn't even know they streamed sports, let alone that they had issues with it. Even now that I do, it doesn't affect how I see their core offering, which is their library of on-demand content.
Netflix's infrastructure is clearly built for static content, not live events, so it's no shock they aren't as polished in this area. Streaming anything live over the internet is a tough technical challenge compared to traditional cable.
From my experience, it works if your not watching it 'live'. But the moment I put my devices to 'live' it perma-breaks. 504 gateway timed out in web developer tools hitting my local CDN. probably works on some CDNs, doesnt on others. Probably works if your not 'live'
edit: literally a nginx gateway timed out screen if you view the response from the cdn... wow
Why do they want to get into the live business? It doesn't seem to synergize with their infrastructure. Sending the same stream in real time to numerous people just isn't the same task as letting people stream optimized artifacts that are prepositioned at the edge of the network.
The marginal cost to add a viewer to broadcast sports is zero! That's what I am getting at. I know why someone would want this business, I just don't see what aspect of Netflix's existing business made them think they'd be good at it.
Most PPV is what, $50-$70? So subscribing to Netflix for $20 or whatever per month sounds like a bargain for anyone who is interested and not already a customer. Then assume some large percentage doesn’t cancel either because they forgot, or because they started watching a show and then decided to keep paying.
Live is the only thing that won’t be commodified entirely. “Anyone” can pump out stream-when-you-want TV shows. Live events are generally exclusive, unpredictable, and cultural moments .
All these engineering blog posts, distributed systems and these complex micro-services clearly didn't help with this issue.
Netflix is clearly not designed nor prepared for scalable multi-region live-streaming, no matter the amount of 'senior' engineers they throw at the problem.
I did some VPN hopping and connecting to an endpoint in Dallas has allowed me to start watching again. Not live though, that throws me back into buffering hell.
Yes, it was utterly boring, but they made their money. I don't like either Paul brother, so I only watched in hopes shorter, much-older Tyson would make Jake look as foolish as he is.
I clicked on this thread to type that exact thing, holy smokes.
You're referring to Hooli's streaming of UFC fight that goes awry and Gavin Belson totally loses it, lol. Great scene and totally relevant to what's happening with Netflix rn.
Weird that an organization like Netflix is having problems with this considering their depth of both experience and pockets. I wonder if they didn't expect the number of people who were interested in finding out what the pay-per-view experience is like without spending any extra money. Still, I suppose we can all be thankful Netflix is getting to cut their live event teeth on "alleged rapist vs convicted rapist" instead of something more important.
I know nothing about boxing and this fight was just ridiculously impressive. I kept tuning out of the earlier fights. They felt like some sort of filler. I didn’t get the allure. But Taylor v Serrano was just obvious talent that even I could appreciate it.
Well said. People tuning in late missed out on the real main event. I have a feeling this Tyson fight will be a waste of staying up late and battling Netflix over.
A friend and I, in separate states, found that it wouldn’t stream from TVs, Roku, etc. but would stream from mobile. And for me, using a mobile hotspot to a laptop; though that implies checking IP address range instead of just user-agent, so that seems unlikely.
Anyway, I wouldn’t be surprised if they were prioritizing mobile traffic because it’s more forgiving of shitty bitrate.
985 comments
[ 3.1 ms ] story [ 139 ms ] threadHe’s definitely got issues..
These kind of reports are the equivalent of saying "I have power" when you're hundreds of miles away from where a hurricane landed. It's uninteresting, it's likely you have power, and it does literally nothing for people that do not have power.
It doesn't advance the topic anywhere. There are other places to report these issues directly and in aggregate with other people -- HN (and many other forums) are not that place.
I hope you find it within yourself to treat strangers nicer.
You butted into a conversation to tell someone their contribution added no value without adding anything constructive. A comment of “your comment is useless” is pure aggression and is ironically even less useful than the one it’s deriding.
> Tone however, matters in any context.
You are getting upset because someone used a swear word. You’ll find that is just deep seated classism and working on that will let you have much more fulfilling interactions.
Tone policing never works. It’s a waste of calories and everyone’s time.
It objectively does not add anything. And there is no "butting in", public comments get public responses.
You are being extremely dishonest in your commentary here.
Again, I hope you find it within yourself to be better.
Being able to livestream a sporting event is the default now and has been for at least over a decade since HBO’s original effort to stream a Game of Thrones season opener failed because of the MSFT guy they hired, and they fixed it by handing it over to MLBAM.
Maybe that’s what Netflix should do. Buy Disney so they can get access to the MLBAM people (now Disney Streaming because they bought it from MLB).
Anyway, network cable is the only way to go!
This sure doesn't help with that impression, and it hasn't just been a momentary glitch but hours of instability. And the Netflix status page saying "Netflix is up! We are not currently experiencing an interruption to our streaming service." doesn't help either...
See: https://openconnect.netflix.com/en/
Netflix's edge nodes are optimized for streaming already encoded videos to end users. They have to transcode some number of formats from the source and send them all to the edge nodes to flow out. It's harder to manage a ton of different streams flowing out to the edge nodes cleanly.
I would guess YouTube, being built on google's infrastructure , has powerful enough edge nodes that they stream one video stream to each edge location and the edges transcode for the clients. Only one stream from source to edge to worry about and is much simpler to support and reason about.
But that's just my wild assed guess.
Ha, no, our edge nodes don't have anywhere near enough spare CPU to do transcoding on the fly.
We have our own issues with livestreaming, but our system's developed differently over the past 15 years compared to Netflix's. While they've historically focused on intelligent pre-placement of data (which of course doesn't work for livestreaming), such an approach was never feasible for YT with the sheer size of our catalog (thanks to user-generated content).
Netflix is still new to the space, and there isn't a good substitute for real-world experience for understanding how your systems behave under wildly different traffic patterns. Give them some time.
I get maybe 1m total of buffering per week, if that.
Seems uncharitable to complain about that.
I think this could be one of upsells that Netflix could use.
Premium: get no delay
Normal users: get cache and delay
You know, like every other broadcaster, streaming platform, and company that does live content has been able to do.
Acting like this is a novel, hard problem that needs to be solved and we need to "upsell" it in tiers because Netflix is incompetent and live broadcasting hasn't been around for 80+ years is so fucking stupid.
Your ISP doesn't have enough bandwidth to the Internet (generally speaking) for all users to get their feed directly from a central location. And that central location doesn't have enough bandwidth to serve all users even if the ISP could. That said, the delay can be pretty small, e.g. the first user to hit the cache goes upstream, the others basically get the stream as it comes in to the cache. This doesn't make things worse, it makes them better.
Even the best latency is dozens of seconds behind live action.
I last saw Tyson at +500 while Jake was around -800 on DraftKings somewhere in the 6th round.
Sample size 1, but...
I saw a ton of buffering and failure on an embedded Netflix app on a TV, including some infinite freezes.
Moved over to laptop, zero buffering.
I assume the web app runs with a lot bigger buffer than whatever is squeezed into the underpowered TV.
E.g. "give me this previous chunk" vs "send me the current stream"
I have a feeling Netflix said 'how hard could this be?' and is finding out right now.
I exited playback and haven't gone back to finish it. I'll wait for it eventually to make it to a Blu-ray release someday.
I think it got worse for sellers recently too. If I search for something, like a specific item using its description, sometimes the only result for it shows "sponsored".
It used to show up as sponsored and also unsponsored below.
If this changed, I assume it is bad for the seller. Either they pay for all search results, or their metrics are skewed because all searches were helped by "sponsorship" (and there are no longer unsponsored hits)
I would put Prime Video at 2nd worst. Absolute worst IME is Paramount+.
Edit: worst for streaming quality
It's been pretty rough the last few years. So many great films and series, not to mention kids programming, removed to make way for mediocre NeTfLiX oRiGiNaLs and Bollywood trash.
The Amazon originals are way better imo. They do the dark pattern crap with paid content, as one would expect from Amazon.
To me they're basically padding their front page.
But honestly that's most of the major streaming platforms these days. I recently cancelled Disney Plus for similar reasons. The only reasons I don't cancel prime or Netflix are because I have family members I split the memberships with to share.
It’s pretty much a two-story townhouse packed head to toe with DVDs (lots of blu rays!)
You don’t realize how limited the streaming collection is until you’re back in a movie store, looking through thousands and thousands of movies you would never find otherwise.
Since I found it, I’ve started doing movie night every week with my friends. It’s such an absolute blast. We go to the store, each pick out a random movie that looks good (or bad, in a good way) or just different.
All of a sudden, I love movies again!!
Heck, mine even have some video games; though from when I've checked they're usually pretty back-reserved.
I suspect life stage is a factor, but it does feel like there are many classes of entertainment (cinema and standup come to mind) that don't resonate like they used to.
I'm in the same boat where as soon as they make it too hard to share, I'll probably cancel it. I think the main reason their sharing crackdown hasn't been a problem so far is that I use it so seldomly, it thinks the "main" address is my parents, which makes it easy for me to pass the "are you traveling" 2FA on my own phone when I do want to watch something.
All of the streaming services do this and I hate it. Netflix is the worst of the bunch, in my experience. I already scrolled past a movie, I don't want to watch it, don't show it to me six more times.
Imagine walking through a Blockbuster where every aisle was the same movies over and over again.
It has been pretty useless. At the moment seems to be working only when running in non-live mode several minutes behind.
So if there are 1 million trying to stream it, that means they would lose $15 million. So.. they might only give a partial refund.
But people should push for an automatic refund instead of a class action.
I've watched ball games on streaming networks where I can also hear a local radio broadcast, and the stream is always delayed compared to the radio, sometimes by quite a lot. But you'd never know it if you were just watching the stream.
i don't bet on sports. but from friends who do: yes, it's a really really big deal.
I guess they could bet before the betting streams caught up.
It seems ridiculous to me that you can bet on individual points, but here we are.
https://en.wikipedia.org/wiki/The_Sting
Offshore combined streaming and betting houses will be cleaning up the rubes.
So for the first hour it was just total frustration until I stopped trying to go back to live mode.
They never tried to do a live reunion again. I suppose they should have to get the experience. Because they are hitting the same problems with a much bigger stake event.
From a livestreaming standpoint, netflix is 0/x - for many large events such as love is blind, etc.
From a livestreaming standpoint, look to broadcast news, sports / Olympics broadcasters, etc and you'll see technology, equipment, bandwidth, planning, and professionalism at 1000x of netflix.
Heck, for publicly traded quarterly earnings livestream meetings, they book direct satellite time in addition to fiber to make sure they don't rely only on terrestrial networks which can fail. From a business standpoint, failure during a quarterly meeting stream can mean the destruction of a company (by making shareholders mad that they can't see and vote during the meeting making them push for internal change) - so the stakes are much higher than live entertainment streaming.
Netflix is good at many things, livestreaming is not one of those things.
Perhaps Netflix still needs a dozen more microservices to get this right...
Netflix events is small potatoes compared to other livestream stalwarts.
Imagine having to stream a cricket match internationally to UK / India / Australia with combined audience that crushes the Superbowl or a football match to all of Europe, or even something like livestreaming F1 racing that has multiple magnitudes of audience than a boxing match and also has 10x the number of cameras (at 8K+ resolution) across a large physical staging arena (the size of the track/course) in realtime, in addition to streaming directly from the cockpit of cars that are racing 200mph++.
Livestream focused outfits do this all day, everyday.
Netflix doesn't even come close to scratching the "beginner" level of these kinds of live events.
It's a matter of competencies. We wouldn't expect Netflix to be able to serve burgers like McDonald's does - Livestreaming is a completely different discipline and it's hubris on Netflix's part to assume just because they're good at sending video across the internet they can competently do livestreaming.
yes, love is blind failed, but was definitely not the most recent attempt. they did some other golf thing too, iirc
chris rock -> love is blind -> mike tyson
they have had other, successful executions in between. the comment i was replying to had cherry picked failures and i’m trying to git rebase them onto main.
But also people were saying they weren't having any issues streaming on Ziply.
[1] https://www.reddit.com/r/ZiplyFiber/comments/1gsenik/netflix...
YouTube, Twitch, Amazon Prime, Hulu, etc have all demonstrated to stream simultaneously to hundreds of millions live without any issues. This was Netflix's chance to do this and they have largely failed at this.
There are no excuses or juniors to blame this time. Quite the inexperience from the 'senior' engineers at Netflix not being able to handle the scale of live-streaming which they may lose contracts for this given the downtime across the world over this high impact event.
Very embarrassing for a multi-billion dollar publicly traded company.
I wouldn't be surprised if lots of engineers at Netflix are currently now writing up a length post mortem of this.
And this is from the company that created the discipline of chaos engineering for resilience.
It is clear they under invested and took the eye of the ball with this.
This is bad, like very very bad.
We will see why it went down and to what extent they underinvested in their post mortem.
This isn't Netflix's first time they had this live streaming problem.
People will see this as an underinvestment from Netflix's part and they will reconsider going to a different streaming partner.
Most third-party internet-based streaming solutions are overlaid on top of a point-to-point network, while broadcast is one-to-many, and even cable tends to use multicast within the cable provider's network.
You have potentially different problems, e.g. limited bandwidth / spectrum. If, say, there are multiple games going on at the same time, you can only watch whichever feed the broadcaster decides to air. And, of course, regardless of the technology in use, there are matters of acquiring rights for various events. One benefit of internet-based streaming is that one service can acquire the rights and be able to reach everyone, whereas an individual cable provider might only reach its direct subscribers.
Live, not so much. One source that you have to fanout from and absolutely no way to get cheap redundancy. Right?
Not trying to downplay their complexity, but last I heard Netflix is splitting the shows in small data chunks and just serves them as static files.
Live streaming is a different beast
With the way that they are designed, you can even use a regular CDN.
Scale has increased but the techniques were figured out 20 years ago. There is not much left to invent in this space at the current moment so screwing up more than once is a bit unacceptable.
Tells you the uselessness of their engineering blogs.
They do make you code but the questions were 1. Not on hacker rank or leetcode 2. Pratical coding questions that didn't require anything more than basic hashmaps/lists/loops/recursion if you want. Some string parsing, etc.
They were still hard, you had to code a fast, but no tricky algorithms required. It also felt very collaborative, it felt like you were driving pair programming. Highly recommended even though didn't get an offer!
If they botch the NFL games, it will surely hurt their reputation.
In particular, they have been revising their compensation structure to issue RSUs, add in a bunch of annoying review process, add in a bunch of leveling and titles, begin hiring down market (e.g. non-sr employees), etc.
In addition to doing this, shuffling headcount, budgets, and title quotas around has in general made the company a lot more bureaucratic.
I think, as streaming matured as a solution space, this (what is equivalent to cost-cutting) was inevitable.
If Netflix was running the same team/culture as it was 10 years ago, I'd like to say that they would have been able to pull of streaming.
Or did they have a lot of needs that they decided didn't require top-skilled people?
Or was this a beancounter thing, of someone deciding that the company was paying more money on staffing than they needed to, without understanding it?
I always assumed youtube was top dog for performance and stability. I can’t remember the last time I had issues with them and don’t they handle basically more traffic than any other video service?
> Some Cricket graphs of our #Netflix cache for the #PaulVsTyson fight. It has a 40 Gbps connection and it held steady almost 100% saturated the entire time.
https://fosstodon.org/@atoponce/113491103342509883
It was actually great that the fight itself was so boring because it justifies never having to spend time / money on that kind of bullshit. It was a farce. A very bright, loud, sparkly, and expensive (for some people) farce.
The value I got from it was the knowledge that missing out on that kind of thing isn't really missing out on anything at all.
They stream plenty of pre recorded video, often collocated. Live streaming seems like something they aren’t yet good at.
It may be "new" to them, but they should have been ready.
I did expect that Netflix would have appropriately accounted for demand and scale, though, especially given the hype for this particular event.
Live sports and VOD movies/TV are very different beasts.
Maybe if we're not counting Youtube as 'streaming', but in my mind no one holds a candle to YT quality in (live)streaming.
In my country every time there's a big football match the people who try to watch it on the internet face issues.
Most people pay Netflix to watch movies and tv shows, not sports. If I hadn't checked Hacker News today, I wouldn't even know they streamed sports, let alone that they had issues with it. Even now that I do, it doesn't affect how I see their core offering, which is their library of on-demand content.
Netflix's infrastructure is clearly built for static content, not live events, so it's no shock they aren't as polished in this area. Streaming anything live over the internet is a tough technical challenge compared to traditional cable.
edit: literally a nginx gateway timed out screen if you view the response from the cdn... wow
Netflix is clearly not designed nor prepared for scalable multi-region live-streaming, no matter the amount of 'senior' engineers they throw at the problem.
It's almost like this platform has been taken over by JavaScripters completely.
That is an incredible way to phrase the sentiment, thank you.
Well, yes. Who would think Netflix was designed for that? They do VOD. They're only trying to move into this now.
You're referring to Hooli's streaming of UFC fight that goes awry and Gavin Belson totally loses it, lol. Great scene and totally relevant to what's happening with Netflix rn.
And you’ll never guess which Presidential candidate they both support!
If anyone was waiting for the main card to tune in, I recommend tuning in now.
Also, no buffering issues on my end. Have to wonder if it's a regional issue.
Anyway, I wouldn’t be surprised if they were prioritizing mobile traffic because it’s more forgiving of shitty bitrate.