If memory serves there were a number of issues with XHTML2, chief among them being it didn't serve the original intent of XHTML very well. HTTP/2 has good meaning but some of the deign choices were made because Google wanted them not because they were necessarily best (at least from my understanding as an interested spectator).
Could you expand on that? Is there a list of these features, or at least a blogpost that interested parties could read to quickly get up to speed with the special interest influences currently observable in the spec?
Giant mistake in the making. HTTP is elegant, HTTP2 is a monstrosity.
Edit: downvoters: please explain what's to like about HTTP2. I have a very hard time finding anything to like.
For example: no more easy debugging on the wire, another TCP like implementation inside the HTTP protocol, tons of binary data rather than text and a whole slew of features that we don't really need but that please some corporate sponsor because their feature made it in. Counter examples appreciated.
Referrer was a nice idea that led to a lot of trouble. Agreed, that wasn't the most brilliant idea but in a world of one-way-links a way to establish the origin of incoming traffic so you could simulate reciprocal links must have seemed like a good idea at the time.
Keep in mind that Hypertext as it was originally envisioned by Ted Nelson had nothing but two way links, doing one-way links broke with the view of the day in a pretty drastic manner (and actually made the whole thing possible), so I think some leeway here is allowed.
Well, in this draft linked here they spell 'server' as 'sever' in one instance so I guess that evens up the score. But the GGP had a point, referrer is a troublesome bit in many ways and he seemed to be aware of the spelling error so I chose to answer the meat of the item rather than to take it in the most simplistic way (after all, making spelling errors like this really does happen, one guy I knew had sandblasted 150 glass doors with a text when I happened to walk by and pointed out that pneumatical has an 'n' in the second position (and not an 'm'...). For some reason he didn't seem all that happy.
The score most certainly not even. HTTP is still way, way behind. There's a qualitative difference between a spelling error in the document describing the protocol and a spelling error in the protocol itself. In my opinion, the substantive point wasn't about the mechanics of referrers, but that the lack of care and attention to detail demonstrated by "Referer" is strong evidence that HTTP is not as elegant as you claim.
I worked a bit for a small newspaper in Amsterdam in the 1980's, called 'the paper about Holland'. It was written by two UK journalists, Abi Daruvalla and Mark Fuller.
We were pretty good at proofreading because spell checkers were non-existent at the time, we barely had a word processor. It would probably surprise you that as a non-native English speaker I would catch typos (not grammar errors) much quicker than the rest of the (native speakers/writers) crew. We figured it was because they didn't actually read the words where I had to. They'd mentally fill in the word that should have been there which allowed them to proofread a piece with three individuals and I'd still catch the mistakes in the spelling.
In spite of that we managed to ship two issues labeled September in one year, and that's with everybody being so focused on the spelling that nobody bothered to check the dates.
I think you're reading way too much in a single letter spelling error (which I spotted the first time I saw the word), the common excuse is that the spell checker used by the person that wrote that part did not include a spelling for 'referrer' (and if it had it would have probably ignored it with the leading capital anyway...).
I've seen some pretty elegant code with terrible spelling errors in the variable names, it's certainly not great to have a spelling error in a text protocol. But I would not use that single letter mistake as a yardstick by which to measure the entire protocol, that seems all too easy a way to discard something that has stood the test of time very well.
Imaginary meat. Sure, you can talk about the implications of the actual header, but you must first acknowledge that this has nothing whatsoever to do with the context of 'elegance'.
Bingo. Also look at the handling of URIs (absolute or relative), encoding issues, and the mess that is the actual text format.
HTTP's popularity outside of the web is mainly cause it's easy to use as a simple wrapper around TCP with read and read/write semantics. Nearly all the rest of HTTP is unneeded fluff (proof: browsers didn't even do other methods besides GET/POST and applications were just fine).
Because any online community where that attitude prevails degenerates into an echo-chamber.
Voting is meant to promote quality contributions and increase the overall quality of the site. A site where dissent is pushed out of site would be very dull.
An upvote should be a sign that the comment-er did a good to great job of explaining a position that may or may not align with your confirmation bias and other thoughts, not a simple "I agree" signal.
A downvote isn't the opposite of an upvote. An upvote can function somewhat like a teacher giving a student a gold star, but for the most part it primarily highlights good content (not necessarily that you agree with the content), above average content. A downvote, however, also functions as a punishment, a demerit, a warning. A message that "We don't want your kind around here", with real consequences (getting banned, etc). The problem is that it lumps inane content (or "graffiti" content) with posts that are an unpopular opinion. I've often seen downvoted comments that still end up generating a lot of interesting discussion, and I'm glad that those comments were made.
But since a downvote is a punishment, often times people will second guess if they should post something interesting just in case it is taken the wrong way by someone. That's happened enough to me that now I very seldom comment on any thread that has any type of controversy in it, just because I know that someone will get upset no matter what. (Maybe a fix for this is to make it free to give upvotes, but downvotes can only be given if you've purchased them with some of your own karma points).
> But since a downvote is a punishment, often times people will second guess if they should post something interesting just in case it is taken the wrong way by someone.
Yes. A thousand times yes. I finally had to create a second "alter ego" account just so I would stop censoring myself.
I have ended up, as expected, getting a small number of downvotes and comment flags from people who simply didn't like what I had to say. Even when what I said was a factual summary backed by strong scientific evidence for which I provided citations (with only minimal opinion sprinkled in).
They have a point. I was way too short in my original comment, a bit of explanation why I thought that way would have been better, too many assumptions on my part.
I'm sure it does. But the overhead associated with SSL connections is something that should be solved at that level, not by shoehorning a complete multiplexing system into an otherwise relatively clean protocol. It seems to me that that violates a whole pile of good design principles.
The major speed increase is not from eliminating ssl handshake (though it helps) but by multiplexing many logical http connections on one tcp stream.
It's solved exactly at the level it had to be solved.
The nitty gritty of why opening a new tcp stream for each http request is slow (and why http pipelining doesn't make it faster the way spdy does) is beyond the scope of this post but https://www.youtube.com/watch?v=-yxQIRl6Qic explains that and other benefits of http/2.0 in great detail.
I think the biggest win for HTTP/2 is multiplexing the requests over a single TCP connection. The average web page downloads over 1MB of data over 80 to 100 Http requests. Each request is requires its own TCP connection.
Modern browsers download 6 assets per host at a time. This decision is so we don't have too many TCP connections competing for the same bandwidth. People optimize the page load times, by combining javascript, combining images with CSS sprites, or adding additional hosts so they can download more of the assets in parallel.
I think that the waterfall chart is an intuitive way to understand how the assets on your website is affecting the page load time. It will be interesting to see what new ways we visualize the process of assets downloading and different browser events, when HTTP/2 starts to be more widely used.
Your statement is only possibly true for a single request. In any real world application, the ability to issue many requests without having to open a new connection each time and to receive results out of order is a massive win over HTTP, even before you consider things like server push allowing resources like stylesheets or scripts to be delivered immediately rather than waiting for the client to finish parsing an HTML document.
Sharding helps with only the connection limit at the expense of extra redundant DNS delays and poor scheduling of the total available network capacity. It cannot help with the need to request resources sequentially and has no equivalent to something like server push avoiding extra round-trips + client delays before even issuing requests for related resources.
>The majority of data has no need for encryption. Some of it does.
No. Wrong. Absolutely, incontrovertibly wrong. Everything that doesn't absolutely need to be plaintext should be encrypted; everything that can be authenticated should be.
My best theory is there is some interested parties (ie Google) that want to use a persistent SPDY/HTTP2 connection host/port as an unavoidable session cookie. Make your services so that people have to log in and continue the same connection in order to keep using the 'app' and you can track them better.
I haven't checked yet but I won't be surprised to find out that Chrome keeps one SPDY/HTTP2 connection open to google-analytics.com that's shared for all domains...
Google doesn't need a way to track you using SPDY, they are present on just about every page already, assuming they don't serve it outright and that you didn't navigate there using their search engine.
That's interesting, because I block analytics and some other Google domains and I use Bing.
Why push SSL so hard for sites many of which are just public info anyway? Because then you can't really use proxies so almost always 1 connection is 1 person.
The HTTP connection timeout is like 15 seconds or less in popular software. One of the justifications for SPDY/HTTP2 is that with one connection it can be kept open for many minutes or even hours without using too many resources.
Instead of many separate, possibly proxied connections to doubleclick you have one connection that lasts maybe even a whole browsing session. Google may not need this, but it's certainly better for them.
Just confirmed that Chromium reuses the same google-analytics connection for two completely separate web sites and it keeps the socket open for 4 minutes. It almost certainly does the same thing for doubleclick.net.
If you spend less than 4 minutes per page then that entire browsing session across all google-analytics pages is uniquely identified to google. Even if you have many people behind a NAT and they have 3rd party cookies disabled, each one's browsing is still a separate clickstream to Google.
Only thing making a person's browsing opaque to google is if they browse behind a proxy with a bunch of other people mixed in, and SPDY/HTTP2 conveniently enough make proxies very difficult by needing to trust the proxy's certificate on each client. So anybody needing a proxy will use HTTP1 and anybody using HTTP2 has their browsing sessions uniquely identified.
PHK simultaneously complains that HTTP/2 doesn't go far enough e.g. replace cookies with a stateful mechanism, while also complaining the header compression and other mechanisms in HTTP/2 introduce the requirement for the server to maintain state.
PHK also refers to HTTP/2 as a political protocol and casts various aspertions about those promoting it but you need to remember that PHK isn't a neutral observer - Varnish does't support TLS yet (though PHK is very clear on why this is the case).
>PHK simultaneously complains that HTTP/2 doesn't go far enough e.g. replace cookies with a stateful mechanism, while also complaining the header compression and other mechanisms in HTTP/2 introduce the requirement for the server to maintain state.
Maybe but changing the state semantics on the client is more work than just changing the protocol stack.
I suspect it will come in time but I'd argue it's better to develop incrementally rather than a big bang approach, and there's plenty in HTTP/2 to allow further improvement.
Not having to use the horrific hack known as CSS spritesheets just to load a lot of images quickly is certainly to like.
I'll miss easy wire debugging too, but it's been obvious for a long time that HTTPS everywhere is the future... and when you go to the effort to use a MitM proxy or whatever for debugging TLS, you may as well throw in a tool that understands the protocol.
Edit: I agree it would be nicer to solve many of the goals of HTTP2 at a lower level, and from the little I've heard about it, that solution already basically exists in the form of QUIC. But server push is a useful feature even with a perfect transport layer, so meh...
Easy wire debugging is a fallacy in a world where Wireshark is a real thing that exists. I can read a binary protocol trivially with the appropriate wireshark filter.
Heck, I can read a binary protocol as hex if I know it well enough - writing a tool which can do this is not a difficult task (it's actually way easier then text parsing!) and gives you more concise answers more quickly.
I can only agree, and further voice my strong disapproval at the continuing, damaging and absurd lack of DNS and IPv6 considerations, most notably the omission of any discussion of endpoint resolution.
Literally so: this protocol document does not specify how you determine which server to connect to. HTTP2 is, in definition, only very loosely coupled to IP despite making significant optimisations for TCP. Thus in implementation we simply get the same old mistakes and undefined behaviours. Issues with floating apex records, hacks based on IPv4/6 race conditions, unnecessary address wastage and so forth will continue; all derived from the colossal architectural wart of overloading the DNS host (A/AAAA) record as a service endpoint discovery mechanism.
Once again, I say unto the peanut gallery: shoulda used SRV. The benefits are many and the downsides greatly overstated. I bemoan the missed opportunity.
What is a "floating apex record"? I googled it in quotations, and you appear to be the only person who has ever said that combination of words in Google's index.
I think "apex record" means the root domain name in a zone, and is unrelated to A records (except for the fact that you would usually make an A record for your apex so web browsers can reach your site even without a "www." subdomain/prefix)
Have you ever considered just saying 'A record' instead of 'Apex record' so the majority of people know what you're talking about? Not all of us are DNS wonks.
An apex record is one at the root of a DNS zone. Sometimes called "naked domains".
For example, in "https://github.com/" they are the records particularly for "github.com", rather than for subdomains that might exist such as "www.github.com" or "gist.github.com".
Apex records have a particular restriction: they cannot be aliases, because the apex includes DNS metadata that is not allowed to be aliased[3]. Read on for how this becomes a problem.
I've used the term "floating" as a visual metaphor, because what I'm about to describe lacks a universal standard name, because it is an ugly hack:
HTTP resolves endpoints using host records, so an URL of "https://github.com" means looking up A and AAAA records for "github.com". Yes, the protocol is arrogant enough[1] to assume that your host address for the whole domain is that of the web server. (This is why we ended up prepending "www" to domain names, as a service selector). In response to the query you get an IP address.
Unfortunately, IP addresses sometimes change without warning. The most common example today is the loadbalancer offered by Amazon Web Services. The solution to this is to use an alias record in your human-friendly domain, pointing at an hidden technical domain that the infrastructure provider keeps up-to-date (e.g. "my-elb-name-1-1160186271.ap-southeast-1.elb.amazonaws.com")
This is fine for "www.example.com" but not the naked "example.com", because aliases are prohibited at the apex.
As a result, DNS providers such as Route 53 have ended up with a hack: a spoofed record at the apex, one that tracks an external resource and synthesizes a fake A/AAAA response. Now you have a naked domain that tracks, or rather hopes to track, the correct endpoint. But it changes with the wind. Hence my description of it as "floating".
There is no consistent name for this kludge. AWS calls it an alias, and for reliability concerns restrict it to their own infrastructure only; DME call it an "ANAME" record [2]. The model can even be readily implemented as a shell script run out of cron on your nameserver. It is fragile, it is often unreliable, it is not at all standardised, and it doesn't scale beyond one service.
One better solution would be to require use of SRV records, which allow one to declare instead, for example, an "https" service for "example.com". Alongside, let's say, the xmpp service, sip service, or any other service you care to announce. SRV records can exist at the apex. They can also bundle the A and AAAA (IPv6) addresses for the resulting endpoints in the answer, and select alternative port numbers without bothering the user about it.
Not quite a universal panacea: there is a minor hazard of zone cuts that could increase the number of client lookups, but that's an edge case, not one you can easily blunder into and also easy to fix.
[1] HTTP/1.0 and earlier are forgiven, because they hail from a time when you just had a web server in a rack and called it "www". But HTTP/2 is supposed to respond to modern architectures.
I can't agree enough about the missed opportunity to use SRV records. This would have been such a monumental step forward.
Edit: It makes me a bit giddy (which makes sense if you factor in my being a sysadmin) to think about what SRV records would've done for load-balancing, running servers on non-standard ports, IP address exhaustion, and server migrations. Anybody who doesn't appreciate proper service-location hasn't ever done serious sysadmin work and, IMO, has no business designing protocols.
I'm a fan of plain text protocols but I think the web has gotten to the point were the adversaries are so intrusive and powerful that it all needs to be encrypted, including the headers.
https would stop injection just the same and is just http-over-ssl/tls. You could also simply sign the the content (but that would still allow wire-tapping).
Both still leave the original text based protocol and would stop ISP header injection.
Another defense against such trickery would be a legal one: make it illegal to tamper with data sent between two peers on the internet.
Imagine the postal service opening your mail and changing a couple of words in your mail just because they could.
And HTTP2 is essentially just a multiplexed wrapper around HTTP1-like semantics, AFAICS.
I'm not an HTTP2 expert, but I would bet that WireShark can make an HTTP2 session look basically the same as a bunch of HTTP1 requests. It's the same "wrapper" concept except it adds multiplexing.
Yeah and once you multiplex, your text based protocol is gonna suck to read anyways. So in practice you'll use Wireshark, and then you no longer care if it's text based.
HTTP/2 won't stop anything. Encryption is not mandatory. Even if it were, browsers are going to be defaulting to HTTP 1.x on port 80 for perhaps a decade. Transparent proxying and downgrade attacks are still totally feasible. Just like every other step in Internet history, user security and privacy has been kicked to the curb.
The text implementation is a mess. I'd be shocked to find out that HTTP as specified is actually compatibly implemented. Moronic things like comments in header values and line wrapping show how far detached the spec authors are from implementation reality.
Text implementations are expensive to parse. You end up doing a lot of nutty hacks (like bitwise operations a word at a time, on strings) just to have good speed. Doing so safely requires a lot of branches. A binary protocol will be easier, more compatibly implemented, safer and far faster to decode. Look at nginx code to see examples (and that's a good codebase).
Debugging, yeah, it's sometimes handy to run tcpdump and see problems. But s/tcpdump/tshark and it's essentially a solved problem. The vast, vast majority of HTTP messages are machine written and read, optimizing for humans is misguided.
And yes, I'm bitter about this, as I'm on my third project implementing a SIP stack. SIP shares HTTP's general format, with tons of added insanity for fun. Like UDP/TCP hopping based on the spec author's misunderstanding of IP reassembly. I've lost weeks of my life due to idiotic spec writers just going off and making stuff up. Cause hey, in a text document, there's no actual technical debt to be paid, no actual real world problems.
Google's approach with SPDY is far better than most of what the IETF has been able to put out.
> Like UDP/TCP hopping based on the spec author's misunderstanding of IP reassembly.
You have any further reading or links for this? SIP is definitely "interesting" (not in a good way) and I'd love to have some background on why it is the way it is.
Search the SIP RFC for MTU. Apparently they thought UDP packets couldn't exceed Ethernet MTUs. Of course UDP support is a dumb idea in the first place and provides no real benefit. It opens up a lot of security holes, since spoofing UDP is trivial, and most of the world's VoIP runs on IP based authentication.
Anyways, because they thought UDP could not exceed Ethernet MTU, they came up with this crazy protocol hopping behavior. So on a per message basis, a SIP server or client can just flip to TCP or UDP, just for fun. This idiocy is what made MS just drop UDP support in their SIP products, because "most of our messages will be over the MTU and compliant implementations will hop to TCP". But popular stacks such as Asterisk had baked UDP assumptions pretty deep.
To make it even more fun, SIP TCP and UDP differ. In UDP, they decided content length isn't needed since you can just use the datagram boundary. So you get this situation where the validity of their text-bsed message depends on which transport it was delivered on.
The sick thing is that the authors of these RFCs take delight in the pointless complexity they've added. Look up SIP torture tests RFC just to see how demented they are.
In their defense, one of them wrote me and said they had started off thinking SIP would be totally HTTP compatible. After they dropped that idea, they left stuff in because it was "too late". While I understand life's tough, this explanation doesn't do much for my confidence in their decisions. Another one wrote and said " hey, C and other programming languages have flexible syntax, why not SIP?". He wasn't joking :/. But hey, one of them is now the CTO of the FCC, so writing terrible specs must be a good career move.
Worse, many people see " text based " and think it's easier than a proper binary protocol. Many shitty coders have pumped out bad SIP implementations that sorta work. But in fact, you cannot unambiguously parse SIP on the Internet today. Popular implementations disagree on basic things such as line endings. This opens up security holes, as you can make one network element read a message one way, knowing the next element will interpret it differently.
It's possible, I suppose. I captured terabytes of SIP on a production VoIP network with around a hundred thousand endpoints from all over. Over 1% of packets had IP fragmentation (~576 byte MTU) and none of them had problems. I did not see any noticeable amount of failed-to-reassemble packets.
And as you point out, the limit is quite low in such cases. So it'd be better to just drop UDP (for many reasons), versus adding protocol hopping and whatnot.
Wow. I suspected SIP was bad, but I never got around to looking at it in detail.
...
Wait, NIST wrote the reference implementation? Also, your description sounds suspiciously like someone was going for maximum confusion and a "design" that practically guarantees bugs in every implementation.
Real life sip is the worst protocol I've seen. There are loads of additions to it that try to fix previous issues. Phone status publishing, nat detection/workarounds, companies coming up with their own extension, etc. etc. Basically forget about things interoperating without proxies that capture and contain/fix all the weirdness. Snom, grandstream and Linksys will not agree on all features natively no matter how you try.
The process for re-invites with proxies in the middle, loose routing, nats, and codec renegotiation is a VoIP engineer's hell.
But all those extensions/rules were created with good intentions.
And they have the nerve to call their IM extension "SIMPLE".
It also shocks me how prevalent SIP is for arbitrary lookups in telcoland. More than one switch uses a SIP call to find number portability info. So they involve at least 3 SIP messages, each hundreds of bytes long, complete with a full SIP transaction state, burning up call resources on their switch, to essentially send and receive 10 bytes.
The amount of stupidity in telecom is mind-boggling. I take solace in hearing that other industries are just as bad, but I don't see them so they just appear to be OK.
Eh, I'm not so sure. TLS is rarely supported, and not really ever used, so it's not like sabotage is needed to make it viewable.
And a lot of this stuff is the result of carrying over design decisions, literally, from the 60s. Someone wrote an RFC codifying what they were doing by hand. And that nonsense gets passed forward for zero reason.
Ever wonder why HTTP uses the idiotic "Thu, 28 Jan, 1998, GMT" format for datetime? Because that format makes sense if you're hand-reading email message headers. And HTTP just copied it forward, zero thinking involved. Probably the same reason you can have comments in headers. SIP gives an example:
Retry-After: 300 (I'm in a meeting)
To real-world engineers, we say, wow, that's idiotic. Comments will never be used, so they'll never be properly implemented and only serve to make things problematic.
To an IETF RFC author, that kind of behaviour is "well RFC xxx does it that way", and ultimately comes to "well, like, if you were sending HTTP messages by fax, extra comments might help".
If you stack up enough hacks like that, then it starts to become the clusterfuck it is. I don't think there's any maliciousness involved. Just lack of critical thinking, or lack of experience actually implementing software. Plus the fact that when writing a spec, you've got a blank paper and can just go all-out with crazy ideas you won't have to support.
Interestingly, I think any magic number that can somehow end up in the hand of developers (or maybe users whom developers then need to support) should always come with a human-readable explanation, or if it's impossible, then that means that should have been a stacktrace somewhere much deeper in the stack. (Because at first it's easy to just look up what those numbers mean, but when 5+ years down the timeline someone has to support something that only barfs up those numbers ... well, I'm sure a lot of sons and moms would be spared.)
Speed and performance "improvements" of the new protocol do not justify all the additional complexity that comes with it for almost everyone in the world. Of course everyone could use some improvements, but they are not the things large corporations benefit from, they are thing that concern people who do not have planet scale infrastructure at their hands. I would like to see changes for easier resilience or even for distributed client/p2p caching for example. Doesn't matter though, we already lost.
Most people don't pay any cost. They just add a few letters to their webserver config and everything gets faster. There's no further complexity required (other than TLS.)
I don't see how SPDY prevents someone coming up with p2p caching. Although a serious use case would be a good start.
My point was: it is not something that everyone needs and yet it is a standard, but something, that everyone could benefit from, is not even on the table.
It also addresses the usual complaints, including the "binary" part.
The short version is that is all about speed and security (by encrypting everything).
As to the issue of binary vs. text. Implementing binary protocols (including tools used for debugging them) is much simpler than implementing text protocol. HTTP1 might look simple to human eyes but it's not an easy protocol to implement (fully and correctly). I know, I tried.
Furthermore, engineering is about tradeoffs. HTTP2 is not simple but the complexity exists to address real problems in failure modes of tcp stack and to maximize speed by both sending as little as possible (compression) and ensuring that one request doesn't block the other, as long as there is bandwidth to send both.
Maybe the purity and simplicity of the protocol is more important to you than improving the speed by few %, but to make it makes perfect sense to trade simplicity for even modest gains in performance in the context of http.
You only write the code once but it'll speed up every http connection until end of time. That's a lot of http connections which adds up to a lot of bandwidth and time savings.
Bandwidth is cheap. I don't buy that argument, but I do see that there are problems with HTTP and the way it is implemented. I think that if you are going to implement something that will replace HTTP that it had better be something that is huge step up from what we have today and in that respect HTTP2 fails to convince me that it is that step.
Keep in mind that we're talking about the protocol that currently powers much of the developed world as we know it and that mistakes will be extremely costly to correct. HTTP has its warts and quirks but for the most part we have worked our way around those.
Minor efficiency gains do not add up for me to a complete overhaul of the web and everything attached to it.
So what we'll get instead is this: a percentage of the web will transition (mostly the bigger players where that few % adds up to a higher bottom line), the rest will not care and wait for HTTP3 or whatever will come after HTTP2 to really address what's wrong with HTTP (and that's not that it isn't fast enough, that's mostly a problem with the underlying transports, HTTP is plenty fast).
Says someone who I'm certain doesn't rely on a prepaid SIM for their primary Internet connection. Not that I do either, but I'm far more aware that tens (if not hundreds) of millions of these people exist today. In fact, it's likely that such people will be a clear majority of Internet users early in the life of HTTP2. Leaving them out of the conversation, as you implicitly do here, would be a colossal mistake for a protocol that intended to stand the test of time.
The difference between HTTP and HTTP2 is not noticeable cost-wise on the consumer side, it is only noticeable in the aggregate when serving millions of requests.
Let's not try to pretend that HTTP2 is about bandwidth savings in the third world, that didn't even enter the consideration at any level.
If you want to save bandwidth at that level then you're better off reducing eye candy and using apps rather than websites and not shipping bloated pages with 100's of requests and 1/2 MB of javascript frameworks across to mobile devices.
Shaving a couple of bytes off a header is not going to move the needle there.
For the record, I ran my internet connection of one of those pre-paid SIM cards for a month and the big killers were websites that keep refreshing their pages, javascript frameworks and advertising.
I agree the developing world wasn't an explicit consideration, but mobile most certainly was. And since better mobile performance and behavior is what developing world Internet clients most need from a new protocol, many of their concerns are being explicitly addressed.
As for whether "shaving a couple of bytes off of a header" makes a practical difference, you should consider the "Why do we need header compression?" answer here: http://http2.github.io/faq/ which references concrete evidence provided by an HTTP2 advocate. Personally, I find that more convincing than your apparently unsupported assertion.
Separately, it's nice to say websites should drop the eye candy and the Javascript frameworks (the hundreds of requests is addressed by HTTP2 multiplexing, I believe), but that's missing the point. As has been repeatedly demonstrated since the earliest days of WAP, mobile users don't want a stripped-down subset of the Internet, they want as much as they can get. From that perspective, raising the bar on what's possible (as HTTP2 tries to do) moves the ball far more than trying to convince websites to fit themselves into today's constraints (especially since the mobile user in the developing world has one of the strictest versions of those constraints).
You'd have to separate out the components in HTTP2 to make sense of the various advantages offered. As long as the cookie abomination persists the header compression makes sense, but if you were to really come up with an improved HTTP then you would get rid of cookies entirely and make HTTP a stateful protocol, since that's what we seem to be headed for anyway, or at a minimum get that session stuff under control.
So now we get this terrible mix, a binary protocol that pretends to be stateful but really isn't, carries over all that's wrong from the past and replaces it with a bunch of fixes that don't seem to address the root causes of the problems.
As for mobile users wanting the full experience, sure, that's why we want better mobile networks, and preferably un-metered ones. (Which is already happening, no limit contracts are making a come-back.)
Anyway, I'm sure that HTTP2 will be implemented because Google has found a way to ram its competitive advantage down our collective throats and we'll have to live with it but I don't think the minor advantages gained here are worth the retooling required. Think of it as a missed chance, if we're going to retool then let's do it well and not in a half-assed and fast-tracked manner like this.
Actually, I think I now understand where we disagree. It isn't exactly where I expected.
You want a New Improved Transport Protocol that is a sharp break with HTTP's past. I agree that would be a good thing. We both agree that HTTP2 isn't that.
Where we disagree is this: You want NITP to be called HTTP2. I think doing that would be the surest way to smother NITP in its crib. In my opinion, an HTTP2 that made a radical change like not supporting cookies wouldn't be a step forward, it would just be perceived as "breaking everything" and would never get adopted.
You can't just slap HTTP's brand on a radically different protocol and expect people to not notice or care about what's inside. If you want to do something really new, that's a much longer and harder path than what we see here. That's why things like HTTP2 (making improvements where practical while carrying forward the concepts and baggage of the past) are worth doing. Because while we're waiting for a bigger leap forward, we've accumulated a set of problems we can solve and that are worth solving.
and the speed of light is constant. http1 has issues accessing all available bandwidth, and speed of light ensures that with a limited bandwidth your page load times will suffer. you don't go to web pages that take forever to load. nobody does.
the bandwidth argument is not the argument you should be making. you should be making the argument that today's pages are bloated and we need to work on reducing page/image size. that is being worked on too.
everybody willing to diff http1 and http2 should just take a look at this with a modern browser and imagine it in a standard site context (bigger images, many more site elements):
"That's a lot of http connections which adds up to a lot of bandwidth and time savings."
I was only addressing the 'bandwidth' part.
The latency bit (which is partially addressed by running multiple concurrent connections via a single socket) is a different issue, for which we also already have multiple solutions (for one, stop making a few hundred requests for a page).
> for one, stop making a few hundred requests for a page
Telling the world "you're doing it wrong" is not a pragmatic solution. HTTP/2 exists to improve the web as it exists today. In this, it demonstrably succeeds.
For the same reason Javascript will remain the assembly language of the web, so too will HTTP/1.1 semantics likely persist forever. It's not because it's what we want but because there is no (yet) known transition away from it. HTTP/2 doesn't replace HTTP/1.1. It just deprecates a horrible wire format and adds some other features.
No, but latency has a linear relationship with throughput
Test at page at 1Mbps thorough 10Mbps and 25ms RTT and watch how small the performance gains are above 5Mbps, then do the same with 10 to 200ms RTT and 10Mbps and watch performance dive.
Bandwidth is just one problem. If you use HTTP/1.1 to download large files, you won't have much problem. If you try to design a memcache-like protocol based on HTTP/1.1, the result will be terrible. You simply can't afford to parse text at 100+k QPS and wait for previous request to finish before sending next one. HTTP/2 is not designed to solve bandwidth issue. It is about latency and throughput in real world.
It's not about bandwidth. It's about latency and how completely inefficiently HTTP uses TCP connections. And actually the fact that our modern connections have so much bandwidth is what is exposing how poorly HTTP utilizes the network.
"HTTP has its warts and quirks but for the most part we have worked our way around those"
No we haven't. Look at any waterfall graph. See all that green? That's the network doing nothing. It's mostly green. HTTP's request/response model creates head-of-line blocking and uses the network connection poorly. (And the HTTP pipelining dream isn't going to happen with 20 years or HTTP devices out there)
The way we have "worked our way around those" are hacky ways to either try and reduce the number of requests we have to make (combine files, CSS sprites, etc), or by hacks to try and run more poorly implemented network connections in parallel to minimize the effect (domain sharding).
The "reduce the number of request" hacks suck for many reasons, of both the devs (build scripts, slicing the CSS images) and the end user (caching opportunities limited to giant JS or CSS bundles that frequently change invalidating the entire bundle, do you know how much memory a 2000 x 2000 px PNG24 CSS sprite takes up?). The "use a bunch of bad connections in parallel" hack sucks a bunch too. It uses more socketsand it uses slower, cold TCP connections, and its a pain to maintain.
I agree that HTTP/2 is making some assumptions about the underlying network (specifically around reducing the size of certain payloads to the fit inside default TCP windows for cold connections). However, HTTP/1.x's way of sending several kilobytes of duplicate plaintext information for every request is a poor approach as well.
> The short version is that is all about speed and security (by encrypting everything).
I'm not up to speed on certificate verification for HTTP2. Have the IETF suggested a mechanism? If so t must be outside the draft-ietf-httpbis-http2-16.
It seems like the only improvement to HTTP is the header compression. The rest of it is just a new transport protocol over TCP.
It would have been so much nicer just to HTTP 1.2 with header compression. Then they could propose a new transport protocol (or just use SCTP) and provide layer 7 implementation over TCP or UDP to ease the transition.
Have you read a protocol document before? Before that section there are very standard stuff like framing, protocol identification, how the headers are laid out, etc.
Section 8 is specifically about the semantics of the 'conversation' between the client and the server. It's important to actually know how to form the messages themselves first!
Think of it as if you were writing a specification of the English language. You might think it's all about conversations, but first you need to know what the letters are, how you form them into words, then spelling, grammar, etc.
What I was trying to imply is: just compare the HTTP/2 spec and the HTTP/1 one.
The HTTP/1 has chapters like Request, Response, Method definitions, Header Field definitions, etc. All things you would expect from an application layer protocol, designed to transfer HTML documents, which made sense at the time.
Now, HTTP/2 pretty much keeps all these things, but most of the spec is about Streams, Multiplexing, Flow Control and such terms. Does that sound like it should be part of an application layer protocol on top of TCP? To me, it definitely does not.
EDIT:
And that's not even touching the question whether it makes sense to build on the original HTTP protocol at all given how much the web has changed since its inception.
What's the debugging problem you're worried about? I've been debugging HTTP for 20 years at this point, and I'm not sweating it.
99% of the time I'm using Wireshark, wget, curl, or my own client or server code, which is always using an HTTP library. 1% of the time I'll telnet to the raw port and type in some incantation.
I don't expect the 99% cases to be different; by the time I'm dealing with it much in the wild, I expect the tools to be fine. And the telnet-by-hand case is still going to work; you'll just have to keep using HTTP 1.0 or 1.1. Which is going to be supported in practice for decades more.
The problem is that a new protocol that will be implemented by everybody and their brother in buggy ways will see a lot of debugging initially, and that's usually when my phone rings. Tools like wireshark will have plug-ins for the protocol, assuming it is implemented correctly those plug ins will show you what's going on at the HTTP2 level. But since that's with a high likelihood exactly where the bugs will be that part isn't going to work, which translates into digging through hexdumps for the foreseeable future trying to figure out why widget 'x' doesn't want to talk to device 'y' when the moon is full and the wind is from the west.
With a text based protocol you at least stand a fighting chance, I've done my share of slogging though dumps and I'm not looking forward to a repeat. One of the main reasons I suspect HTTP caught on as fast as it did was because people could actually look under the hood and understand the basics and figure out where things went wrong without resorting to dumping the the data and counting out which variable length header bit got it wrong this time.
HTTP isn't perfect, don't get me wrong (not specifying an end-of-line for the header with a single character was a mistake in my opinion, and there are a few quirks that make a much faster header parser impossible but that's minor stuff).
So, the telnet by hand case will still work, but that's not where the bugs will be, in fact, testing using 'telnet' will quite possible show you a situation inconsistent with the one using the newer version of the protocol. Having two delivery methods for the same data under the hood is a bad idea to begin with.
Having developed HTTP/2, SPDY, and HTTP/1.1 framers, I can tell you that HTTP/1.1 is by far more error prone. Length encoding (i.e. binary format) is a god-send for reducing bugs.
Well I was going to give an example of a non-binary format with length prefixing but I've ended up entirely unsure what the definition of 'binary format' is.
Instead I'll say that length prefixing is good and useful but doesn't necessarily provide a correctness advantage over simple delimiters. It doesn't really shine until you've already decided to allow unrestricted binary data inside of strings.
Out there in the real world, people are going to keep putting unrestricted binary data inside of strings regardless of whether or not anyone has decided to allow it.
I think these fears will turn out to be fairly minor in practice. I've been implementing some protocols over the last few weeks (for controlling satellite terminal units), one text based and one binary. It was a huge reminder of how much simpler a well defined binary protocol can be to implement - it took far less time (and a lot less debugging) for the binary one.
Even a well specified text based protocol can have so many surprising edge cases that just don't exist in binary...
It does require a bit more tooling for debugging, but that's not that hard...
Your statement is incorrect. HTTP/2 essentially sends HTTP requests/responses through a multiplex channel. The semantics is nearly identical to HTTP/1.1. SPDY has approved it in real world for several years, and many major companies adopted SPDY long time ago (like Amazon Kindle).
HTTP/1.1 is nowhere near simple or elegant. The current spec has 6 RFCs, and a real world implementation is totally non-trivial. RFC1945 is HTTP/1.0, which nobody uses it.
Unless someone proposes an acceptable better solution (Microsoft tried), the world needs to decide on something and move on.
HTTP/2 helps reduce the latency penalty that's one of the biggest constraints to HTTP/1.x performance.
Sure we can work around this using techniques like CSS sprites, merging JS files, CSS files etc. but these are hacks that come with their own tradeoffs - most notably caching but also sprites bring memory challenges on some devices too.
Using a single TCP connection opens up some issues where there's packet loss but it also means the TCP congestion window will grow at it's maximum rate without the risks that opening multiple connections and domain sharding bring.
Multiplexing also offers interesting possibilities of partial resource download e.g. download part of a progressive JPEG, and then the rest later.
Push allows servers to send content e.g. CSS, fonts e.g. that's in the browser's critical rendering path before the client has even discovered it needs it.
Header compression reduces both the request and response overhead, and request overhead if often forgotten in web development which is funny as most connections have asymmetric speeds.
Prioritisation allows the browser and server to schedule content download 'more intelligently' than our current browser heuristics.
In a world where most people are looking at HTTP using DevTools, WebPageTest and Wireshark rather than the native text format a binary protocol isn't a problem.
It's mostly more efficient (in terms of average latency, speed and total bandwidth).
Kinda like BMP format to PNG format (with compression and alpha channels). Yes, BMP is a lot simpler and you can find out the RGB of any pixel by looking at pixels[y*width+x] while with PNG you have non-trivial complexity with compression, etc. But the size efficiency is worthwhile.
What you do get with HTTP/2 is the ability to use bandwidth a lot more efficiently. Because of TCP's congestion control, such as slow-start, doin a different TCP connection for each file, as HTTP/1.1 does, is actually a fairly terribly inefficient for transferring small files. One way that web developers have tried to get around this is to combine resources together - such as having huge combined javascript and CSS files, and big sprite sheet images. But this messes up caching - if I change a 20KB source file that is part of a half meg combined JS file, then you have to redownload the entire file.
Another problem is that you're limited to how many HTTP requests the browser will make to a single domain at once, so you have to wait around for files to finish before others will download. You can try sharding the files across different subdomains but it's a suboptimal solution.
The multiplexing in HTTP/2 solves these problems. You can send a bunch of files at once in one connection without repeating the (necessary) slowness of TCP slow start for every file, and the browser realises that they're different, so can cache them separately.
This can translate into noticeably faster page load times.
Web development in 2014 consists of using four dozen CSS files, two hundred JavaScript files, and upwards of 750 HTTP requests to an API in order to display a single blog post. It is unthinkable to impose any sort of discipline on the use of resources by developers, even at companies which theoretically have an incentive to be efficient.
As a result, those companies have banded together to produce a bloated, multiplexed blob-fish of a protocol which they have vomited into the standards body for fast approval, over the objections of the people who will actually have to implement it.
It feels like HTTP2 is a classic case of "something must be done, this is something, therefor it must be done". Clearly there are shortcomings in HTTP 1.1 which would be nice to address. Google to their credit spent a lot of resources coming up with a solution which met their needs. The problem is that when Google then went to httpbis the people on the WG apparently took it as an imperative that _something_ must be released as HTTP2 in relatively short order. There was a halfhearted attempt to open things up to competing ideas, but unsurprisingly SPDY was by far the most mature of the proposals. Thus SPDY became the heir apparent to HTTP by default, despite being a mud ball of complexity and layering violations.
Why is it undeployable? I mean, we're talking about a new standard here. Anything goes. It doesn't have to happen tomorrow. Google only has influence over their HTTP client and server implementation and with such restrictions, SPDY was the best they could come up with. That doesn't mean the new global standard can't dump TCP for example.
Mozilla and Google no longer have any sponsorship agreements. And I can guarantee you that to most developers in Mozilla, that agreement never gave any technical or political credence to Google and its actions.
HTTP2 is a layering violation because it implements a new layer of multiplexing and flow control on top of the existing layer of multiplexing and flow control. Rather than solving the problems with TCP they slapped a band-aid on top because that allowed them to get to market faster. In the short term it's a win for Google et al, but in the long term this sort of thing will turn the internet into (even more of) an unmanageable mess.
There's a huge cost to "solving problems with TCP": the existing protocol is intricately coupled between implementations (see: every attempt to improve TCP congestion control since Vegas) and so widely deployed as to be impossible to forklift out. Meanwhile: new protocols deployed alongside TCP aren't firewall-friendly.
If there's a multiplexing problem to be solved on the Internet, it more-or-less must be solved above TCP, no matter what the "layering" guidelines are.
Meanwhile: I'm still not clear on why these "layers" exist, or need to be dignified. The whole idea of a "layer" of complicated functionality is in tension with the End To End Argument: if there's a debate about how something should be implemented, that thing should be implemented as close to the endpoints and as close to the application as possible.
Sure, it would be a huge amount of work to fix/replace TCP. Google is one of the few companies with the resources to even attempt it, so it's disappointing that they aren't making it a priority.
The really frustrating thing is that Google is already working on QUIC, which is probably the best approach with a realistic chance of success. It removes TCP from the stack and provides a general purpose transport layer rather than one which is tightly coupled to HTTP. Unfortunately Google decided to push SPDY with it's own kludged up transport rather than taking the time to do it right with SPDY over QUIC.
SSL/TLS is something that needs to be thrown away and start over (not that it would happen realistically without immense pressure after another spectacular failure). The over-complexity of X509 and the ease of which one can acquire legitimate certs for domains one doesn't own is appalling. From recent revelations, it's even more troublesome the number and scale of exfiltration of private keys, making it possible for some state actors to MITM 10's-100's megaconnections. (One has to put on their tinfoil hat to estimate how many countries have successfully placed staff in core IT/webops positions of Fortune 100 that are then able to leverage that access... Not to mention high-level engagement. [The direct approach conversation might go like this: "gives your keys or we will send in agents to expose embarrassing details about your org and we will still get the keys anyway."])
Perhaps folks like 'cperciva would be kind enough to propose a single, simple TOML-based cert system that is extremely lightweight with the fewest of features. (Not that TLS/SSL would change without focused, sustained herculean effort immediately after yet another Heartbleed.)
Extremely light weight? It may be complex, but x509 is based on ASN.1 and one of the most compact and efficient way to represent and handle certificates.
There are so many tools supporting it properly, and many of them are extremely hard to change (especially HSMs) because there are actual dependencies, that I wouldn't bother.
The problems are mostly in SSL & TLS protocols, not in the certificates. We should get a new alternative that would be designed to be easily implementable, and it should get proper reference implementation (with proofs of correctness, which by the way are available for X509... That's the only part that is verifiable afaik).
I'm so glad HTTP/2 is finally here to save us from the horrors of the web stack by providing a decent session layer, privacy preserving defaults, cross-domain and efficient differential caching, as-near-as-can-be bulletproof password-based authentication, and mandatory encryption.
The fact that M. Belshe is listed as the primary author, when he didn't even work on the document, says it all. This is just Google forcing the IETF to gold plate SPDY.
Back in 1989 Sir Tim Berners-Lee put a lot of careful thought into the design of a protocol for sharing documents using IP/TCP. However, when Ajax and Web 2.0 got going circa 2004, the emphasis was on offering software over TCP, and for that the HTTP protocol was poorly suited. Rather than carefully rethink the entire stack, and ideally come up with a new stack, the industry invented what amount to clever hacks, such as WebSockets, which were then bolted into the existing system, even relying on HTTP to handle the initial "handshake" before the upgrade.
What I would like to see is the industry ask itself, can HTTP be retro-fitted to work for software over TCP or UDP? It is clear that HTTP is a fantastic protocol for sharing documents. But it is what we want when our goal is to offer software as a service?
I'll briefly focus on one particular issue. WebSockets undercuts a lot of the original ideas that Sir Tim Berners-Lee put into the design of the Web. In particular, the idea of the URL is undercut when WebSockets are introduced. The old idea was:
1 URL = 1 document = 1 page = 1 DOM
Right now, in every web browser that exists, there is still a so-called "address bar" into which you can type exactly 1 address. And yet, for a system that uses WebSockets, what would make more sense is a field into which you can type or paste multiple URLs (a vector of URLs), since the page will end up binding to potentially many URLs. This is a fundamental change, that takes us to a new system which has not been thought through with nearly the soundness of the original HTTP.
Slightly off-topic, but even worse is the extent to which the whole online industry is still relying on HTML/XML, which are fundamentally about documents. Just to give one example of how awful this is, as soon as you use HTML or XML, you end up with a hierarchical DOM. This makes sense for documents, but not for software. With software you often want either no DOM at all, or you want multiple DOMs. Again, the old model was:
1 URL = 1 document = 1 page = 1 DOM
We have been pushing technologies, such as Javascript and HTML and HTTP, to their limits, trying to get the system that we really want. The unspecified, informal system that many of us now work towards is an ugly hybrid:
1 URL = multiple URLs via Ajax, Websockets, etc = 1 document (containing what we treat as multiple documents) = 1 DOM (which we struggle against as it often doesn't match the structure, or lack of structure, that we actually want).
Much of the current madness that we see with the multiplicity of Javascript frameworks arises from the fact that developers want to get away from HTTP and HTML and XML and DOMs and the url=page binding, but the stack fights against them every step of the way.
Perhaps the most extreme example of the brokenness are all the many JSON APIs that now exist. If you do an API call against many of these APIs, you get back multiple JSON documents, and yet, if you look at the HTTP headers, the HTTP protocol is under the misguided impression that it just sent you 1 document. At a minimum, it would be useful to have a protocol that was at least aware of how many documents it was sending to you, and had first-class support for counting and sorting and sending and re-sending each of the documents that you are suppose to receive. A protocol designed for software would at least offer as much first-class support for multiple documents/objects/entities as TCP allows for multiple packets. And even that would only be a small step down the road that we nee
d to go.
A new stack, designed for software instead of documents, is needed.
I would have been happy if they simply let HTTP remain at 1.1 forever -- it is a fantastic protocol for exchanging documents. And then the industry could have focused its energy on a different protocol, designed from the ground up for offering software over TCP.
Technology ebbs and flows, I feel like this is a backdrift like XHTML but it will flow again.
Binary in Hyper Text Transfer will never seem right. I understand it is more performant but it always creates more bugs, ask any game developer, binary needed but also living on the edge of indexes/ordering/headers/harder to debug/etc. Indexing, overflows, incorrect implementations, will follow.
Many of the advancements in HTTP2 are good but there are some steps backwards we'll have to re-learn again. It isn't all about performance when it comes to correct interoperability as standards lead to many interpretations, it is why XML then JSON won data transfer, it is easy to interoperate, yes binary is more efficient over the wire but not to interoperate. Should we go back to binary formats for data exchange on the network? The protocol level is lower level but still it has been beneficial in the current standards to spreading innovation with lower barriers to understanding.
HTTP2 is one of those 'version 2' of an app that some of the legacy genius of it was lost and overlooked in the redesign, like simplicity. An engineers job is to make something complex into something simple and blackboxing data isn't simplifying it.
>HTTP2 is one of those 'version 2' of an app that some of the legacy genius of it was lost and overlooked in the redesign, like simplicity.
Calling HTTP1/1.1 genius sounds like an "intelligent design" argument (as opposed to "evolution"), and I think detracts from what makes it good.
What makes more sense to me is HTTP1/1.1 was invented and then we hacked/adapted/"evolved" on top of it to get it to do what we want. It wasn't the spec that was genius - it was the effort of countless engineers overtime the crammed a genius, trillion dollar industry into an "okay" spec (The same way it was done for HTML/CSS/JS).
in that vein, the whole binary/plaintext header arguments seem a lot closer to "this is the way my father did it" rather than "this is the most efficient way". To counter your XML/JSON example - I would argue they won over binary formats because there a huge need for humans to write & edit data exchange structures. OTOH, I can't remember the last time I sent/edited/created HTTP Headers by hand. While JSON has tons of uses and is stored in countless places(config, user data, state data), HTTP servers are the only services that seem to care about HTTP headers.
And moreover, the new push is towards protocols which seamlessly turn JSON into binary and back again. Which is a very good way to do things, it's just so rare that anyone ever bothers to maintain and standardize both sides of something.
HTTP2 is yet another in a long series of developments that feel like the corporate takeover of the commons. Sure there are plenty of excellent features in it but they are primarily of benefit to systems doing huge (on lots of dimensions) stuff.
Is this the inevitable path of any technology which has initial promise for enabling individual public expression?
My opinion is exactly the opposite. Large organizations have the resources (person-hours, server infrastructure, etc.) to do all of the hacks required to get decent performance out of HTTP/1.x. Smaller shops aren't as likely to have the time to do domain sharding, sprites, etc.; yes, there are tools and services to do a lot of that work for you, but it still adds complexity.
150 comments
[ 1.3 ms ] story [ 73.0 ms ] threadTons of change for marginal gain. And of course the old stuff will need to be supported at the same time.
Edit: downvoters: please explain what's to like about HTTP2. I have a very hard time finding anything to like.
For example: no more easy debugging on the wire, another TCP like implementation inside the HTTP protocol, tons of binary data rather than text and a whole slew of features that we don't really need but that please some corporate sponsor because their feature made it in. Counter examples appreciated.
Compare: http://tools.ietf.org/html/rfc1945
Keep in mind that Hypertext as it was originally envisioned by Ted Nelson had nothing but two way links, doing one-way links broke with the view of the day in a pretty drastic manner (and actually made the whole thing possible), so I think some leeway here is allowed.
Right ;)
I worked a bit for a small newspaper in Amsterdam in the 1980's, called 'the paper about Holland'. It was written by two UK journalists, Abi Daruvalla and Mark Fuller.
We were pretty good at proofreading because spell checkers were non-existent at the time, we barely had a word processor. It would probably surprise you that as a non-native English speaker I would catch typos (not grammar errors) much quicker than the rest of the (native speakers/writers) crew. We figured it was because they didn't actually read the words where I had to. They'd mentally fill in the word that should have been there which allowed them to proofread a piece with three individuals and I'd still catch the mistakes in the spelling.
In spite of that we managed to ship two issues labeled September in one year, and that's with everybody being so focused on the spelling that nobody bothered to check the dates.
I think you're reading way too much in a single letter spelling error (which I spotted the first time I saw the word), the common excuse is that the spell checker used by the person that wrote that part did not include a spelling for 'referrer' (and if it had it would have probably ignored it with the leading capital anyway...).
I've seen some pretty elegant code with terrible spelling errors in the variable names, it's certainly not great to have a spelling error in a text protocol. But I would not use that single letter mistake as a yardstick by which to measure the entire protocol, that seems all too easy a way to discard something that has stood the test of time very well.
Imaginary meat. Sure, you can talk about the implications of the actual header, but you must first acknowledge that this has nothing whatsoever to do with the context of 'elegance'.
HTTP's popularity outside of the web is mainly cause it's easy to use as a simple wrapper around TCP with read and read/write semantics. Nearly all the rest of HTTP is unneeded fluff (proof: browsers didn't even do other methods besides GET/POST and applications were just fine).
Voting is meant to promote quality contributions and increase the overall quality of the site. A site where dissent is pushed out of site would be very dull.
But since a downvote is a punishment, often times people will second guess if they should post something interesting just in case it is taken the wrong way by someone. That's happened enough to me that now I very seldom comment on any thread that has any type of controversy in it, just because I know that someone will get upset no matter what. (Maybe a fix for this is to make it free to give upvotes, but downvotes can only be given if you've purchased them with some of your own karma points).
Yes. A thousand times yes. I finally had to create a second "alter ego" account just so I would stop censoring myself.
I have ended up, as expected, getting a small number of downvotes and comment flags from people who simply didn't like what I had to say. Even when what I said was a factual summary backed by strong scientific evidence for which I provided citations (with only minimal opinion sprinkled in).
edit: *was
It's solved exactly at the level it had to be solved.
The nitty gritty of why opening a new tcp stream for each http request is slow (and why http pipelining doesn't make it faster the way spdy does) is beyond the scope of this post but https://www.youtube.com/watch?v=-yxQIRl6Qic explains that and other benefits of http/2.0 in great detail.
I think the biggest win for HTTP/2 is multiplexing the requests over a single TCP connection. The average web page downloads over 1MB of data over 80 to 100 Http requests. Each request is requires its own TCP connection.
Modern browsers download 6 assets per host at a time. This decision is so we don't have too many TCP connections competing for the same bandwidth. People optimize the page load times, by combining javascript, combining images with CSS sprites, or adding additional hosts so they can download more of the assets in parallel.
I think that the waterfall chart is an intuitive way to understand how the assets on your website is affecting the page load time. It will be interesting to see what new ways we visualize the process of assets downloading and different browser events, when HTTP/2 starts to be more widely used.
Using SPDY only shows improvement over HTTPS. Over HTTP you get a huge performance hit!
The majority of data has no need for encryption. Some of it does. Where it does, SPDY certainly makes sense.
No. Wrong. Absolutely, incontrovertibly wrong. Everything that doesn't absolutely need to be plaintext should be encrypted; everything that can be authenticated should be.
Don't believe me? The IETF and IAB can't really be much stronger than this: https://tools.ietf.org/html/rfc7258 https://www.iab.org/2014/11/14/iab-statement-on-internet-con...
I haven't checked yet but I won't be surprised to find out that Chrome keeps one SPDY/HTTP2 connection open to google-analytics.com that's shared for all domains...
Why push SSL so hard for sites many of which are just public info anyway? Because then you can't really use proxies so almost always 1 connection is 1 person.
The HTTP connection timeout is like 15 seconds or less in popular software. One of the justifications for SPDY/HTTP2 is that with one connection it can be kept open for many minutes or even hours without using too many resources.
Instead of many separate, possibly proxied connections to doubleclick you have one connection that lasts maybe even a whole browsing session. Google may not need this, but it's certainly better for them.
If you spend less than 4 minutes per page then that entire browsing session across all google-analytics pages is uniquely identified to google. Even if you have many people behind a NAT and they have 3rd party cookies disabled, each one's browsing is still a separate clickstream to Google.
Only thing making a person's browsing opaque to google is if they browse behind a proxy with a bunch of other people mixed in, and SPDY/HTTP2 conveniently enough make proxies very difficult by needing to trust the proxy's certificate on each client. So anybody needing a proxy will use HTTP1 and anybody using HTTP2 has their browsing sessions uniquely identified.
https://www.varnish-cache.org/docs/trunk/phk/http20.html
previous discussion on HN:
https://news.ycombinator.com/item?id=6014976
PHK also refers to HTTP/2 as a political protocol and casts various aspertions about those promoting it but you need to remember that PHK isn't a neutral observer - Varnish does't support TLS yet (though PHK is very clear on why this is the case).
Sounds valid to me. Either retain state or don't.
I suspect it will come in time but I'd argue it's better to develop incrementally rather than a big bang approach, and there's plenty in HTTP/2 to allow further improvement.
I'll miss easy wire debugging too, but it's been obvious for a long time that HTTPS everywhere is the future... and when you go to the effort to use a MitM proxy or whatever for debugging TLS, you may as well throw in a tool that understands the protocol.
Edit: I agree it would be nicer to solve many of the goals of HTTP2 at a lower level, and from the little I've heard about it, that solution already basically exists in the form of QUIC. But server push is a useful feature even with a perfect transport layer, so meh...
Heck, I can read a binary protocol as hex if I know it well enough - writing a tool which can do this is not a difficult task (it's actually way easier then text parsing!) and gives you more concise answers more quickly.
Literally so: this protocol document does not specify how you determine which server to connect to. HTTP2 is, in definition, only very loosely coupled to IP despite making significant optimisations for TCP. Thus in implementation we simply get the same old mistakes and undefined behaviours. Issues with floating apex records, hacks based on IPv4/6 race conditions, unnecessary address wastage and so forth will continue; all derived from the colossal architectural wart of overloading the DNS host (A/AAAA) record as a service endpoint discovery mechanism.
Once again, I say unto the peanut gallery: shoulda used SRV. The benefits are many and the downsides greatly overstated. I bemoan the missed opportunity.
apex record -> alias for 'A' record (DNS parlance)
So a 'floating apex record' is an A-record pointing to an old IP.
I'm not a DNS wonk either.
For example, in "https://github.com/" they are the records particularly for "github.com", rather than for subdomains that might exist such as "www.github.com" or "gist.github.com".
Apex records have a particular restriction: they cannot be aliases, because the apex includes DNS metadata that is not allowed to be aliased[3]. Read on for how this becomes a problem.
I've used the term "floating" as a visual metaphor, because what I'm about to describe lacks a universal standard name, because it is an ugly hack:
HTTP resolves endpoints using host records, so an URL of "https://github.com" means looking up A and AAAA records for "github.com". Yes, the protocol is arrogant enough[1] to assume that your host address for the whole domain is that of the web server. (This is why we ended up prepending "www" to domain names, as a service selector). In response to the query you get an IP address.
Unfortunately, IP addresses sometimes change without warning. The most common example today is the loadbalancer offered by Amazon Web Services. The solution to this is to use an alias record in your human-friendly domain, pointing at an hidden technical domain that the infrastructure provider keeps up-to-date (e.g. "my-elb-name-1-1160186271.ap-southeast-1.elb.amazonaws.com")
This is fine for "www.example.com" but not the naked "example.com", because aliases are prohibited at the apex.
As a result, DNS providers such as Route 53 have ended up with a hack: a spoofed record at the apex, one that tracks an external resource and synthesizes a fake A/AAAA response. Now you have a naked domain that tracks, or rather hopes to track, the correct endpoint. But it changes with the wind. Hence my description of it as "floating".
There is no consistent name for this kludge. AWS calls it an alias, and for reliability concerns restrict it to their own infrastructure only; DME call it an "ANAME" record [2]. The model can even be readily implemented as a shell script run out of cron on your nameserver. It is fragile, it is often unreliable, it is not at all standardised, and it doesn't scale beyond one service.
One better solution would be to require use of SRV records, which allow one to declare instead, for example, an "https" service for "example.com". Alongside, let's say, the xmpp service, sip service, or any other service you care to announce. SRV records can exist at the apex. They can also bundle the A and AAAA (IPv6) addresses for the resulting endpoints in the answer, and select alternative port numbers without bothering the user about it.
Not quite a universal panacea: there is a minor hazard of zone cuts that could increase the number of client lookups, but that's an edge case, not one you can easily blunder into and also easy to fix.
[1] HTTP/1.0 and earlier are forgiven, because they hail from a time when you just had a web server in a rack and called it "www". But HTTP/2 is supposed to respond to modern architectures.
[2] http://www.dnsmadeeasy.com/services/aname-records/
[3] none of you comedians are allowed to mention DNAME records as the exotic counterexample.
Do you have any clue why SRV is not more widely used ?
I suspect they aren't more popular because it requires some DNS knowledge before you think of them. It's a pity because they are very useful.
Aside: Cloudflare's free DNS hosting service supports them, with low TTL.
Edit: It makes me a bit giddy (which makes sense if you factor in my being a sysadmin) to think about what SRV records would've done for load-balancing, running servers on non-standard ports, IP address exhaustion, and server migrations. Anybody who doesn't appreciate proper service-location hasn't ever done serious sysadmin work and, IMO, has no business designing protocols.
I think agl did; the state of DNS resolution is, I'm afraid, really not pretty.
You say you agree, because "the protocol does not specify how you determine what server to connect to."
In other words, HTTP2 sucks because it simultaneously includes too many features, and not enough features.
At least everyone can agree that they don't like it for some reason, even if the reasons themselves contradict each other.
How does too many features and missing features contradict each other?
It's entirely possible for something to both have too many, unneeded features, while at the same time missing other important features.
No, they are agreeing that it's a heap of extraneous features.
For example, HTTP2 will stop ISP header injection such as this: http://www.propublica.org/article/somebodys-already-using-ve...
Both still leave the original text based protocol and would stop ISP header injection.
Another defense against such trickery would be a legal one: make it illegal to tamper with data sent between two peers on the internet.
Imagine the postal service opening your mail and changing a couple of words in your mail just because they could.
SSL is not a text-based protocol.
Inside the SSL/TLS wrapper it is still just HTTP.
It's no longer 'in the clear' but it is still very much a text based protocol.
I'm not an HTTP2 expert, but I would bet that WireShark can make an HTTP2 session look basically the same as a bunch of HTTP1 requests. It's the same "wrapper" concept except it adds multiplexing.
Text implementations are expensive to parse. You end up doing a lot of nutty hacks (like bitwise operations a word at a time, on strings) just to have good speed. Doing so safely requires a lot of branches. A binary protocol will be easier, more compatibly implemented, safer and far faster to decode. Look at nginx code to see examples (and that's a good codebase).
Debugging, yeah, it's sometimes handy to run tcpdump and see problems. But s/tcpdump/tshark and it's essentially a solved problem. The vast, vast majority of HTTP messages are machine written and read, optimizing for humans is misguided.
And yes, I'm bitter about this, as I'm on my third project implementing a SIP stack. SIP shares HTTP's general format, with tons of added insanity for fun. Like UDP/TCP hopping based on the spec author's misunderstanding of IP reassembly. I've lost weeks of my life due to idiotic spec writers just going off and making stuff up. Cause hey, in a text document, there's no actual technical debt to be paid, no actual real world problems.
Google's approach with SPDY is far better than most of what the IETF has been able to put out.
You have any further reading or links for this? SIP is definitely "interesting" (not in a good way) and I'd love to have some background on why it is the way it is.
Anyways, because they thought UDP could not exceed Ethernet MTU, they came up with this crazy protocol hopping behavior. So on a per message basis, a SIP server or client can just flip to TCP or UDP, just for fun. This idiocy is what made MS just drop UDP support in their SIP products, because "most of our messages will be over the MTU and compliant implementations will hop to TCP". But popular stacks such as Asterisk had baked UDP assumptions pretty deep.
To make it even more fun, SIP TCP and UDP differ. In UDP, they decided content length isn't needed since you can just use the datagram boundary. So you get this situation where the validity of their text-bsed message depends on which transport it was delivered on.
The sick thing is that the authors of these RFCs take delight in the pointless complexity they've added. Look up SIP torture tests RFC just to see how demented they are.
In their defense, one of them wrote me and said they had started off thinking SIP would be totally HTTP compatible. After they dropped that idea, they left stuff in because it was "too late". While I understand life's tough, this explanation doesn't do much for my confidence in their decisions. Another one wrote and said " hey, C and other programming languages have flexible syntax, why not SIP?". He wasn't joking :/. But hey, one of them is now the CTO of the FCC, so writing terrible specs must be a good career move.
Worse, many people see " text based " and think it's easier than a proper binary protocol. Many shitty coders have pumped out bad SIP implementations that sorta work. But in fact, you cannot unambiguously parse SIP on the Internet today. Popular implementations disagree on basic things such as line endings. This opens up security holes, as you can make one network element read a message one way, knowing the next element will interpret it differently.
And as you point out, the limit is quite low in such cases. So it'd be better to just drop UDP (for many reasons), versus adding protocol hopping and whatnot.
...
Wait, NIST wrote the reference implementation? Also, your description sounds suspiciously like someone was going for maximum confusion and a "design" that practically guarantees bugs in every implementation.
This stinks of BULLRUN-style sabotage.
The process for re-invites with proxies in the middle, loose routing, nats, and codec renegotiation is a VoIP engineer's hell.
But all those extensions/rules were created with good intentions.
It also shocks me how prevalent SIP is for arbitrary lookups in telcoland. More than one switch uses a SIP call to find number portability info. So they involve at least 3 SIP messages, each hundreds of bytes long, complete with a full SIP transaction state, burning up call resources on their switch, to essentially send and receive 10 bytes.
The amount of stupidity in telecom is mind-boggling. I take solace in hearing that other industries are just as bad, but I don't see them so they just appear to be OK.
And a lot of this stuff is the result of carrying over design decisions, literally, from the 60s. Someone wrote an RFC codifying what they were doing by hand. And that nonsense gets passed forward for zero reason.
Ever wonder why HTTP uses the idiotic "Thu, 28 Jan, 1998, GMT" format for datetime? Because that format makes sense if you're hand-reading email message headers. And HTTP just copied it forward, zero thinking involved. Probably the same reason you can have comments in headers. SIP gives an example:
To real-world engineers, we say, wow, that's idiotic. Comments will never be used, so they'll never be properly implemented and only serve to make things problematic.To an IETF RFC author, that kind of behaviour is "well RFC xxx does it that way", and ultimately comes to "well, like, if you were sending HTTP messages by fax, extra comments might help".
If you stack up enough hacks like that, then it starts to become the clusterfuck it is. I don't think there's any maliciousness involved. Just lack of critical thinking, or lack of experience actually implementing software. Plus the fact that when writing a spec, you've got a blank paper and can just go all-out with crazy ideas you won't have to support.
I don't see how SPDY prevents someone coming up with p2p caching. Although a serious use case would be a good start.
It also addresses the usual complaints, including the "binary" part.
The short version is that is all about speed and security (by encrypting everything).
As to the issue of binary vs. text. Implementing binary protocols (including tools used for debugging them) is much simpler than implementing text protocol. HTTP1 might look simple to human eyes but it's not an easy protocol to implement (fully and correctly). I know, I tried.
Furthermore, engineering is about tradeoffs. HTTP2 is not simple but the complexity exists to address real problems in failure modes of tcp stack and to maximize speed by both sending as little as possible (compression) and ensuring that one request doesn't block the other, as long as there is bandwidth to send both.
Maybe the purity and simplicity of the protocol is more important to you than improving the speed by few %, but to make it makes perfect sense to trade simplicity for even modest gains in performance in the context of http.
You only write the code once but it'll speed up every http connection until end of time. That's a lot of http connections which adds up to a lot of bandwidth and time savings.
Keep in mind that we're talking about the protocol that currently powers much of the developed world as we know it and that mistakes will be extremely costly to correct. HTTP has its warts and quirks but for the most part we have worked our way around those.
Minor efficiency gains do not add up for me to a complete overhaul of the web and everything attached to it.
So what we'll get instead is this: a percentage of the web will transition (mostly the bigger players where that few % adds up to a higher bottom line), the rest will not care and wait for HTTP3 or whatever will come after HTTP2 to really address what's wrong with HTTP (and that's not that it isn't fast enough, that's mostly a problem with the underlying transports, HTTP is plenty fast).
Says someone who I'm certain doesn't rely on a prepaid SIM for their primary Internet connection. Not that I do either, but I'm far more aware that tens (if not hundreds) of millions of these people exist today. In fact, it's likely that such people will be a clear majority of Internet users early in the life of HTTP2. Leaving them out of the conversation, as you implicitly do here, would be a colossal mistake for a protocol that intended to stand the test of time.
Let's not try to pretend that HTTP2 is about bandwidth savings in the third world, that didn't even enter the consideration at any level.
If you want to save bandwidth at that level then you're better off reducing eye candy and using apps rather than websites and not shipping bloated pages with 100's of requests and 1/2 MB of javascript frameworks across to mobile devices.
Shaving a couple of bytes off a header is not going to move the needle there.
For the record, I ran my internet connection of one of those pre-paid SIM cards for a month and the big killers were websites that keep refreshing their pages, javascript frameworks and advertising.
As for whether "shaving a couple of bytes off of a header" makes a practical difference, you should consider the "Why do we need header compression?" answer here: http://http2.github.io/faq/ which references concrete evidence provided by an HTTP2 advocate. Personally, I find that more convincing than your apparently unsupported assertion.
Separately, it's nice to say websites should drop the eye candy and the Javascript frameworks (the hundreds of requests is addressed by HTTP2 multiplexing, I believe), but that's missing the point. As has been repeatedly demonstrated since the earliest days of WAP, mobile users don't want a stripped-down subset of the Internet, they want as much as they can get. From that perspective, raising the bar on what's possible (as HTTP2 tries to do) moves the ball far more than trying to convince websites to fit themselves into today's constraints (especially since the mobile user in the developing world has one of the strictest versions of those constraints).
So now we get this terrible mix, a binary protocol that pretends to be stateful but really isn't, carries over all that's wrong from the past and replaces it with a bunch of fixes that don't seem to address the root causes of the problems.
As for mobile users wanting the full experience, sure, that's why we want better mobile networks, and preferably un-metered ones. (Which is already happening, no limit contracts are making a come-back.)
Anyway, I'm sure that HTTP2 will be implemented because Google has found a way to ram its competitive advantage down our collective throats and we'll have to live with it but I don't think the minor advantages gained here are worth the retooling required. Think of it as a missed chance, if we're going to retool then let's do it well and not in a half-assed and fast-tracked manner like this.
Feel free to disagree.
You want a New Improved Transport Protocol that is a sharp break with HTTP's past. I agree that would be a good thing. We both agree that HTTP2 isn't that.
Where we disagree is this: You want NITP to be called HTTP2. I think doing that would be the surest way to smother NITP in its crib. In my opinion, an HTTP2 that made a radical change like not supporting cookies wouldn't be a step forward, it would just be perceived as "breaking everything" and would never get adopted.
You can't just slap HTTP's brand on a radically different protocol and expect people to not notice or care about what's inside. If you want to do something really new, that's a much longer and harder path than what we see here. That's why things like HTTP2 (making improvements where practical while carrying forward the concepts and baggage of the past) are worth doing. Because while we're waiting for a bigger leap forward, we've accumulated a set of problems we can solve and that are worth solving.
and the speed of light is constant. http1 has issues accessing all available bandwidth, and speed of light ensures that with a limited bandwidth your page load times will suffer. you don't go to web pages that take forever to load. nobody does.
the bandwidth argument is not the argument you should be making. you should be making the argument that today's pages are bloated and we need to work on reducing page/image size. that is being worked on too.
everybody willing to diff http1 and http2 should just take a look at this with a modern browser and imagine it in a standard site context (bigger images, many more site elements):
https://http2.golang.org/gophertiles
http://http2.golang.org/gophertiles
Latency != bandwidth.
Installing ghostery will do more for your page load times than http2 will.
yes, and you're conflating the two.
"That's a lot of http connections which adds up to a lot of bandwidth and time savings."
I was only addressing the 'bandwidth' part.
The latency bit (which is partially addressed by running multiple concurrent connections via a single socket) is a different issue, for which we also already have multiple solutions (for one, stop making a few hundred requests for a page).
Telling the world "you're doing it wrong" is not a pragmatic solution. HTTP/2 exists to improve the web as it exists today. In this, it demonstrably succeeds.
For the same reason Javascript will remain the assembly language of the web, so too will HTTP/1.1 semantics likely persist forever. It's not because it's what we want but because there is no (yet) known transition away from it. HTTP/2 doesn't replace HTTP/1.1. It just deprecates a horrible wire format and adds some other features.
No, but latency has a linear relationship with throughput
Test at page at 1Mbps thorough 10Mbps and 25ms RTT and watch how small the performance gains are above 5Mbps, then do the same with 10 to 200ms RTT and 10Mbps and watch performance dive.
Bandwidth is just one problem. If you use HTTP/1.1 to download large files, you won't have much problem. If you try to design a memcache-like protocol based on HTTP/1.1, the result will be terrible. You simply can't afford to parse text at 100+k QPS and wait for previous request to finish before sending next one. HTTP/2 is not designed to solve bandwidth issue. It is about latency and throughput in real world.
It's not about bandwidth. It's about latency and how completely inefficiently HTTP uses TCP connections. And actually the fact that our modern connections have so much bandwidth is what is exposing how poorly HTTP utilizes the network.
"HTTP has its warts and quirks but for the most part we have worked our way around those"
No we haven't. Look at any waterfall graph. See all that green? That's the network doing nothing. It's mostly green. HTTP's request/response model creates head-of-line blocking and uses the network connection poorly. (And the HTTP pipelining dream isn't going to happen with 20 years or HTTP devices out there)
The way we have "worked our way around those" are hacky ways to either try and reduce the number of requests we have to make (combine files, CSS sprites, etc), or by hacks to try and run more poorly implemented network connections in parallel to minimize the effect (domain sharding).
The "reduce the number of request" hacks suck for many reasons, of both the devs (build scripts, slicing the CSS images) and the end user (caching opportunities limited to giant JS or CSS bundles that frequently change invalidating the entire bundle, do you know how much memory a 2000 x 2000 px PNG24 CSS sprite takes up?). The "use a bunch of bad connections in parallel" hack sucks a bunch too. It uses more socketsand it uses slower, cold TCP connections, and its a pain to maintain.
I agree that HTTP/2 is making some assumptions about the underlying network (specifically around reducing the size of certain payloads to the fit inside default TCP windows for cold connections). However, HTTP/1.x's way of sending several kilobytes of duplicate plaintext information for every request is a poor approach as well.
I'm not up to speed on certificate verification for HTTP2. Have the IETF suggested a mechanism? If so t must be outside the draft-ietf-httpbis-http2-16.
It would have been so much nicer just to HTTP 1.2 with header compression. Then they could propose a new transport protocol (or just use SCTP) and provide layer 7 implementation over TCP or UDP to ease the transition.
8. HTTP Message Exchanges
What the hell is the rest of the document about then?
Section 8 is specifically about the semantics of the 'conversation' between the client and the server. It's important to actually know how to form the messages themselves first!
Think of it as if you were writing a specification of the English language. You might think it's all about conversations, but first you need to know what the letters are, how you form them into words, then spelling, grammar, etc.
The HTTP/1 has chapters like Request, Response, Method definitions, Header Field definitions, etc. All things you would expect from an application layer protocol, designed to transfer HTML documents, which made sense at the time.
Now, HTTP/2 pretty much keeps all these things, but most of the spec is about Streams, Multiplexing, Flow Control and such terms. Does that sound like it should be part of an application layer protocol on top of TCP? To me, it definitely does not.
EDIT: And that's not even touching the question whether it makes sense to build on the original HTTP protocol at all given how much the web has changed since its inception.
99% of the time I'm using Wireshark, wget, curl, or my own client or server code, which is always using an HTTP library. 1% of the time I'll telnet to the raw port and type in some incantation.
I don't expect the 99% cases to be different; by the time I'm dealing with it much in the wild, I expect the tools to be fine. And the telnet-by-hand case is still going to work; you'll just have to keep using HTTP 1.0 or 1.1. Which is going to be supported in practice for decades more.
With a text based protocol you at least stand a fighting chance, I've done my share of slogging though dumps and I'm not looking forward to a repeat. One of the main reasons I suspect HTTP caught on as fast as it did was because people could actually look under the hood and understand the basics and figure out where things went wrong without resorting to dumping the the data and counting out which variable length header bit got it wrong this time.
HTTP isn't perfect, don't get me wrong (not specifying an end-of-line for the header with a single character was a mistake in my opinion, and there are a few quirks that make a much faster header parser impossible but that's minor stuff).
So, the telnet by hand case will still work, but that's not where the bugs will be, in fact, testing using 'telnet' will quite possible show you a situation inconsistent with the one using the newer version of the protocol. Having two delivery methods for the same data under the hood is a bad idea to begin with.
I'm curious what you think about my analysis of the binary vs text argument: https://www.youtube.com/watch?v=-yxQIRl6Qic&feature=player_d...
Instead I'll say that length prefixing is good and useful but doesn't necessarily provide a correctness advantage over simple delimiters. It doesn't really shine until you've already decided to allow unrestricted binary data inside of strings.
Even a well specified text based protocol can have so many surprising edge cases that just don't exist in binary...
It does require a bit more tooling for debugging, but that's not that hard...
HTTP/1.1 is nowhere near simple or elegant. The current spec has 6 RFCs, and a real world implementation is totally non-trivial. RFC1945 is HTTP/1.0, which nobody uses it.
Unless someone proposes an acceptable better solution (Microsoft tried), the world needs to decide on something and move on.
Sure we can work around this using techniques like CSS sprites, merging JS files, CSS files etc. but these are hacks that come with their own tradeoffs - most notably caching but also sprites bring memory challenges on some devices too.
Using a single TCP connection opens up some issues where there's packet loss but it also means the TCP congestion window will grow at it's maximum rate without the risks that opening multiple connections and domain sharding bring.
Multiplexing also offers interesting possibilities of partial resource download e.g. download part of a progressive JPEG, and then the rest later.
Push allows servers to send content e.g. CSS, fonts e.g. that's in the browser's critical rendering path before the client has even discovered it needs it.
Header compression reduces both the request and response overhead, and request overhead if often forgotten in web development which is funny as most connections have asymmetric speeds.
Prioritisation allows the browser and server to schedule content download 'more intelligently' than our current browser heuristics.
In a world where most people are looking at HTTP using DevTools, WebPageTest and Wireshark rather than the native text format a binary protocol isn't a problem.
Kinda like BMP format to PNG format (with compression and alpha channels). Yes, BMP is a lot simpler and you can find out the RGB of any pixel by looking at pixels[y*width+x] while with PNG you have non-trivial complexity with compression, etc. But the size efficiency is worthwhile.
Converting from uncompressed BMP to compressed PNG can easily save over half of the file size; even 10:1 compression is common for some images.
The bandwidth savings in HTTP2 are much smaller, and are probably only significant in aggregate.
What you do get with HTTP/2 is the ability to use bandwidth a lot more efficiently. Because of TCP's congestion control, such as slow-start, doin a different TCP connection for each file, as HTTP/1.1 does, is actually a fairly terribly inefficient for transferring small files. One way that web developers have tried to get around this is to combine resources together - such as having huge combined javascript and CSS files, and big sprite sheet images. But this messes up caching - if I change a 20KB source file that is part of a half meg combined JS file, then you have to redownload the entire file.
Another problem is that you're limited to how many HTTP requests the browser will make to a single domain at once, so you have to wait around for files to finish before others will download. You can try sharding the files across different subdomains but it's a suboptimal solution.
The multiplexing in HTTP/2 solves these problems. You can send a bunch of files at once in one connection without repeating the (necessary) slowness of TCP slow start for every file, and the browser realises that they're different, so can cache them separately.
This can translate into noticeably faster page load times.
As a result, those companies have banded together to produce a bloated, multiplexed blob-fish of a protocol which they have vomited into the standards body for fast approval, over the objections of the people who will actually have to implement it.
If there's a multiplexing problem to be solved on the Internet, it more-or-less must be solved above TCP, no matter what the "layering" guidelines are.
Meanwhile: I'm still not clear on why these "layers" exist, or need to be dignified. The whole idea of a "layer" of complicated functionality is in tension with the End To End Argument: if there's a debate about how something should be implemented, that thing should be implemented as close to the endpoints and as close to the application as possible.
The really frustrating thing is that Google is already working on QUIC, which is probably the best approach with a realistic chance of success. It removes TCP from the stack and provides a general purpose transport layer rather than one which is tightly coupled to HTTP. Unfortunately Google decided to push SPDY with it's own kludged up transport rather than taking the time to do it right with SPDY over QUIC.
Perhaps folks like 'cperciva would be kind enough to propose a single, simple TOML-based cert system that is extremely lightweight with the fewest of features. (Not that TLS/SSL would change without focused, sustained herculean effort immediately after yet another Heartbleed.)
There are so many tools supporting it properly, and many of them are extremely hard to change (especially HSMs) because there are actual dependencies, that I wouldn't bother.
The problems are mostly in SSL & TLS protocols, not in the certificates. We should get a new alternative that would be designed to be easily implementable, and it should get proper reference implementation (with proofs of correctness, which by the way are available for X509... That's the only part that is verifiable afaik).
Oh, wait... maybe that was a dream.
What I would like to see is the industry ask itself, can HTTP be retro-fitted to work for software over TCP or UDP? It is clear that HTTP is a fantastic protocol for sharing documents. But it is what we want when our goal is to offer software as a service?
I'll briefly focus on one particular issue. WebSockets undercuts a lot of the original ideas that Sir Tim Berners-Lee put into the design of the Web. In particular, the idea of the URL is undercut when WebSockets are introduced. The old idea was:
1 URL = 1 document = 1 page = 1 DOM
Right now, in every web browser that exists, there is still a so-called "address bar" into which you can type exactly 1 address. And yet, for a system that uses WebSockets, what would make more sense is a field into which you can type or paste multiple URLs (a vector of URLs), since the page will end up binding to potentially many URLs. This is a fundamental change, that takes us to a new system which has not been thought through with nearly the soundness of the original HTTP.
Slightly off-topic, but even worse is the extent to which the whole online industry is still relying on HTML/XML, which are fundamentally about documents. Just to give one example of how awful this is, as soon as you use HTML or XML, you end up with a hierarchical DOM. This makes sense for documents, but not for software. With software you often want either no DOM at all, or you want multiple DOMs. Again, the old model was:
1 URL = 1 document = 1 page = 1 DOM
We have been pushing technologies, such as Javascript and HTML and HTTP, to their limits, trying to get the system that we really want. The unspecified, informal system that many of us now work towards is an ugly hybrid:
1 URL = multiple URLs via Ajax, Websockets, etc = 1 document (containing what we treat as multiple documents) = 1 DOM (which we struggle against as it often doesn't match the structure, or lack of structure, that we actually want).
Much of the current madness that we see with the multiplicity of Javascript frameworks arises from the fact that developers want to get away from HTTP and HTML and XML and DOMs and the url=page binding, but the stack fights against them every step of the way.
Perhaps the most extreme example of the brokenness are all the many JSON APIs that now exist. If you do an API call against many of these APIs, you get back multiple JSON documents, and yet, if you look at the HTTP headers, the HTTP protocol is under the misguided impression that it just sent you 1 document. At a minimum, it would be useful to have a protocol that was at least aware of how many documents it was sending to you, and had first-class support for counting and sorting and sending and re-sending each of the documents that you are suppose to receive. A protocol designed for software would at least offer as much first-class support for multiple documents/objects/entities as TCP allows for multiple packets. And even that would only be a small step down the road that we nee d to go.
A new stack, designed for software instead of documents, is needed.
I would have been happy if they simply let HTTP remain at 1.1 forever -- it is a fantastic protocol for exchanging documents. And then the industry could have focused its energy on a different protocol, designed from the ground up for offering software over TCP.
Binary in Hyper Text Transfer will never seem right. I understand it is more performant but it always creates more bugs, ask any game developer, binary needed but also living on the edge of indexes/ordering/headers/harder to debug/etc. Indexing, overflows, incorrect implementations, will follow.
Many of the advancements in HTTP2 are good but there are some steps backwards we'll have to re-learn again. It isn't all about performance when it comes to correct interoperability as standards lead to many interpretations, it is why XML then JSON won data transfer, it is easy to interoperate, yes binary is more efficient over the wire but not to interoperate. Should we go back to binary formats for data exchange on the network? The protocol level is lower level but still it has been beneficial in the current standards to spreading innovation with lower barriers to understanding.
HTTP2 is one of those 'version 2' of an app that some of the legacy genius of it was lost and overlooked in the redesign, like simplicity. An engineers job is to make something complex into something simple and blackboxing data isn't simplifying it.
Calling HTTP1/1.1 genius sounds like an "intelligent design" argument (as opposed to "evolution"), and I think detracts from what makes it good.
What makes more sense to me is HTTP1/1.1 was invented and then we hacked/adapted/"evolved" on top of it to get it to do what we want. It wasn't the spec that was genius - it was the effort of countless engineers overtime the crammed a genius, trillion dollar industry into an "okay" spec (The same way it was done for HTML/CSS/JS).
in that vein, the whole binary/plaintext header arguments seem a lot closer to "this is the way my father did it" rather than "this is the most efficient way". To counter your XML/JSON example - I would argue they won over binary formats because there a huge need for humans to write & edit data exchange structures. OTOH, I can't remember the last time I sent/edited/created HTTP Headers by hand. While JSON has tons of uses and is stored in countless places(config, user data, state data), HTTP servers are the only services that seem to care about HTTP headers.
Waiting months/years for HTTP\2 support to appear in all the tools I use - :( ....
Is this the inevitable path of any technology which has initial promise for enabling individual public expression?