I was super excited to dive in an find the RSA code so I could preen about Bleichenbacher's vulnerability, but she neatly sidestepped that by doing ECDH. Then I thought, well, maybe it's P-curve ECDH and I can preen about invalid curve attacks on static-ephemeral ECDH. But nope, X25519! My point here, apart from making fun of myself for being the kind of person who would write this stuff on a message board, is TLS 1.3 is pretty solid.
The "block thing" that's kind of weird is, I assume, the TLS Record Layer. TLS runs (ordinarily) over TCP, which provides a non-demarcated stream of bytes. TLS breaks that stream up into records, and runs its handshake messages over one type of record, (say) HTTPS over another, and "alerts" over a third. The Record Layer also interacts, I think, with TLS's misbegotten compression system?
In the same vein as this project (but with different goals) is Trevor Perrin's tlslite, which is implemented in pure Python: https://github.com/trevp/tlslite
i guess TLS needs records because it wants to mix signalling data with the data stream (stuff like alerts, keyupdate, renegotiation for tls1.2). but i guess also they want to use block ciphers and for stream ciphers they want some kind of authentication so i'm not sure how you would do this without some form of framing. there is also some more strangeness with the TLS record layer because handshake messages and potentially other messages are allowed to be fragmented over multiple TLS records or you can have multiple handshake messages in the same TLS record (i think there are some restrictions on TLS1.3 about mixing different message types). this also might be a way to mess with TLS censorship middle boxes because they might not be robustly coded. i have a plugin for mitmproxy that does TLS interception using knowledge of the shared key but it doesn't handle fragmentation of handshake messages and a bunch of other quirks correctly.
The primary reason you need records is that AEAD algorithms run on blocks so you need to know where the end of one block is and the start of the next. And because you are running over TCP, you don't get that framing information from TLS.
As you say, it's also the case that you have different types of data and so the record type indicates what you are processing, but you'd need records even if you didn't do that.
Seems like you ought to know TLS 1.3 famously doesn't have a way to do RSA key exchange. There's a fun video where some presumably American EDCO people are like, "surely if we all turn up in person we've got more votes so we can get RSA back into TLS 1.3" and er, no, the IETF isn't a democracy.
Looking at GCM or any other authenticated encryption without Sum Types makes me sad. If decrypting the message fails, there is no plaintext and I feel like an API should be able to actually express that properly not return a plaintext [even if it's empty] and a flag saying "Don't use the plaintext". :(
The "block thing" is a necessary choice if we're going to have authenticated encryption, because authenticating costs us overhead. If we wanted to reliably authenticate individual bytes (so that we can have what naively feels like a stream of bytes) we'd be paying that overhead for each byte. That's pretty outrageous. On the other hand if I'm downloading a 10MB file with my 24kbps modem and none of that was authenticated yet because the authentication only happens once at the end of the encrypted data, I'm going to get very anxious about my download counter correctly showing 0% successfully downloaded for so long. So TLS authenticates whole records.
Yep, when digging through the TLS 1.3 RFC (after learning 1.2) the thing that jumped out at me is how many _fewer_ options there are, and how deliberately chosen they seemed to be.
- suites (as far as covering cipher + exchange + mac) are gone as a concept
- only 5 supported cipher algorithms (all AEAD, and all but one built on AES)
- only non-PSK key exchange is (EC)DHE
- no compression supported
- all the various key derivations are built around a single new primitive function, HKDF
I'm sure that given enough time the above options will balloon like we had with latter-day TLS 1.2 but at this point it feels so clean.
I'm sure the options will balloon a bit. After all, if someone develops a new good cipher/etc, we'll want to use it in TLS. But it's nice that as of now, there's no bad choices available (other than potentially allowing downgrading).
It's entirely possible we will never need to replace AES (or indeed ChaCha20). AES 128 GCM is mandatory to implement, and implementers are recommended to offer ChaCha20 too. If you don't have AES hardware, ChaCha20 looks very attractive and you'd be a fool not to.
So while you can get code points for a vanity cipher, there not only isn't an appetite from the working group to endorse such ciphers, there is unlikely to ever be a scenario where a client or server would actually pick it unless you control both and choose to prefer the vanity cipher. Whereupon why bother?
The downgrade prevention in TLS 1.3 is pretty effective, so an attack which needs a downgrade also needs to defeat the downgrade protection, an additional hurdle. Obviously people should get rid of TLS 1.0 and TLS 1.1 but meanwhile I think the anti-downgrade story is the best it has ever been.
We may never need to replace AES 256, but Grover's quantum search algorithm can break AES 128 with a work factor of 2^64. AES 256 GCM and a post-quantum authenticated key agreement algorithm may be all we ever need.
While it's true what you say about Grover's algorithm, I remain sceptical that this can actually be made practical. And if it does become practical as you point out the post-quantum kex is the big must-have anyway. As this toy implementation shows, TLS has at least four symmetric keys for each session, whereas you'd only need to break one kex to get them all.
So I'm comfortable with my claim as it stood, we might never even need AES 256 in practice (but like ChaCha20 it's a SHOULD in the RFC for TLS 1.3).
Everyone likes to bikeshed, but I'm a bit disappointed that TLS 1.3 kept the ability to use pre-shared-keys without (EC)DHE.
I get that the intended use case is not paying the overhead of asymmetric crypto for embedded applications. However, that use case is already covered by something like WolfSSL stripped down to only support TLS 1.2 PSK, and they can still even use modern AEAD cihper suites like AES 256 GCM in TLS 1.2.
All of the TLS 1.3 key agreement mechanisms should have forward secrecy, even when pre-shared-keys are used. In some applications, there's too much temptation to get rid of that safety net for efficiency. We shouldn't clutter up the TLS 1.3 spec and implementations to cover that use case. Furthermore, engineers who go that route should have to explain to their boss solid engineering reasons why they're staying back at TLS 1.2. There are a small number of valid use cases for symmetric-only key exchange, but there are way more use cases where an engineer just thinks the reduced overhead is worth the risk, and their manager sees the TLS 1.3 box checked and thinks they're fine.
We can't force people to do security. There definitely are (hopefully not publicly exposed) TLS 1.3 server implementations which always pick the same private key to do ECDHE, even though this means they lose forward secrecy - because they did not want actual security and we can't make them.
The Cloudflare blog entries on TLS 1.3 have some commentary (and link to more commentary from other sources) about this topic. Besides just getting rid of legacy cruft, simplifying the protocol helps limit downgrade attacks, reduces maintenance burden, and limits the observable behavior that various implementations might end up inadvertently depending on.
Learning a protocol by writing a toy client (or toy server) is a blast. It's so satisfying to see a real, production-quality server sending real responses to your little mess.
You'd probably enjoy work as a software pentester, where the docket --- at least for non-web-applications, which admittedly are the most common project if you don't specialize --- is almost entirely building tooling-grade implementations of random protocols so you can test for vulnerabilities.
E.g. https://google.github.io/clusterfuzzlite/ is likely not so great at protocols because that requires testing concurrent and distributed systems and TLAplus, which at least currently can't find side channels FWIU.
Hopefully I have actually insightful things to say about this fun toy. However before that
> elliptic curve “multiplication”, where n * P means “add P to itself n times”
Not very smoothly described but this is all multiplication meant for the natural numbers you learned in primary school too! Why is 7 x 7 = 49? Because if you start with zero and add 7, seven times, you get 49. Try it. This is an important and re-usable insight, it's part of a larger beautiful framework of mathematics and I believe is much better instructed via modern teaching of arithmetic in schools than "rote learning" of times tables did for my parents.
There are entire worlds of math locked away from you if you don't know your times table by heart. Algebra becomes particularly difficult if you do not know how to not only multiply, but factor, quickly.
So yeah, it's great that you know 9 * 6 is 9 added to itself 6 times, but you also have to learn the times table by rote if you want to get anywhere in math.
> it's great that you know 96 is 9 added to itself 6 times
For a while I tried to figure out what the joke was here, and then I realised all that's going on is HN emphasis asterisks
Anyway, no, it's well known that mathematicians are often terrible at this sort of "basic arithmetic". There are a bunch of mathematical disciplines where arithmetic is utterly irrelevant, but even in the disciplines where it seems like it ought to be important if you're an actual mathematician you don't care anyway. That's just mechanical stuff, use a calculator, we're busy doing mathematics.
It's a nice party trick (but you'll need to go a lot higher than 9 x 6 to impress anybody), and there are mathematicians who are good at arithmetic, but I assure you that by the time I gave up mathematics (just below degree level) there was no use for times tables, which I had never memorised, what was an obstacle was all the bloody integration. Pure, Mechanics, even the Statistics had integration in it by the time I was eighteen.
> When the HTTP response is done, we get these bytes: []byte{48, 13, 10, 13, 10, 23}. I don’t know what this is supposed to mean exactly but it seems to signal the end of the connection.
HTTP headers include Content-Length, so you should know when you get a truncated response, but TLS is supposed to be more general purpose, so it includes its own crypto secure end of connection indicator.
These bytes are an ASCII zero, then two carriage-return/linefeed pairs and then the byte with value 23, so that 0\r\n\r\n might just be HTTP saying we're done sending you this chunked HTTP response.
I think that 23 byte is the TLS content type, meaning "This is the actual application data you were transporting over TLS". In TLS 1.3 the original content type byte (from previous TLS versions) just always says 23 regardless so that middle boxes leave us alone, "Huh I guess everything is encrypted application data, I'm just a dumb middlebox so this isn't suspicious"
This new, inner content type, safely encrypted and thus invisible to the middleboxes, is at the end of the data because it allows a clever trick with zero padding. See, content type zero isn't a thing. So if we get some data and the last byte is zero, that's not the content type it must be padding, remove it, try again. Still zero? Still padding, remove that and so on. Because the zeroes were encrypted, an adversary can't tell they're padding, but because they're zero the real recipient can discard them safely, yet if we don't need padding we aren't wasting space for a "no padding" indicator, we just don't add any of those zeroes.
However, if 23 here is a content type, this means our intrepid TLS implementer got things a little wrong and all their HTTP data has stray 0x17 bytes at the end of each record received. I don't see code to handle that, but I may have missed it. It seems plausible that in terminal output you wouldn't notice?
I'm super impressed that they got it to work so quickly. What I hate about crypto is that is either wrong or right, there is no "mostly working" that you can identify easily and use as a debugging aid.
This drove me crazy when I was working on kTLS in FreeBSD. When I worked on other features (like getting checksum offload right in NIC firmware) there were easy tricks I could use for debugging, like sending a stream of all zeros. For crypto, it was basically back to first principals and code examination..
TLS 1.3 has some published test vectors where if you understand the inputs (which is tricky) and do the same operations, you'll end up with the same outputs as the test vectors and that should maybe work with a real server too.
Still, it helps a ton to get error logs (or step through debugging) from a real server to figure out what you messed up. For every value you send in a TLS handshake, it feels like you've got to send the length three different times, in different numbers of bytes and sometimes adding the bytes it takes to send the length.
But, if you can smash through all that, you can get a reasonably working TLS 1.3 client in about a week of fiddling. If your runtime has a decent api to validate certificates, you can call out to that too (the less you personally do with x.509, the better)
It's super nice. The only issue I have is you released it after I had done my TLS 1.3 work, so I couldn't have referred to it when it would have been most useful to me.
We can relate to that! It is extremely helpful for our current efforts to implement TLS 1.3 for the Botan crypto library. Especially to get a foothold with TLS and gain a feeling for the protocol structure. Amazing work!
I was one of the folks kinda down on the DNS server implementation...but this, this is awesome. TLS is a serious black box to 90%+ of even veteran programmers I meet.
To break it down and write a small implementation showing the steps is seriously both impressive and extremely generous. Kudos!
> This was pretty annoying to get working because I kept passing the wrong arguments to things.
I think this sentiment shows up twice in fact in the article.
I’m not sure what language the listings are in, but it looked static-type-ish.
Are there things the current batch of safety focused languages do to mitigate these types of ordering errors?
In Kotlin, Python, and Swift, we make it a practice to always use the keywords in calls if there are multiple arguments of the same type (e.g. two or more ints). I wish I could figure out how to configure any of their linters to enforce it better.
Despite its late bound unsafe nature, this was just never an error I made (argument ordering) back when I used to do a lot of Smalltalk with its interwoven keyword syntax.
I assumed passing the wrong argument to things to include wrong streams of bytes to HKDF and what not. There's a lot of keys to generate, and it's easy to get the bits of stuff not quite right.
> Are there things the current batch of safety focused languages do to mitigate these types of ordering errors?
Rust's NewType idiom can often help here. The author makes a bunch of arrays of bytes, like "Keys" is just all arrays of bytes with different meanings, and so if you mistakenly try to use the ServerPublic key as a ClientHandshake IV the types match up but it can't work. NewType would encourage you to have types named AESKey and InitialisationVector and X25519Public and so on, whereupon the compiler will tell you that your AESKey isn't an InitialisationVector etc.
If you use an IDE of some sort, your IDE will probably even prompt you while writing the code, you should put an AESKey here, not an InitialisationVector, something like Intellisense probably even scans your variables for the type and suggests you should either use my_sending_key or my_receiving_key and at that point it does seem like the programmer is only required to pay a modest amount of attention to produce programs which do what they intended.
NewType would apply for simple integers too. Rust not only has a type for the abstract concept of file handles (pretty common in high level languages, even C can do this with FILE*) but it has a type for Unix file descriptors, even though of course Unix file descriptors are just integers with some promises about which values they can have.
This is a fundamental idea in the language, Rust's str built-in is an array of bytes, but you could write [u8; N] and that's also an array of bytes, however str promises that the array of bytes is definitely UTF8 encoded Unicode text whereas u8 claims no such thing. At runtime they're identical, but in your program they're very different.
Rust gets to do this because it promises types don't actually exist at run time. The program you are writing, and which the compiler is compiling, may distinguish a CodeNumber, an ArticleID, a RowNum, a UserIdentifier, and a mere counter, but the machine code output just uses the same say, 64-bit machine register and 8 bytes of RAM to represent all five things interchangeably, the distinction existed only to make your program easier to understand.
This is certainly something interesting and worth trying, but I'd recommend starting with 1.2 first, because the handshake isn't encrypted so you can follow along more easily.
I only looked at the TLS 1.3 RFC for a few small things.
As someone who has written implementations of various widely-considered-nontrivial things (mostly image and video codecs, some crypto -- but not TLS, although upon seeing this article, that may change...) for self-educational purposes, I strongly recommend that the official standard be the first thing you look at and refer to during the process. It will clarify a lot of doubts, and in the case of TLS, the RFCs are relatively readable as far as standards go.
I’m not sure why the blocks have the size they do (maybe it’s so that each one will fit inside a TCP packet ???), but in theory I think they could be up to 65535 bytes, since their size field is 2 bytes.
That's due to either MTU or server-side write fragmentation, and the limit is 16K(+overhead). It is a stream to the application layer, but the record-layer protocol breaks it up into chunks of smaller limited size to allow for the integrity checks.
I assume a real TLS implementation would use a thread pool or coroutines or something to manage this.
That's a very strange assumption. Depending on the API, the ones I've used will either just read in a loop until the length is fulfilled or an error occurs; or return an "incomplete message" error meaning that the caller is the one to continue reading. Getting a partial read from a TCP socket is a very common "trap for young players" --- if you don't keep it in mind, you'll write code that appears to work on localhost or inside a fast LAN, but fails intermittently and sometimes mysteriously over the Internet or when there's more latency. You can never assume message boundaries when using TCP.
100% agree on reading the RFC. I've implemented various protocols as well (SMTP, IMAP, POP, FTP, some crypto and other things, all a couple decades ago), and the cases where I glossed over the RFC have always bitten me.
I wish common test frameworks and/or stdlibs made it easier to "fuzz" this. I've written more than one io.Reader implementation (Golang) that breaks up results from an underlying io.Reader according to a random seed.
>> I assume a real TLS implementation would use a thread pool or coroutines or something to manage this.
> That's a very strange assumption. Depending on the API, the ones I've used will either just read in a loop until the length is fulfilled or an error occurs;
When you're receiving from a network socket over the Internet, blocking a (real) thread until all the data arrives is pretty inefficient.
> or return an "incomplete message" error meaning that the caller is the one to continue reading.
That's pretty un-user-friendly.
There might be rare cases where a caller really needs precise control, but I'd expect any serious general-purpose library in a serious general-purpose language to "use a thread pool or coroutines or something to manage this". In the overwhelmingly common case what you want while waiting for data from the Internet is to yield to other fibers; a library should guide you towards doing the right thing and make the easy case easy.
> When you're receiving from a network socket over the Internet, blocking a (real) thread until all the data arrives is pretty inefficient.
Yes, but it's still how the overwhelming majority of network code functions. It's not really till the rise of libev/libuv and friends that people start moving to non-blocking as a default (I spent pretty much an entire year of my life convincing an AWS team to reclaim 4 GB of thread stacks by moving to non-blocking I/O).
>> or return an "incomplete message" error meaning that the caller is the one to continue reading.
> That's pretty un-user-friendly.
It's also how all non-blocking I/O worked prior to languages adding native async functionality.
> Yes, but it's still how the overwhelming majority of network code functions. It's not really till the rise of libev/libuv and friends that people start moving to non-blocking as a default
> It's also how all non-blocking I/O worked prior to languages adding native async functionality.
Meh. AOLServer showed that you could do better back when I was in high school. I've given up worrying about ecosystems that haven't got it yet.
When you're receiving from a network socket over the Internet, blocking a (real) thread until all the data arrives is pretty inefficient.
I'm perplexed at this statement, and the rest of your post in general. How is that "inefficient"? What else can the thread do if it needs to process data that hasn't arrived yet?
On the other hand, I've rewritten a lot of code where someone thought adding lots of complexity to a fundamentally simple task would somehow be better, and gotten some real performance gains from it. Almost always, the straightforward and simple approach wins.
> I'm perplexed at this statement, and the rest of your post in general. How is that "inefficient"? What else can the thread do if it needs to process data that hasn't arrived yet?
Some other task, such as processing data from a different socket that has arrived now. A library monopolising a thread for this one operation is like those DOS-era applications that expected to monopolise the whole CPU for their application - sure, it's "simpler", but it's vastly less efficient.
(And doing your network I/O async should not be complex or confusing in any serious modern language)
One thing I like about cryptography, and wish we could get politicians to understand, the mathematical steps to implement many of the core concepts are not hard. Obviously implementing them robustly such that they’re safe at scale is another matter, but “criminals” won’t have any difficulty getting access to secure implementations, and adding any kind of weakness only means that the implementations used by the public are more likely to have bugs, in addition to the legally mandated ones.
51 comments
[ 2.8 ms ] story [ 115 ms ] threadI was super excited to dive in an find the RSA code so I could preen about Bleichenbacher's vulnerability, but she neatly sidestepped that by doing ECDH. Then I thought, well, maybe it's P-curve ECDH and I can preen about invalid curve attacks on static-ephemeral ECDH. But nope, X25519! My point here, apart from making fun of myself for being the kind of person who would write this stuff on a message board, is TLS 1.3 is pretty solid.
The "block thing" that's kind of weird is, I assume, the TLS Record Layer. TLS runs (ordinarily) over TCP, which provides a non-demarcated stream of bytes. TLS breaks that stream up into records, and runs its handshake messages over one type of record, (say) HTTPS over another, and "alerts" over a third. The Record Layer also interacts, I think, with TLS's misbegotten compression system?
In the same vein as this project (but with different goals) is Trevor Perrin's tlslite, which is implemented in pure Python: https://github.com/trevp/tlslite
"PEP 543 – A Unified TLS API for Python" #interfaces (-2016) https://peps.python.org/pep-0543/#interfaces
As you say, it's also the case that you have different types of data and so the record type indicates what you are processing, but you'd need records even if you didn't do that.
Looking at GCM or any other authenticated encryption without Sum Types makes me sad. If decrypting the message fails, there is no plaintext and I feel like an API should be able to actually express that properly not return a plaintext [even if it's empty] and a flag saying "Don't use the plaintext". :(
The "block thing" is a necessary choice if we're going to have authenticated encryption, because authenticating costs us overhead. If we wanted to reliably authenticate individual bytes (so that we can have what naively feels like a stream of bytes) we'd be paying that overhead for each byte. That's pretty outrageous. On the other hand if I'm downloading a 10MB file with my 24kbps modem and none of that was authenticated yet because the authentication only happens once at the end of the encrypted data, I'm going to get very anxious about my download counter correctly showing 0% successfully downloaded for so long. So TLS authenticates whole records.
You don't need the TLS Record Layer in order to do authenticated encryption.
1. Something to delimit the boundaries of the message. 2. Some way to determine the nonce.
OTOH, the TLS 1.3 record layer is barely more than authenticated encryption,
- suites (as far as covering cipher + exchange + mac) are gone as a concept
- only 5 supported cipher algorithms (all AEAD, and all but one built on AES)
- only non-PSK key exchange is (EC)DHE
- no compression supported
- all the various key derivations are built around a single new primitive function, HKDF
I'm sure that given enough time the above options will balloon like we had with latter-day TLS 1.2 but at this point it feels so clean.
So while you can get code points for a vanity cipher, there not only isn't an appetite from the working group to endorse such ciphers, there is unlikely to ever be a scenario where a client or server would actually pick it unless you control both and choose to prefer the vanity cipher. Whereupon why bother?
The downgrade prevention in TLS 1.3 is pretty effective, so an attack which needs a downgrade also needs to defeat the downgrade protection, an additional hurdle. Obviously people should get rid of TLS 1.0 and TLS 1.1 but meanwhile I think the anti-downgrade story is the best it has ever been.
So I'm comfortable with my claim as it stood, we might never even need AES 256 in practice (but like ChaCha20 it's a SHOULD in the RFC for TLS 1.3).
I get that the intended use case is not paying the overhead of asymmetric crypto for embedded applications. However, that use case is already covered by something like WolfSSL stripped down to only support TLS 1.2 PSK, and they can still even use modern AEAD cihper suites like AES 256 GCM in TLS 1.2.
All of the TLS 1.3 key agreement mechanisms should have forward secrecy, even when pre-shared-keys are used. In some applications, there's too much temptation to get rid of that safety net for efficiency. We shouldn't clutter up the TLS 1.3 spec and implementations to cover that use case. Furthermore, engineers who go that route should have to explain to their boss solid engineering reasons why they're staying back at TLS 1.2. There are a small number of valid use cases for symmetric-only key exchange, but there are way more use cases where an engineer just thinks the reduced overhead is worth the risk, and their manager sees the TLS 1.3 box checked and thinks they're fine.
EDIT: https://www.imperialviolet.org/2016/05/16/agility.html is a good post about this.
Django-ca also does OCSP and certbot-compatible ACMEv2 w/ known limitations: https://django-ca.readthedocs.io/en/latest/acme.html#known-l...
E.g. https://google.github.io/clusterfuzzlite/ is likely not so great at protocols because that requires testing concurrent and distributed systems and TLAplus, which at least currently can't find side channels FWIU.
https://github.com/secfigo/Awesome-Fuzzing#network-protocol-...
OSS-Fuzz runs CloudFuzz[Lite?] for many open source repos and feeds OSV OpenSSF Vulnerability Format: https://github.com/google/osv#current-data-sources
> elliptic curve “multiplication”, where n * P means “add P to itself n times”
Not very smoothly described but this is all multiplication meant for the natural numbers you learned in primary school too! Why is 7 x 7 = 49? Because if you start with zero and add 7, seven times, you get 49. Try it. This is an important and re-usable insight, it's part of a larger beautiful framework of mathematics and I believe is much better instructed via modern teaching of arithmetic in schools than "rote learning" of times tables did for my parents.
So yeah, it's great that you know 9 * 6 is 9 added to itself 6 times, but you also have to learn the times table by rote if you want to get anywhere in math.
For a while I tried to figure out what the joke was here, and then I realised all that's going on is HN emphasis asterisks
Anyway, no, it's well known that mathematicians are often terrible at this sort of "basic arithmetic". There are a bunch of mathematical disciplines where arithmetic is utterly irrelevant, but even in the disciplines where it seems like it ought to be important if you're an actual mathematician you don't care anyway. That's just mechanical stuff, use a calculator, we're busy doing mathematics.
It's a nice party trick (but you'll need to go a lot higher than 9 x 6 to impress anybody), and there are mathematicians who are good at arithmetic, but I assure you that by the time I gave up mathematics (just below degree level) there was no use for times tables, which I had never memorised, what was an obstacle was all the bloody integration. Pure, Mechanics, even the Statistics had integration in it by the time I was eighteen.
HTTP headers include Content-Length, so you should know when you get a truncated response, but TLS is supposed to be more general purpose, so it includes its own crypto secure end of connection indicator.
I think that 23 byte is the TLS content type, meaning "This is the actual application data you were transporting over TLS". In TLS 1.3 the original content type byte (from previous TLS versions) just always says 23 regardless so that middle boxes leave us alone, "Huh I guess everything is encrypted application data, I'm just a dumb middlebox so this isn't suspicious"
This new, inner content type, safely encrypted and thus invisible to the middleboxes, is at the end of the data because it allows a clever trick with zero padding. See, content type zero isn't a thing. So if we get some data and the last byte is zero, that's not the content type it must be padding, remove it, try again. Still zero? Still padding, remove that and so on. Because the zeroes were encrypted, an adversary can't tell they're padding, but because they're zero the real recipient can discard them safely, yet if we don't need padding we aren't wasting space for a "no padding" indicator, we just don't add any of those zeroes.
However, if 23 here is a content type, this means our intrepid TLS implementer got things a little wrong and all their HTTP data has stray 0x17 bytes at the end of each record received. I don't see code to handle that, but I may have missed it. It seems plausible that in terminal output you wouldn't notice?
This drove me crazy when I was working on kTLS in FreeBSD. When I worked on other features (like getting checksum offload right in NIC firmware) there were easy tricks I could use for debugging, like sending a stream of all zeros. For crypto, it was basically back to first principals and code examination..
Still, it helps a ton to get error logs (or step through debugging) from a real server to figure out what you messed up. For every value you send in a TLS handshake, it feels like you've got to send the length three different times, in different numbers of bytes and sometimes adding the bytes it takes to send the length.
But, if you can smash through all that, you can get a reasonably working TLS 1.3 client in about a week of fiddling. If your runtime has a decent api to validate certificates, you can call out to that too (the less you personally do with x.509, the better)
Edit: link to vectors https://datatracker.ietf.org/doc/rfc8448/
Readers, i made a "squee" noise just now. Glad it was useful, this is exactly the sort of thing I wrote it for. <3
https://github.com/randombit/botan/pull/2922
To break it down and write a small implementation showing the steps is seriously both impressive and extremely generous. Kudos!
> This was pretty annoying to get working because I kept passing the wrong arguments to things.
I think this sentiment shows up twice in fact in the article.
I’m not sure what language the listings are in, but it looked static-type-ish.
Are there things the current batch of safety focused languages do to mitigate these types of ordering errors?
In Kotlin, Python, and Swift, we make it a practice to always use the keywords in calls if there are multiple arguments of the same type (e.g. two or more ints). I wish I could figure out how to configure any of their linters to enforce it better.
Despite its late bound unsafe nature, this was just never an error I made (argument ordering) back when I used to do a lot of Smalltalk with its interwoven keyword syntax.
It's golang. No keyword args.
Rust's NewType idiom can often help here. The author makes a bunch of arrays of bytes, like "Keys" is just all arrays of bytes with different meanings, and so if you mistakenly try to use the ServerPublic key as a ClientHandshake IV the types match up but it can't work. NewType would encourage you to have types named AESKey and InitialisationVector and X25519Public and so on, whereupon the compiler will tell you that your AESKey isn't an InitialisationVector etc.
If you use an IDE of some sort, your IDE will probably even prompt you while writing the code, you should put an AESKey here, not an InitialisationVector, something like Intellisense probably even scans your variables for the type and suggests you should either use my_sending_key or my_receiving_key and at that point it does seem like the programmer is only required to pay a modest amount of attention to produce programs which do what they intended.
NewType would apply for simple integers too. Rust not only has a type for the abstract concept of file handles (pretty common in high level languages, even C can do this with FILE*) but it has a type for Unix file descriptors, even though of course Unix file descriptors are just integers with some promises about which values they can have.
This is a fundamental idea in the language, Rust's str built-in is an array of bytes, but you could write [u8; N] and that's also an array of bytes, however str promises that the array of bytes is definitely UTF8 encoded Unicode text whereas u8 claims no such thing. At runtime they're identical, but in your program they're very different.
Rust gets to do this because it promises types don't actually exist at run time. The program you are writing, and which the compiler is compiling, may distinguish a CodeNumber, an ArticleID, a RowNum, a UserIdentifier, and a mere counter, but the machine code output just uses the same say, 64-bit machine register and 8 bytes of RAM to represent all five things interchangeably, the distinction existed only to make your program easier to understand.
I only looked at the TLS 1.3 RFC for a few small things.
As someone who has written implementations of various widely-considered-nontrivial things (mostly image and video codecs, some crypto -- but not TLS, although upon seeing this article, that may change...) for self-educational purposes, I strongly recommend that the official standard be the first thing you look at and refer to during the process. It will clarify a lot of doubts, and in the case of TLS, the RFCs are relatively readable as far as standards go.
I’m not sure why the blocks have the size they do (maybe it’s so that each one will fit inside a TCP packet ???), but in theory I think they could be up to 65535 bytes, since their size field is 2 bytes.
That's due to either MTU or server-side write fragmentation, and the limit is 16K(+overhead). It is a stream to the application layer, but the record-layer protocol breaks it up into chunks of smaller limited size to allow for the integrity checks.
I assume a real TLS implementation would use a thread pool or coroutines or something to manage this.
That's a very strange assumption. Depending on the API, the ones I've used will either just read in a loop until the length is fulfilled or an error occurs; or return an "incomplete message" error meaning that the caller is the one to continue reading. Getting a partial read from a TCP socket is a very common "trap for young players" --- if you don't keep it in mind, you'll write code that appears to work on localhost or inside a fast LAN, but fails intermittently and sometimes mysteriously over the Internet or when there's more latency. You can never assume message boundaries when using TCP.
I wish common test frameworks and/or stdlibs made it easier to "fuzz" this. I've written more than one io.Reader implementation (Golang) that breaks up results from an underlying io.Reader according to a random seed.
> That's a very strange assumption. Depending on the API, the ones I've used will either just read in a loop until the length is fulfilled or an error occurs;
When you're receiving from a network socket over the Internet, blocking a (real) thread until all the data arrives is pretty inefficient.
> or return an "incomplete message" error meaning that the caller is the one to continue reading.
That's pretty un-user-friendly.
There might be rare cases where a caller really needs precise control, but I'd expect any serious general-purpose library in a serious general-purpose language to "use a thread pool or coroutines or something to manage this". In the overwhelmingly common case what you want while waiting for data from the Internet is to yield to other fibers; a library should guide you towards doing the right thing and make the easy case easy.
Yes, but it's still how the overwhelming majority of network code functions. It's not really till the rise of libev/libuv and friends that people start moving to non-blocking as a default (I spent pretty much an entire year of my life convincing an AWS team to reclaim 4 GB of thread stacks by moving to non-blocking I/O).
>> or return an "incomplete message" error meaning that the caller is the one to continue reading.
> That's pretty un-user-friendly.
It's also how all non-blocking I/O worked prior to languages adding native async functionality.
> It's also how all non-blocking I/O worked prior to languages adding native async functionality.
Meh. AOLServer showed that you could do better back when I was in high school. I've given up worrying about ecosystems that haven't got it yet.
I'm perplexed at this statement, and the rest of your post in general. How is that "inefficient"? What else can the thread do if it needs to process data that hasn't arrived yet?
On the other hand, I've rewritten a lot of code where someone thought adding lots of complexity to a fundamentally simple task would somehow be better, and gotten some real performance gains from it. Almost always, the straightforward and simple approach wins.
Some other task, such as processing data from a different socket that has arrived now. A library monopolising a thread for this one operation is like those DOS-era applications that expected to monopolise the whole CPU for their application - sure, it's "simpler", but it's vastly less efficient.
(And doing your network I/O async should not be complex or confusing in any serious modern language)