Tunnels without the ceremony

Lorem ipsum dolor sit amet consectetur. A tunnel is meant to be the boring part of a network: you bring it up, it carries packets, you forget about it. In practice the bringing up is where most of the engineering goes, and most of that engineering is negotiation about things neither end actually disagrees on.

This is a walk through what we removed, in the order we removed it, and what each removal cost us.

What a handshake costs

Nullam sed tellus pharetra egestas. Every round trip is a full path latency, and a path that crosses an ocean is not short. The interesting number is not the count of round trips but where they sit relative to the first byte the user asked for.

The three round trips

Sed tellus pharetra egestas vitae. Convention gives you one trip to agree on a cipher, one to prove identity, and one to bind the session to a key. Two of the three are answering a question we already know the answer to, because both ends run our build.

• The cipher list has one entry, and it is not negotiable.
• The identity is a device certificate that was issued at enrolment.
• The session binding is the only part that has to be fresh.

Where the time actually goes

Egestas vitae consectetur adipiscing. Measured against a warm path, the negotiation is around a fifth of time-to-first-byte. Measured against a cold path with a captive portal in front of it, the negotiation is most of the wait, because every retry pays for it again.

Moving the ceremony off the hot path

Adipiscing elit sed do eiusmod. The device already holds a certificate, so the parts of the handshake that establish who is speaking can happen at enrolment rather than at connect. What is left at connect is a key exchange and nothing else.

1. Enrol once, out of band, and keep the material on the device.
2. At connect, exchange fresh keys and start sending.
3. Re-key on a timer, not on a reconnect.

Session resumption, without the session store

Tempor incididunt ut labore et dolore. A resumption cache is a piece of shared state, and shared state across twenty-three locations is a replication problem wearing a performance hat. Deriving the resumption secret from material both ends already hold removes the cache and the problem with it.

Reading the wire

Magna aliqua ut enim ad minim. The client opens with a single fixed-length frame; if you are watching a capture, the frame reads as HELLO followed by a nonce and nothing else. There is no version byte, because there is no second version to distinguish.

$ doxxctl tunnel status --verbose
state        up
transport    wireslammer/quic
rtt          38ms
rekey in     00:04:12

Nothing in this paragraph is a quotation from a person. It is here because a blockquote with no attribution line is a plain quote, and the renderer has to draw it as one rather than as a pull-quote with an empty byline.

What we gave up

Veniam quis nostrud exercitation ullamco. Interoperability, mostly. A tunnel that assumes both ends run the same build cannot speak to a third-party client, and that is a real cost rather than a rounding error — see the DNS root piece for the same trade taken in a different layer.

Laboris nisi ut aliquip ex ea commodo. The rest of the trade is documented where it is made, which is the point of writing any of this down.