added a post about cURL and SNI

Signed-off-by: Olivier Gayot <og@satcom1.com>

1 files changed, 240 insertions, 0 deletions

diff --git a/posts/curl-and-the-tls-sni-extension.rst b/posts/curl-and-the-tls-sni-extension.rst
new file mode 100644
index 0000000..6bea31a
--- /dev/null
+++ b/posts/curl-and-the-tls-sni-extension.rst
@@ -0,0 +1,240 @@
+.. title: cURL and the TLS SNI extension
+.. slug: curl-and-the-tls-sni-extension
+.. date: 2016-08-01 12:18:13 UTC+02:00
+.. tags:
+.. link:
+.. description: How to customize the Host header with TLS SNI extension.
+.. type: text
+
+Let's say that you are hosting multiple websites on the same port of a single
+machine which IPv4 address is 176.31.99.217. Both websites must be accessible
+via HTTP and HTTPS.
+
+You possess the NS domain *example.org* and it points to your machine.
+
+You are using virtualhosts to manage your differents websites. i.e. the *Host*
+HTTP header is analysed by your webserver so it can make the decision of which
+website it is going to serve.
+
+However, these virtualhosts are not pointing to your machine yet. Only
+*example.org* does.
+
+Here is an excerpt of a nginx configuration using three virtualhosts:
+
+* example.org (it will be served by default because of the **default_server** directive),
+* vhost1.example.org,
+* vhost2.example.org
+
+.. code-block:: nginx
+
+    server {
+        listen 0.0.0.0:80   default_server;
+        listen 0.0.0.0:443  ssl default_server;
+
+        server_name         example.org;
+
+        root                /srv/www/example.org/www;
+
+        ssl_certificate     ssl/example.org.crt;
+        ssl_certificate_key ssl/example.org.key;
+
+        try_files $uri $uri/ =404;
+    }
+
+    server {
+        listen 0.0.0.0:80;
+        listen 0.0.0.0:443  ssl;
+
+        server_name         vhost1.example.org;
+
+        root                /srv/www/vhost1.example.org/www;
+
+        ssl_certificate     ssl/vhost1.example.org.crt;
+        ssl_certificate_key ssl/vhost1.example.org.key;
+
+        try_files $uri $uri/ =404;
+    }
+
+    server {
+        listen 0.0.0.0:80;
+        listen 0.0.0.0:443  ssl;
+
+        server_name         vhost2.example.org;
+
+        root                /srv/www/vhost2.example.org/www;
+
+        ssl_certificate     ssl/vhost2.example.org.crt;
+        ssl_certificate_key ssl/vhost2.example.org.key;
+
+        try_files $uri $uri/ =404;
+    }
+
+For testing purposes:
+
+* the file */srv/www/example.org/www/index.html* contains the string **default**.
+* the file */srv/www/vhost1.example.org/www/index.html* contains the string **vhost1**.
+* the file */srv/www/vhost2.example.org/www/index.html* contains the string **vhost2**.
+
+HTTP
+----
+
+Let's try that out with HTTP::
+
+    $ curl http://example.org
+    default
+
+    $ curl http://176.31.99.217
+    default
+
+The default server is served because 172.31.99.217 does not match any
+virtualhost *server_name* directive::
+
+    $ curl http://vhost1.example.org
+    curl: (6) Could not resolve host: vhost1.example.org
+    $ curl http://vhost2.example.org
+    curl: (6) Could not resolve host: vhost2.example.org
+
+Since the DNS does not know about vhost1.example.org and vhost2.example.org, we
+can't test our websites this way.
+We are offered (at least) two possibilities:
+
+* adding **vhost1.example.org** and **vhost2.example.org** to the file */etc/hosts*.
+  Example::
+
+        $ cat /etc/hosts
+        127.0.0.1       localhost
+        176.31.99.217   vhost1.example.org
+        176.31.99.217   vhost2.example.org
+
+        $ curl http://vhost1.example.org
+        vhost1
+        $ curl http://vhost2.example.org
+        vhost2
+
+* specifying the **Host** header manually using the option **-H** of cURL. This
+  is the recommended way in most cases.
+  Example::
+
+        $ curl http://176.31.99.217 -H 'Host: vhost1.example.org'
+        vhost1
+        $ curl http://176.31.99.217 -H 'Host: vhost2.example.org'
+        vhost2
+
+  The **Host** header has been placed in the HTTP request. We can verify this way::
+
+        $ curl -v http://176.31.99.217 -H 'Host: vhost1.example.org'
+        * Rebuilt URL to: http://176.31.99.217/
+        * Hostname was NOT found in DNS cache
+        *   Trying 176.31.99.217...
+        * Connected to 176.31.99.217 (176.31.99.217) port 80 (#0)
+        > GET / HTTP/1.1
+        > User-Agent: curl/7.38.0
+        > Accept: */*
+        > Host: vhost1.example.org
+
+
+HTTPS
+-----
+
+Now what happens if we use the HTTPS version ?::
+
+    $ curl https://example.org
+    default
+
+    $ curl https://176.31.99.217
+    curl: (51) SSL: certificate subject name 'example.org' does not match target host name '176.31.99.217'
+
+The second command fails because the default webserver answered with its
+certificate, but it is not valid for *176.31.99.217*, only *example.org*.
+
+This is a normal behaviour. We can solve this issue by creating a certificate
+valid for *example.org* and *176.31.99.217* for example.
+
+But what about requesting our virtualhosts ? Well, pretty much the same as
+before (unless you didn't clear your */etc/hosts* file.::
+
+    $ curl https://vhost1.example.org
+    curl: (6) Could not resolve host: vhost1.example.org
+    $ curl https://vhost2.example.org
+    curl: (6) Could not resolve host: vhost2.example.org
+
+The good news is: it will still work if you modify your */etc/hosts* file (huray)::
+
+        $ cat /etc/hosts
+        127.0.0.1       localhost
+        176.31.99.217   vhost1.example.org
+        176.31.99.217   vhost2.example.org
+
+        $ curl https://vhost1.example.org
+        vhost1
+        $ curl https://vhost2.example.org
+        vhost2
+
+And the bad news is::
+
+        $ curl https://176.31.99.217 -H 'Host: vhost1.example.org'
+        curl: (51) SSL: certificate subject name 'example.org' does not match target host name '176.31.99.217'
+        $ curl https://176.31.99.217 -H 'Host: vhost2.example.org'
+        curl: (51) SSL: certificate subject name 'example.org' does not match target host name '176.31.99.217'
+
+So why does it fail ?
+
+Well, as we showed previously, the *Host* header is analysed to figure out which
+website to serve. Nevertheless, when HTTPS is being used, the first thing that
+the user agent and the server have to do is negociate the certificate. At this
+point, the HTTP headers have just NOT been sent yet.
+
+So how does the webserver decides which certificate to send ?
+
+.. note:: 10 years ago, it was simply not possible. The webservers had to send the same certificate for every virtualhost behind the same address and port.
+
+Nowadays, there is a TLS extension, SNI (which stands for Server Name
+Indication) that is supported by most browsers and operating systems. The TLS
+stack present in Windows XP does not support it. As a consequence, Internet
+Explorer under Windows XP cannot use SNI. But most other browsers use their own
+TLS stack and can thus use SNI even under XP.
+
+.. note:: See more on Wikipedia: https://en.wikipedia.org/wiki/Server_Name_Indication
+
+Briefly, the extension works by sending the server name along with the first
+TLS packet. This way, the remote server knows which certificate to reply with.
+Only then, after the certificate has been negociated, that the server can
+analyse the *Host* header.
+
+.. note:: The host header does not require to match with the server name that has been sent in the TLS packet.
+
+So what can we do ?
+
+* We can still use the */etc/hosts* file as previously stated.
+* We can also use the **--resolve** option of cURL (which does pretty much the
+  same thing as modyfiying the */etc/hosts* file (the 443 is the port being
+  used): example::
+
+        $ curl https://vhost1.example.org --resolve vhost1.example.org:443:176.31.99.217
+        > vhost1
+        $ curl https://vhost2.example.org --resolve vhost2.example.org:443:176.31.99.217
+        > vhost2
+
+* Or we can negociate one certificate that we know is good and then use the
+  *Host* header to get the content of another website::
+
+        $ curl https://example.org -H 'Host: vhost1.example.org'
+        > vhost1
+        $ curl https://example.org -H 'Host: vhost2.example.org'
+        > vhost2
+
+  We just negociated the certificate of *example.org* and got the content of
+  another virtualhost ! We can verify::
+
+        $ curl -v https://example.org -H 'Host: vhost1.example.org'
+        ...
+        > * Server certificate:
+        ...
+        > *     common name: example.org (matched)
+        ...
+        > *     SSL certificate verify ok.
+        > ...
+        > ...
+        vhost1
+
+That's all folks.