how to tell if SNI is causing TLS to not work (with LTE-M/NB-IoT)?

Question

I'm trying to get https going with XBee cellular in API mode and always get a transmit status frame with delivery status 85 (Unknown error) back. I'm trying to determine if SNI is the problem.

The user guide says SNI may break things, but it doesn't describe the exact circumstances or which error will result.

Based on this test:

$ openssl s_client -servername 66.230.114.174 -tlsextdebug -connect 66.230.114.174:443 2>/dev/null | grep "server name"
TLS server extension "server name" (id=0), len=0

The server does support SNI. But I would expect that most would at this point. Does support SNI == require SNI == XBee not work?

From other talk on this question:

https://serverfault.com/questions/506177/how-can-i-detect-if-a-server-is-using-sni-for-https

It sounds like SNI is client-initiated. But the XBee user guide makes it sound like a requirement imposed by servers.

Any advice how to diagnose if SNI is the problem?

Answer 1

Hi!

I looked into this, and I think the error you're getting is actually due to a cipher suite mismatch.

I was able to confirm using just "openssl s_client -tlsextdebug -connect 66.230.114.174:443" that your server is not (apparently) _requiring_ SNI, so that's not the issue.

Using "nmap -sV --script ssl-enum-ciphers -p 443 66.230.114.174" I listed the ciphers supported by your server, and these are what I got:

- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
- TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (dh 2048) - A
- TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (dh 2048) - A
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A
- TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (dh 2048) - A
- TLS_DHE_RSA_WITH_AES_256_CBC_SHA (dh 2048) - A

Comparing that to our documentation (https://www.digi.com/resources/documentation/Digidocs/90002258/Reference/r_tls_cipher.htm) as well as the u-blox manual, none of those cipher suites are supported. (It does support TLS_RSA_WITH_AES_256_CBC_SHA and _SHA256, but the _DHE does make it different.)

This means that the 0x85 error that you are getting is because the TLS stack in the u-blox modem is not able to negotiate a secure connection with the server. If you reconfigure the server to accept one of the supported cipher suites, it should work fine.

Comments

Ok, I think the server now supports the right suite, the output of the nmap command you gave now includes this line:

|       TLS_RSA_WITH_AES_256_CBC_SHA - strong

but I still get the same result (status 85).

I don't think the data is making it to the server at all, I tried this tcpdump command on the server:

tcpdump -i any port 443

and it shows nothing from the time I send a frame to the time the status 85 comes back.  It does show stuff when I browse to server with web browser.  The same command with port 80 does show stuff when I send a non-encrypted frame (frame tyep 0x20) via XCTU.

Is there some way to get more feedback besides the status 85?  A micropython recipe or something?  Thanks

---

Can you confirm what you have the following settings set to:

ATTL
AT$0

And can you reply with a copy of the API frame you are using to send? (Hex form - e.g. 7E 00 00 20 ...)

Looking at our internal notes, the 0x85 error when using TLS could indicate that the server certificate file is incorrect. Additionally, make sure the server certificate file you are using only has one certificate: http://cms.digi.com/resources/documentation/Digidocs/90002258/Reference/r_tls_cert_limits.htm

---

ATTL is 3 (TLSv1.2)
AT$0 is apache-selfsigned.crt;;

Using XCTU File Manager, I can retrieve from remote path /flash/certs/apache-selfsigned.crt

The 'file' utility reports it to be a PEM certificate:

$ file apache-selfsigned.crt
apache-selfsigned.crt: PEM certificate

The apache-selfsigned.crt file contains:

-----BEGIN CERTIFICATE-----
MIIEBjCCAu6gAwIBAgIJAJW6W1Zfto2wMA0GCSqGSIb3DQEBCwUAMIGWMQswCQYD
VQQGEwJVUzEPMA0GA1UECAwGQWxhc2thMRIwEAYDVQQHDAlGYWlyYmFua3MxEDAO
BgNVBAoMB05ldGZsb3cxFDASBgNVBAsMC0VuZ2luZWVyaW5nMRcwFQYDVQQDDA42
Ni4yMzAuMTE0LjE3NDEhMB8GCSqGSIb3DQEJARYSbm9ib2R5QG5vd2hlcmUuY29t
MCAXDTE5MDUwMTE5MjU1M1oYDzIwNzQwMzE1MTkyNTUzWjCBljELMAkGA1UEBhMC
VVMxDzANBgNVBAgMBkFsYXNrYTESMBAGA1UEBwwJRmFpcmJhbmtzMRAwDgYDVQQK
DAdOZXRmbG93MRQwEgYDVQQLDAtFbmdpbmVlcmluZzEXMBUGA1UEAwwONjYuMjMw
LjExNC4xNzQxITAfBgkqhkiG9w0BCQEWEm5vYm9keUBub3doZXJlLmNvbTCCASIw
DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALhy8IZFRmGl79XfVvZJrMkP5Y1d
Vstx0zc+U/KYMrh3Dg9/zcfBxj6RbK/OQcHbBofCWBpiV7SF+zvdyHciYZoFleE7
0d7dkCcda07Sg/ts8ZMRN3vWHqoDkSfSNu3QqwJsxEDL6wQB65fiBdhdsJ7A+NJK
kGiAIQwXNrrOsPFe3WF7g2yBUtZ3Tywf57rKHi9zkBOivQ3LVWw90ZsEtURMiW61
m9+tncwucom8X98lfNqT3CA9kOgXqe6YqewVGmp8qgH6pZdbhfR4JurAlqXCZyKI
yff0ujFmDHAbt12zOnniZss6ico19twrwEF8uhXGGc49mkHPyBT0OmExTu0CAwEA
AaNTMFEwHQYDVR0OBBYEFAXAnOKFM44NmR3qcs5cxyYf84t+MB8GA1UdIwQYMBaA
FAXAnOKFM44NmR3qcs5cxyYf84t+MA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcN
AQELBQADggEBAJb3G0rJ1vTKifG3d1qwuQwNrg90t8O190Kki0Gn6aBQodf6yw9y
vAeWai54ixaYuvcoKSIALEHbdN+LXl5Bb3q/CGH5knhuS+usguelrOyLWAYv1dAP
2R3fk1avEAASCjjN6pxScm9cFU70zAhFAJDHS1C0ynaVF8+fHCOWsQYwsw0ag+5g
Xsjtf7jTxyoefxVXg/FIuJ798+lgX7geTZOZZLKZQkPeq1PpiSgWi0IqeK5Pj/a7
eLSRXHv+/KqStd8q6FRAaZL25r7fqjOGQSOc9JcDOVh21+sjPQMKeelNlyan8h6m
638NgjHktty/iJchxF5TsJhbL3iFtRj7xjY=
-----END CERTIFICATE-----

I created this certificate using the procedure described here:
 https://www.digitalocean.com/community/tutorials/how-to-create-a-self-signed-ssl-certificate-for-apache-in-debian-9

It seems to behave as expected from the browser.

The frame hex is:

7E 00 15 23 C4 42 E6 72 AE 01 BB 00 00 00 00 74 65 73 74 5F 64 61 74 61 5B

---

Does the 0x85 status frame come back immediately, or does it come back after a long delay?

You could try the same connection logic using MicroPython, but I think the error you'll get there will be more or less the same, since API mode sockets and MicroPython use the same code in the end.

Looking at your certificate (using "openssl x509 -in apache-selfsigned.crt -text -noout") I see the following:

Subject: C = US, ST = Alaska, L = Fairbanks, O = Netflow, OU = Engineering, CN = 66.230.114.174, emailAddress = nobody@nowhere.com

I can't say that we've ever tested using certificates whose CN fields are IP addresses instead of FQDNs. Does the connection work if you leave the TLS profile empty (AT$0 ;;)? You may have found a bug in the u-blox modem with your certificate.

---

It does work in API mode with AT$0.  IIRC I did originally check in transparent mode but it didn't work that way.

I still can't get it to work with a certificate though.  I set up DNS service for the server: netflowmeter.com now points to 66.230.114.174 and is well propagated.

I regenerated the cert/key pair according to https://www.digitalocean.com/community/tutorials/how-to-create-a-self-signed-ssl-certificate-for-apache-in-debian-9 again, but with CN = netflowmeter.com.  The pair seems to work from the browser.

I uploaded the new certificate to the XBee and reset $0 to apache-selfsigned.crt;;.

The failure now seems different than it was in that the tcpdump -i any port 443 command on the server now always shows activity when XCTU is used to send a frame.  But a status 85 frame is still always returned on the XBee.

I appreciate all your help so far.  Unfortunately I don't know what to try next.  Thoughts?

---

The apache2 server says the following when the XBee tries to connect:

[Wed May 22 01:59:17.599721 2019] [ssl:info] [pid 20187:tid 2965328944] [client
162.244.250.195:37009] AH01964: Connection to child 74 established (server netflowmeter.com:443)
[Wed May 22 01:59:17.610306 2019] [ssl:debug] [pid 20187:tid 2965328944] ssl_engine_kernel.c(2143): [client 162.244.250.195:37009] AH02645: Server name not provided via TLS extension (using default/first virtual host)
[Wed May 22 01:59:18.180208 2019] [ssl:debug] [pid 20187:tid 2965328944] ssl_engine_io.c(1308): (70014)End of file found: [client 162.244.250.195:37009] AH02007: SSL handshake interrupted by system [Hint: Stop button pressed in browser?!]
[Wed May 22 01:59:18.181613 2019] [ssl:info] [pid 20187:tid 2965328944] [client
162.244.250.195:37009] AH01998: Connection closed to child 74 with abortive shutdown (server netflowmeter.com:443)

This makes it sound like the XBee is for some reason not actually sending the certificate file, or quitting early instead of sending full contents or something.  Coming from the browser I get instead:

[Wed May 22 01:25:14.990415 2019] [ssl:info] [pid 19584:tid 2973721648] [client
107.242.121.7:38087] AH01964: Connection to child 73 established (server netflowmeter.com:443)
[Wed May 22 01:25:14.993077 2019] [socache_shmcb:debug] [pid 19584:tid 2973721648] mod_socache_shmcb.c(532): AH00835: socache_shmcb_retrieve (0xf5 -> subcache 21)
[Wed May 22 01:25:14.994602 2019] [socache_shmcb:debug] [pid 19584:tid 2973721648] mod_socache_shmcb.c(884): AH00849: match at idx=0, data=0
[Wed May 22 01:25:14.995733 2019] [socache_shmcb:debug] [pid 19584:tid 2973721648] mod_socache_shmcb.c(542): AH00836: leaving socache_shmcb_retrieve successfully
[Wed May 22 01:25:14.997320 2019] [ssl:debug] [pid 19584:tid 2973721648] ssl_engine_kernel.c(2115): [client 107.242.121.7:38087] AH02043: SSL virtual host for servername netflowmeter.com found
[Wed May 22 01:25:14.999636 2019] [ssl:info] [pid 19584:tid 2965328944] [client
107.242.121.7:38523] AH01964: Connection to child 74 established (server netflowmeter.com:443)
...

So it's a different path on the server because SNI I guess.  I'm not sure how to simulate a non-SNI connection to check that from the browser or other tool yet.  I think the "default/first virtual host" alluded to should be the right one though, as /etc/apache2/sites-available/default-ssl.conf contains:
<IfModule mod_ssl.c>
        <VirtualHost _default_:443>
                ServerAdmin webmaster@localhost
                ServerName netflowmeter.com
...

If anyone at Digi would like access to this server to check out details I'd be happy to give it.  Without some more feedback from the client I don't know what else to try.

Answer 2

This ublox document:

https://www.u-blox.com/sites/default/files/AT-CommandsExamples_AppNote_%28UBX-13001820%29.pdf

discusses different certification levels. Level 0 is no validation, level 1 is a basic certificate check, level 2 adds an 'additional URL integrity check', and level 3 adds on to that a check about the certificate validity date.

Can anyone tell me which of these levels the XBee uses internally? Also, what does the additional URL integrity check entail?