Connect through a TLS broker

Before you begin

• TLS and authenticationProtect MQTT connections with TLS, authenticate clients and split topic permissions.
• MQTT in PDCommWhere you use MQTT in PDComm and how the simulator connects to a broker.

Goal

The PDComm simulator connects to brokers over plain TCP only. In this tutorial, you put a local broker (the edge) next to the simulator and let it forward the data to a remote broker (the site) over TLS with a client certificate.

 simulator ── plain TCP ──▶ edge broker ══ TLS bridge ══▶ site broker ══ TLS ══▶ operator
 (publisher)  127.0.0.1:1883  (edge.conf)  cert CN=pdcomm-edge  127.0.0.1:8883     cert CN=operator

• The site broker accepts only TLS on port 8883 and requires every client to present a certificate issued by the tutorial CA.
• The edge broker accepts plain MQTT on port 1883 of the loopback address and bridges the opcua/# topics to the site.
• An operator connects to the site with their own certificate and receives the data.

Both brokers run on one computer here. In a real deployment, only the site address and the certificate names change.

Before you begin

• Mosquitto 2.x and mosquitto_sub
• The OpenSSL command-line tool. The certificate commands work with both OpenSSL 3 and the LibreSSL that ships with macOS; the openssl s_client options in step 8 assume OpenSSL 3.
• The PDComm OPC UA simulator package and publisher-scenario.json from OPC UA PubSub over MQTT
• Four terminal windows, and no other broker on ports 1883 and 8883

Steps

1. Create a working directory and save five files in it:

   mkdir mqtt-tls
   cd mqtt-tls
   cp ../mqtt-lab/publisher-scenario.json .

   These are the certificate extensions. The subjectAltName of the broker certificate lists the names and addresses that clients connect to.

   server.ext

   # X.509 extensions for the broker certificate. The subjectAltName entries
   # must match the host name or address that clients and bridges connect to.
   basicConstraints = critical, CA:FALSE
   keyUsage = critical, digitalSignature, keyEncipherment
   extendedKeyUsage = serverAuth
   subjectAltName = DNS:localhost, IP:127.0.0.1

   client.ext

   # X.509 extensions for client certificates (the edge bridge and operators).
   basicConstraints = critical, CA:FALSE
   keyUsage = critical, digitalSignature, keyEncipherment
   extendedKeyUsage = clientAuth

   This is the site broker configuration:

   site.conf

   # Site broker: MQTT over TLS on port 8883. Every client must present a
   # certificate issued by the tutorial CA; its common name becomes the user name.
   listener 8883 127.0.0.1
   cafile ca.crt
   certfile site.crt
   keyfile site.key
   require_certificate true
   use_identity_as_username true
   allow_anonymous false
   
   persistence false
   log_dest stdout

   Setting	Meaning
   cafile, certfile, keyfile	The CA that verifies client certificates, the broker certificate, and the broker’s private key
   require_certificate true	Drop clients that present no certificate during the TLS handshake.
   use_identity_as_username true	Use the CN of the client certificate as the user name.
   allow_anonymous false	Refuse connections without a user name.

   This is the edge broker configuration:

   edge.conf

   # Edge broker: plain TCP for local clients such as the PDComm simulator,
   # and a TLS bridge that forwards their OPC UA topics to the site broker.
   listener 1883 127.0.0.1
   allow_anonymous true
   persistence false
   log_dest stdout
   
   connection site
   address 127.0.0.1:8883
   bridge_protocol_version mqttv50
   bridge_cafile ca.crt
   bridge_certfile edge.crt
   bridge_keyfile edge.key
   remote_clientid pdcomm-edge-bridge
   topic opcua/# out 1

   Setting	Meaning
   connection site	Start a bridge named site.
   address 127.0.0.1:8883	The site broker the bridge connects to; the address must appear in the site certificate’s subjectAltName.
   bridge_protocol_version mqttv50	Bridge with MQTT 5.0 so that the Content Type and user properties pass through unchanged.
   bridge_cafile, bridge_certfile, bridge_keyfile	The CA that verifies the site certificate, and the bridge’s client certificate and private key
   topic opcua/# out 1	Send local opcua/# messages to the site (out) at QoS 1.

2. Create the tutorial CA and three certificates:

   openssl req -x509 -new -newkey rsa:2048 -nodes -sha256 -days 365 \
     -subj "/CN=PDComm Tutorial CA" -keyout ca.key -out ca.crt
   openssl req -new -newkey rsa:2048 -nodes -subj "/CN=localhost" -keyout site.key -out site.csr
   openssl x509 -req -in site.csr -CA ca.crt -CAkey ca.key -CAcreateserial -sha256 -days 365 \
     -extfile server.ext -out site.crt
   openssl req -new -newkey rsa:2048 -nodes -subj "/CN=pdcomm-edge" -keyout edge.key -out edge.csr
   openssl x509 -req -in edge.csr -CA ca.crt -CAkey ca.key -CAcreateserial -sha256 -days 365 \
     -extfile client.ext -out edge.crt
   openssl req -new -newkey rsa:2048 -nodes -subj "/CN=operator" -keyout operator.key -out operator.csr
   openssl x509 -req -in operator.csr -CA ca.crt -CAkey ca.key -CAcreateserial -sha256 -days 365 \
     -extfile client.ext -out operator.crt
   chmod 600 *.key

   Check that every certificate chains to the CA:

   openssl verify -CAfile ca.crt site.crt edge.crt operator.crt

   site.crt: OK
   edge.crt: OK
   operator.crt: OK

3. Work in the mqtt-tls directory in all four terminals. In the first terminal, start the site broker:

   mosquitto -c site.conf

   Config loaded from site.conf.
   Opening ipv4 listen socket on port 8883.
   mosquitto version 2.1.2 running

4. In the second terminal, connect to the site with the operator certificate and subscribe:

   mosquitto_sub -h 127.0.0.1 -p 8883 --cafile ca.crt --cert operator.crt --key operator.key \
     -V 5 -t 'opcua/#' -F '%t qos=%q retain=%r content-type=%C user-properties=%P %l bytes'

   The site log shows the TLS 1.3 connection and the user name taken from the certificate:

   New client connected from 127.0.0.1:53292 as auto-A75E2731-… (p5, c1, k60, u'operator').
   Client auto-A75E2731-… negotiated TLSv1.3 cipher TLS_AES_256_GCM_SHA384

5. In the third terminal, start the edge broker:

   mosquitto -c edge.conf

   Opening ipv4 listen socket on port 1883.
   Connecting bridge site (127.0.0.1:8883)
   mosquitto version 2.1.2 running

   The bridge connection appears in the site log, with pdcomm-edge, the CN of the edge certificate, as the user name:

   New client connected from 127.0.0.1:53293 as pdcomm-edge-bridge (p5, c0, k60, u'pdcomm-edge').
   Client pdcomm-edge-bridge negotiated TLSv1.3 cipher TLS_AES_256_GCM_SHA384

6. In the fourth terminal, run the simulator publisher. It connects to the edge at 127.0.0.1:1883 over plain TCP:

   "$PDCOMM_APPS/bin/pdcomm-opcua-headless" --scenario publisher-scenario.json

   In a new terminal without PDCOMM_APPS, first run PDCOMM_APPS=<simulator package directory>. The scenario exits with code 0, and both messages reach the operator in the second terminal over TLS:

   opcua/uadp/data/100/line1 qos=0 retain=0 content-type=application/opcua+uadp user-properties=UAMessageType:ua-data 52 bytes
   opcua/uadp/data/100/line1 qos=0 retain=0 content-type=application/opcua+uadp user-properties=UAMessageType:ua-data 52 bytes

   The Content Type and UAMessageType survive the bridge, so an OPC UA subscriber on the site side can accept these messages too.

7. Check that a connection without a certificate is refused. Run this in the fourth terminal:

   mosquitto_sub -h 127.0.0.1 -p 8883 --cafile ca.crt -t 'opcua/#' -C 1 -W 3

   Error: The connection was lost

   The site log records why:

   OpenSSL Error while trying to get the error[0]: error:0A0000C7:SSL routines::peer did not return a certificate
   Client 127.0.0.1 [127.0.0.1:53299] disconnected: Protocol error.

8. You can also check the TLS setup on its own, without MQTT, with openssl s_client:

   openssl s_client -connect 127.0.0.1:8883 -CAfile ca.crt -cert operator.crt -key operator.key \
     -verify_ip 127.0.0.1 -verify_return_error -brief < /dev/null

   The key lines of the output are:

   CONNECTION ESTABLISHED
   Protocol version: TLSv1.3
   Ciphersuite: TLS_AES_256_GCM_SHA384
   Peer certificate: CN=localhost

   -verify_ip checks that the IP address appears in the certificate’s subjectAltName. When you connect by host name, use -verify_hostname <name> instead.

Clean up

1. Stop mosquitto_sub in the second terminal with Ctrl+C.
2. Stop the brokers in the third and first terminals with Ctrl+C.
3. Delete the mqtt-tls directory when you’re done, and don’t leave a copy of ca.key anywhere.

Common errors

Output	Cause	Fix
Error: The connection was lost, and peer did not return a certificate in the site log	The client presented no certificate.	Pass --cert and --key.
Error: Protocol error, and tlsv1 alert unknown ca in the site log	The client doesn’t trust the site certificate.	Pass --cafile ca.crt.
certificate verify failed and Bridge site next backoff will be … ms in the edge log	The bridge couldn’t verify the site certificate.	Check that bridge_cafile and address match the certificate.
MQTT broker connection failed: MQTT transport I/O: ConnectionReset from the simulator	You pointed the simulator at the TLS port (8883).	Point the simulator at the edge’s plain port (1883).

TLS problems explains each message in more detail.

Caution

Open the edge broker’s plain listener on the loopback address (127.0.0.1) only. If other computers can connect in plain text, the TLS setup no longer protects anything.