In the last two posts Configuring a hierarchy of IoT Edge Devices at Home Part 1 – Configuring the IT Proxy and Configuring a hierarchy of IoT Edge Devices at Home Part 2 – Configuring the Enterprise Network (IT) we have set up the proxy and the top layer of the hierarchical IoT Edge network. This post will describe how to set up and configure the first nested layer in the Purdue network architecture for manufacturing – the Business Planning and Logistics (IT) layer. So, let’s get going!

A lot of the steps to configure the IoT Edge runtime on the device are repeating from the previous post. Nevertheless, though, let’s go over them again.

Configuring the Network for Layer 4 Device

The tricky part with the Layer 4 device is that it should have no Internet access. The problem with that is that you will need Internet access to install the IoT Edge runtime. In a typical scenario, the Layer 4 device will come with the IoT Edge runtime pre-installed and you will only need to plug in the device and configure it to talk to its parent. For our scenario though, we need to have the device Internet-enabled while installing the IoT Edge and lock it down after that. Here is the initial dhcpcd.conf configuration for the Layer 4 device:

interface eth0
static ip_address=10.16.6.4/16
static routers=10.16.8.4
static domain_name_servers=1.1.1.1

The only difference in the network configuration is the IP address of the device. This configuration will allow us to download the necessary software to the device before we restrict its access.

While speaking of necessary software, let’s install the netfilter-persistent and iptables-persistent and have them ready for locking the device networking later on:

sudo DEBIAN_FRONTEND=noninteractive apt install -y netfilter-persistent iptables-persistent

Create the Azure Cloud Resources

We already have created the resource group and the IoT Hub resource in Azure. The only new resource that we need to create is the IoT Edge device for Layer 4. One difference here is that, unlike the Layer 5 device, the Layer 4 device will have a parent. And this parent will be the Layer 5 device. Use the following command to create the Layer 4 device

az iot hub device-identity create --device-id L4-edge-pi --edge-enabled --hub-name tsm-ioth-eus-test
az iot hub device-identity parent set --device-id L4-edge-pi --parent-device-id L5-edge-pi --hub-name tsm-ioth-eus-test
az iot hub device-identity connection-string show --device-id L5-edge-pi --hub-name tsm-ioth-eus-test

The first command creates the IoT Edge device for Layer 4. The second command sets the Layer 5 device as the parent for the Layer 4 device. And, the third command returns the connection string for the Layer 4 device that we can use for the IoT Edge runtime configuration.

Installing Azure IoT Edge Runtime on the Layer 4 Device

Surprisingly to me, it the time between posting my last post and starting this one, the Tutorial to Create a Hierarchy of IoT Edge Devices changed. I will still use the steps from my previous post to provide consistency.

Creating Certificates

We will start again with the creation of certificates – this time for the Layer 4 device. We already have the root and the intermediate certificates for the chain. The only certificate that we need to create is the Layer 4 device certificate. We will use the following command for that:

./certGen.sh create_edge_device_ca_certificate "l4_certificate"

You should see the new certificates in the <WORKDIR>/certs folder:

drwxrwxrwx 1 toddysm toddysm 4096 Apr 16 10:46 .
drwxrwxrwx 1 toddysm toddysm 4096 Apr 16 10:46 ..
...
-rwxrwxrwx 1 toddysm toddysm 5891 Apr 16 10:46 iot-edge-device-ca-l4_certificate-full-chain.cert.pem
-r-xr-xr-x 1 toddysm toddysm 1931 Apr 16 10:46 iot-edge-device-ca-l4_certificate.cert.pem
-rwxrwxrwx 1 toddysm toddysm 7240 Apr 16 10:46 iot-edge-device-ca-l4_certificate.cert.pfx
...

The private keys are in the <WORKDIR>/private folder together with the root and intermediate keys:

drwxrwxrwx 1 toddysm toddysm 4096 Apr 16 10:46 .
drwxrwxrwx 1 toddysm toddysm 4096 Apr 16 10:46 ..
-r-xr-xr-x 1 toddysm toddysm 3326 Apr  8 14:54 azure-iot-test-only.intermediate.key.pem
-r-xr-xr-x 1 toddysm toddysm 3326 Apr  8 14:54 azure-iot-test-only.root.ca.key.pem
-r-xr-xr-x 1 toddysm toddysm 3243 Apr 16 10:46 iot-edge-device-ca-l4_certificate.key.pem
...

As before, we need to upload the relevant certificates to the Layer 4 device. From the <WORKDIR> folder on the workstation use the following commands:

scp ./certs/azure-iot-test-only.root.ca.cert.pem pi@pi-l4:.
scp ./certs/iot-edge-device-ca-l4_certificate-full-chain.cert.pem pi@pi-l4:.
scp ./private/iot-edge-device-ca-l4_certificate.key.pem pi@pi-l4:.

Connect to the Layer 4 device with ssh pi@pi-l4 and install the root CA using:

sudo cp ~/azure-iot-test-only.root.ca.cert.pem /usr/local/share/ca-certificates/azure-iot-test-only.root.ca.cert.pem.crt
sudo update-ca-certificates

Verify that the cert is installed with:

ls /etc/ssl/certs/ | grep azure

We will also move the device certificate chain and the private key to the /var/secrets folder:

sudo mkdir /var/secrets
sudo mkdir /var/secrets/aziot
sudo mv iot-edge-device-ca-l4_certificate-full-chain.cert.pem /var/secrets/aziot/
sudo mv iot-edge-device-ca-l4_certificate.key.pem /var/secrets/aziot/

We are all set to install and configure the Azure IoT Edge runtime.

Configuring the IoT Edge Runtime

As mentioned before the steps are described in Install or uninstall Azure IoT Edge for Linux and here is the configuration that we need to use in /etc/aziot/config.tomlfile:

hostname = "10.16.6.4"

parent_hostname = "10.16.7.4"

trust_bundle_cert = "file:///etc/ssl/certs/azure-iot-test-only.root.ca.cert.pem.pem"

[provisioning]
source = "manual"
connection_string = "HostName=tsm-ioth-eus-test.azure-devices.net;DeviceId=L4-edge-pi;SharedAccessKey=s0m#VeRYcRYpT1c$tR1nG"

[agent]
name = "edgeAgent"
type = "docker"
[agent.config]
image = "10.16.7.4:<port>/azureiotedge-agent:1.2"

[edge_ca]
cert = "file:///var/secrets/aziot/iot-edge-device-ca-l4_certificate-full-chain.cert.pem"
pk = "file:///var/secrets/aziot/iot-edge-device-ca-l4_certificate.key.pem"

Note two differences in this configuration compared to the configuration for the Layer 5 device:

  1. There is a parent_hostname configuration that has the Layer 5 device’s IP address as a value.
  2. The agent image is pulled from the parent registry 10.16.7.4:<port>/azureiotedge-agent:1.2 instead from mcr.microsoft.com (don’t forget to remove/change the <port> in the configuration depending on how you set up the parent registry).

Also, if the registry running on your Layer 5 device requires authentication, you will need to provide credentials in the [agent.config.auth] section.

Restricting the Network for the Layer 4 Device

Now, that we have the IoT Edge runtime configured, we need to restrict the traffic to and from the Layer 4 device. Here is what are we going to do:

  • We will allow SSH connectivity only from the IP addresses in the Demo/Support network, i.e. where our workstation is. However, we will use a stricter CIDR to allow only  10.16.9.x addresses. Here are the commands that will make that configuration:
    sudo iptables -A INPUT -i eth0 -p tcp --dport 22 -s 10.16.9.0/24 -m state --state NEW,ESTABLISHED -j ACCEPT
    sudo iptables -A OUTPUT -p tcp --sport 22 -d 10.16.9.0/24 -m state --state ESTABLISHED -j ACCEPT
  • We will enable connectivity on the following ports 8080 (or whatever the registry port is), 8883, 443, and 5671 to the Layer 5 network only, i.e. 10.16.7.x addresses. Here the commands that will make that configuration:
    sudo iptables -A OUTPUT -p tcp --dport <registry-port> -d 10.16.7.0/24 -m state --state NEW,ESTABLISHED -j ACCEPT
    sudo iptables -A INPUT -i eth0 -p tcp --sport <registry-port> -s 10.16.7.0/24 -m state --state ESTABLISHED -j ACCEPT
    sudo iptables -A OUTPUT -p tcp --dport 8883 -d 10.16.7.0/24 -m state --state NEW,ESTABLISHED -j ACCEPT
    sudo iptables -A INPUT -i eth0 -p tcp --sport 8883 -s 10.16.7.0/24 -m state --state ESTABLISHED -j ACCEPT
    sudo iptables -A OUTPUT -p tcp --dport 5671 -d 10.16.7.0/24 -m state --state NEW,ESTABLISHED -j ACCEPT
    sudo iptables -A INPUT -i eth0 -p tcp --sport 5671 -s 10.16.7.0/24 -m state --state ESTABLISHED -j ACCEPT
    sudo iptables -A OUTPUT -p tcp --dport 443 -d 10.16.7.0/24 -m state --state NEW,ESTABLISHED -j ACCEPT
    sudo iptables -A INPUT -i eth0 -p tcp --sport 443 -s 10.16.7.0/24 -m state --state ESTABLISHED -j ACCEPT
    
  • We will also enable connectivity on the following ports 8080 (or whatever the registry port is), 8883, 443, and 5671 from the OT Proxy network only, i.e. 10.16.5.x addresses. Here the commands that will make that configuration:
    sudo iptables -A OUTPUT -p tcp --dport <registry-port> -d 10.16.5.0/24 -m state --state ESTABLISHED -j ACCEPT
    sudo iptables -A INPUT -i eth0 -p tcp --sport <registry-port> -s 10.16.5.0/24 -m state --state NEW,ESTABLISHED -j ACCEPT
    sudo iptables -A OUTPUT -p tcp --dport 8883 -d 10.16.5.0/24 -m state --state ESTABLISHED -j ACCEPT
    sudo iptables -A INPUT -i eth0 -p tcp --sport 8883 -s 10.16.5.0/24 -m state --state NEW,ESTABLISHED -j ACCEPT
    sudo iptables -A OUTPUT -p tcp --dport 5671 -d 10.16.5.0/24 -m state --state ESTABLISHED -j ACCEPT
    sudo iptables -A INPUT -i eth0 -p tcp --sport 5671 -s 10.16.5.0/24 -m state --state NEW,ESTABLISHED -j ACCEPT
    sudo iptables -A OUTPUT -p tcp --dport 443 -d 10.16.5.0/24 -m state --state ESTABLISHED -j ACCEPT
    sudo iptables -A INPUT -i eth0 -p tcp --sport 443 -s 10.16.5.0/24 -m state --state NEW,ESTABLISHED -j ACCEPT
    
  • Last, we need to disable any other traffic to and from the Layer 4 device. Here the commands for that:
    sudo iptables -P INPUT DROP
    sudo iptables -P OUTPUT DROP
    sudo iptables -P FORWARD DROP
    

Don’t forget to save the configuration and restart the device with:

sudo netfilter-persistent save
sudo systemctl reboot

Deploying Modules on the Layer 4 Device

Similar to the deployment of modules on the Layer 5 Device, we need to deploy the standard $edgeAgent and $edgeHub modules as well as a registry module. For this layer, I experimented with the API proxy module. Hence, the configurations include also the $IoTEdgeApiProxy module. There is also an API Proxy configuration that you need to set on the $IoTEdgeApiProxy module’s twin – for details, take a look at Configure the API proxy module for your gateway hierarchy scenario. Here are the Gists that you can use for deploying the modules on the Layer 4 device:

By using the API proxy, you can also leverage the certificates deployed on the device to connect to the registry over HTTPS. Thus, you don’t need to configure Docker on the clients to connect to an insecured registry.

Testing the Registry Module

To test the registry module, use the following commands:

docker login -u <sync_token_name> -p <sync_token_password> 10.16.6.4:<port_if_any>
docker pull 10.16.6.4:<port_if_any>/azureiotedge-simulated-temperature-sensor:1.0

Now, we have set up the first nested layer (Layer 4) for our nested IoT Edge infrastructure. In the next post, I will describe the steps to set up the OT Proxy and the second nested layer for IoT Edge (Layer 3).

Image by ejaugsburg from Pixabay

In my last post, I started explaining how to configure the IoT Edge device hierarchy’s IT Proxy. This post will go one layer down and set up the Layer 5 device from the Purdue model for manufacturing networks.

Reconfiguring The Network

While implementing network segregation in the cloud is relatively easy, implementing it with a limited number of devices and a consumer-grade network switch requires a bit more design. In Azure, the routing between subnets within a VNet is automatically configured using the subnet gateways. In my case, though, the biggest challenge was connecting each Raspberry Pi device to two separate subnets using the available interfaces – one WiFi and one Ethernet. In essence, I couldn’t use the WiFi interface because then I was limited to my Eero’s (lack of) capabilities (Well, I have an idea how to make this work, but this will be a topic of a future post :)). Thus, my only option was to put all devices in the same subnet and play with the firewall on each device to restrict the traffic. Here is how the picture looks like.

To be able to connect to each individual device from my laptop (i.e. playing the role of the jumpbox from the Azure IoT Edge for Industrial IoT sample), I had to configure a second network interface on it and give it an IP address from the 10.16.0.0/16 network (the Workstation in the picture above). There are multiple ways to do that with easiest one to buy a USB networking dongle and connect it to the switch with the rest of the devices. One more thing that will be helpful to do to speed up the work is to edit the /etc/hosts file on my laptop and add DNS names for each of the devices:

10.16.8.4       pi-itproxy
10.16.7.4       pi-l5
10.16.6.4       pi-l4
10.16.5.4       pi-otproxy
10.16.4.4       pi-l3
10.16.3.4       pi-opcua

The next thing I had to do is to go back to the IT Proxy and change the subnet mask to enable a broader addressing range. Connect to the IT Proxy device that we configured in Configuring a hierarchy of IoT Edge Devices at Home Part 1 – Configuring the IT Proxy using ssh pi@10.16.8.4. Then edit the DHCP configuration file with sudo vi /etc/dhcpcd.conf and change the subnet mask from /24 to /16:

interface eth0
static ip_address=10.16.8.4/16

Now, the IT Proxy is configured to address the broader 10.16.0.0/16 network. To restrict the communication between the devices, we will configure each individual device’s firewall.

Now, getting back to the Layer 5 configuration. Normally, the Layer 5 device is configured to have access to the Internet via the IT Proxy as a gateway. Edit the DHCP configuration file with sudo vi /etc/dhcpcd.conf and add the following at the end:

interface eth0
static ip_address=10.16.7.4/16
static routers=10.16.8.4
static domain_name_servers=1.1.1.1

Note that I added the Cloudflare’s DNS server to the list of DNS servers. The reason for that is that the proxy device will not do DNS resolution. You can also configure it with your home network’s DNS server. This should be enough for now, and we can start installing the IoT Edge runtime on the Layer 5 device.

Testing from my laptop, I am able to connect to both devices using their 10.16.0.0/16 network IP addresses:

ssh pi@pi-itproxy

connects me to the IT Proxy, and:

ssh pi@pi-l5

connects me to the Layer 5 Iot Edge device.

Create the Azure Cloud Resources

Before we start installing the Azure IoT Edge runtime on the device, we need to create an Azure IoT Hub and register the L5 device with it. This is well described in Microsoft’s documentation explaining how to deploy code to a Linux device. Here are the Azure CLI commands that will create the resource group and the IoT Hub resources:

az group create --name tsm-rg-eus-test --location eastus
az iot hub create --resource-group tsm-rg-eus-test --name tsm-ioth-eus-test --sku S1 --partition-count 4

Next is to register the IoT Edge device and get the connection string. Here the commands:

az iot hub device-identity create --device-id L5-edge-pi --edge-enabled --hub-name tsm-ioth-eus-test
az iot hub device-identity connection-string show --device-id L5-edge-pi --hub-name tsm-ioth-eus-test

The last command will return the connection string that we should use to connect the new device to Azure IoT Hub. It has the following format:

{
  "connectionString": "HostName=tsm-ioth-eus-test.azure-devices.net;DeviceId=L5-edge-pi;SharedAccessKey=s0m#VeRYcRYpT1c$tR1nG"
}

Save the connection string because we will need it in the next section to configure the Layer 5 device.

Installing Azure IoT Edge Runtime on the Layer 5 Device

Installing the Azure IoT Edge is described in the Tutorial: Create a hierarchy of IoT Edge devices article. The tutorial describes how to build a hierarchy with two devices only. The important part of the nested configuration is to generate the certificates and transfer them to the devices. So, let’s go over this step by step for the Layer 5 device.

Create Certificates

The first thing we need to do on the workstation, after cloning the IoT Edge GitHub repository with

git clone https://github.com/Azure/iotedge.git

,is to generate the root and the intermediate certificates (check folder /tools/CACertificates):

./certGen.sh create_root_and_intermediate

Those certificates will be used to generate the individual devices’ certificates. For now, we will only create a certificate for the Layer 5 device.

./certGen.sh create_edge_device_ca_certificate "l5_certificate"

After those two command, you should have the following in your <WORKDIR>/certs folder:

drwxrwxrwx 1 toddysm toddysm 4096 Apr  8 15:04 .
drwxrwxrwx 1 toddysm toddysm 4096 Apr  8 15:04 ..
-rwxrwxrwx 1 toddysm toddysm 3960 Apr  8 14:54 azure-iot-test-only.intermediate-full-chain.cert.pem
-r-xr-xr-x 1 toddysm toddysm 1976 Apr  8 14:54 azure-iot-test-only.intermediate.cert.pem
-rwxrwxrwx 1 toddysm toddysm 5806 Apr  8 14:54 azure-iot-test-only.intermediate.cert.pfx
-r-xr-xr-x 1 toddysm toddysm 1984 Apr  8 14:54 azure-iot-test-only.root.ca.cert.pem
-rwxrwxrwx 1 toddysm toddysm 5891 Apr  8 15:04 iot-edge-device-ca-l5_certificate-full-chain.cert.pem
-r-xr-xr-x 1 toddysm toddysm 1931 Apr  8 15:04 iot-edge-device-ca-l5_certificate.cert.pem
-rwxrwxrwx 1 toddysm toddysm 7240 Apr  8 15:04 iot-edge-device-ca-l5_certificate.cert.pfx

The content of the <WORKDIR>/private folder should be the following:

drwxrwxrwx 1 toddysm toddysm 4096 Apr  8 15:04 .
drwxrwxrwx 1 toddysm toddysm 4096 Apr  8 15:04 ..
-r-xr-xr-x 1 toddysm toddysm 3326 Apr  8 14:54 azure-iot-test-only.intermediate.key.pem
-r-xr-xr-x 1 toddysm toddysm 3326 Apr  8 14:54 azure-iot-test-only.root.ca.key.pem
-r-xr-xr-x 1 toddysm toddysm 3243 Apr  8 15:04 iot-edge-device-ca-l5_certificate.key.pem

We need to upload the relevant certificates to the Layer 5 device. From the <WORKDIR> folder on the workstation issue the following commands:

scp ./certs/azure-iot-test-only.root.ca.cert.pem pi@pi-l5:.
scp ./certs/iot-edge-device-ca-l5_certificate-full-chain.cert.pem pi@pi-l5:.

The above two commands will upload the public key for the root certificate and the certificate chain to the Layer 5 device. The following command will upload the device’s private key:

scp ./private/iot-edge-device-ca-l5_certificate.key.pem pi@pi-l5:.

Connect to the Layer 5 device with ssh pi@pi-l5 and install the root CA using:

sudo cp ~/azure-iot-test-only.root.ca.cert.pem /usr/local/share/ca-certificates/azure-iot-test-only.root.ca.cert.pem.crt
sudo update-ca-certificates

The response from this command should be:

Updating certificates in /etc/ssl/certs...
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

To verify that the cert is installed, you can use:

ls /etc/ssl/certs/ | grep azure

We should also move the device certificate chain and the private key to the /var/secrets folder:

sudo mkdir /var/secrets
sudo mkdir /var/secrets/aziot
sudo mv iot-edge-device-ca-l5_certificate-full-chain.cert.pem /var/secrets/aziot/
sudo mv iot-edge-device-ca-l5_certificate.key.pem /var/secrets/aziot/

Configuring the IoT Edge Runtime

Installation and configuration of the IoT Edge runtime is described in the official Microsoft documentation (see Install or uninstall Azure IoT Edge for Linux ) but here the values you should set for the Layer 5 device configuration when editing the /etc/aziot/config.toml file:

hostname = "10.16.7.4"

trust_bundle_cert = "file:///etc/ssl/certs/azure-iot-test-only.root.ca.cert.pem.pem"

[provisioning]
source = "manual"
connection_string = "HostName=tsm-ioth-eus-test.azure-devices.net;DeviceId=L5-edge-pi;SharedAccessKey=s0m#VeRYcRYpT1c$tR1nG"

[agent.config]
image = "mcr.microsoft.com/azureiotedge-agent:1.2.0-rc4"
[edge_ca]
cert = "file:///var/secrets/aziot/iot-edge-device-ca-l5_certificate-full-chain.cert.pem"
pk = "file:///var/secrets/aziot/iot-edge-device-ca-l5_certificate.key.pem"

Apply the IoT Edge configuration with sudo iotedge config apply and check it with sudo iotedge check. At this point, the IoT Edge runtime should be running on the Layer 5 device. You can check the status with sudo iotedge system status.

Deploying Modules on the Layer 5 Device

The last thing we need to do is to deploy the required modules that will support the lower-layer device. In addition to the standard IoT Edge modules $edgeAgent and $edgeHub we need also to deploy a registry module.  The registry module is intended to serve the container images for the Layer 4 device. You should also deploy the API proxy module to enable a single endpoint for all services deployed on the IoT Edge device. Here are several Gists that you can use for deploying the modules on the Layer 5 device:

Testing the Registry Module

Before setting up the next layer of the hierarchical IoT edge infrastructure, we need to make sure that Layer 5 registry module is working properly. Without it, you will not be able to set up the next layer. From your laptop, you should be able to pull Docker images from the registry module. Depending on how you have deployed the registry module (using one of the Gists above), the pull commands should look something like this:

docker login -u <registry_username> -p <registry_password> 10.16.8.4:<port_if_any>
docker pull 10.16.8.4:<port_if_any>/azureiotedge-simulated-temperature-sensor:1.0

Now, we have set up the top layer (Layer 5) for our nested IoT Edge infrastructure. In the next post, I will describe the steps to set up the first nested layer of the hierarchical infrastructure – Layer 4.

Image by falco from Pixabay.

To provide support for the hierarchical Azure IoT Edge scenarios we started working on a connected registry implementation that will allow extension of the Azure container registry functionality to on-premises. For those of you who are not familiar with what a hierarchical IoT Edge scenario is, take a look at the Purdue network model used in the ISA 95 and ISA 99 standards – TL;DR: it is the network architecture that allows segregation of OT and IT traffic in manufacturing networks. While the Azure IoT team has provided a sample of the hierarchical IoT Edge environment, I wanted to reproduce it using physical ARM-based devices.

The problem with configuring a hierarchy of IoT Edge devices at home is that home networking devices do not allow advanced network configurations that you would normally expect from an enterprise-grade switch or router. While my Eero is good at its mesh WiFi capabilities, it is a very poor implementation of a switch and doesn’t allow the creation of multiple virtual networks or WiFI SSIDs. The routing capabilities are also quite limited, which to be honest, impact the ability to create secure IoT networks at home. However, I derail… To implement my configuration, I gathered a bunch of Raspberry Pi 4s, Pi Zeros, and an Nvidia Jetson Nano and got to work.

Here is the big picture of what are we implementing:

Hierarchy of IoT Edge devices - Purdue Networks

 

In this post, I will walk over the configuration of the IT Proxy in the IT DMZ layer shown in the picture above. For that, I decided to use a Raspberry Pi 4 device and configure it with the Squid proxy similar to the Azure IoT sample linked above. Before that though, let’s look at the device configuration.

Configuring the Raspberry Pi Network Interfaces

Raspberry Pi 4 has two network interfaces – WiFi one and Ethernet one. The idea here is to configure the WiFi interface to connect to my home network using the home network IP range  192.168.0.0/24 and wire the Ethernet interface to a simple switch and assign it a static IP address from the  10.16.8.0/24 network. Routing between the two interfaces should also be established so traffic from the  10.16.8.0/24 network can flow to the 192.168.0.0/24 network. Pulling some details from the Setting up a Raspberry Pi as a routed wireless access point article on the Raspberry Pi’s official website, I ended up with the following configuration.

Warning: Be careful here and don’t copy the commands directly from the Pi’s article! We are routing in the reverse direction – from the Ethernet interface to the WiFi interface.

  • Install the netfilter-persistent and the iptables-persistent plugin to be able to persist the routing rules between reboots:
    sudo DEBIAN_FRONTEND=noninteractive apt install -y netfilter-persistent iptables-persistent
  • Configure a static IP address for the Ethernet interface. To follow the Azure IoT sample, I choose 10.16.8.4. Edit the DHCP configuration file with sudo vi /etc/dhcpcd.conf and add the following at the end:
    interface eth0
    static ip_address=10.16.8.4/24
  • Enable routing by creating a new file with sudo vi /etc/sysctl.d/routed-proxy.conf and add the following in it:
    # Enable IPv4 routing
    net.ipv4.ip_forward=1
  • Next, create a firewall rule with:
    sudo iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE
  • Last, save the configuration and reboot the device:
    sudo netfilter-persistent save
    sudo systemctl reboot

Checking the configuration with ifconfig after reboot should give you something similar to:

pi@raspberrypi:~ $ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.16.8.4  netmask 255.255.255.0  broadcast 10.16.8.255
        inet6 fe80::6816:2aa8:ab8d:9f93  prefixlen 64  scopeid 0x20<link>
        ether dc:a6:32:77:fb:99  txqueuelen 1000  (Ethernet)
        RX packets 5565  bytes 1359978 (1.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 131  bytes 14399 (14.0 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 22  bytes 1848 (1.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 22  bytes 1848 (1.8 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.0.101  netmask 255.255.0.0  broadcast 192.168.255.255
        inet6 fe80::6788:d8e2:4e8a:558  prefixlen 64  scopeid 0x20<link>
        ether dc:a6:32:77:fb:9a  txqueuelen 1000  (Ethernet)
        RX packets 7619  bytes 1671937 (1.5 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1494  bytes 346279 (338.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

To test the connectivity through the newly configured router, I had to set up my Mac with a wired connection using a USB Ethernet adapter. Here the configuration:

traceroute yields satisfactory results showing that the traffic goes through the Raspberry Pi:

toddysm@MacBook-Pro ~ % traceroute 192.168.0.101
traceroute to 192.168.0.101 (192.168.0.101), 64 hops max, 52 byte packets
 1  192.168.0.101 (192.168.0.101)  9.146 ms  7.669 ms  10.125 ms
toddysm@MacBook-Pro ~ % traceroute 172.217.3.164
traceroute to 172.217.3.164 (172.217.3.164), 64 hops max, 52 byte packets
 1  10.16.8.4 (10.16.8.4)  2.555 ms  0.661 ms  0.377 ms
 2  * *^C
toddysm@MacBook-Pro ~ %

Installing and Configuring Squid Proxy

Installing Squid proxy is as trivial as typing the following command:

sudo apt install squid

However, there are two configuration settings you need to do to make sure that the proxy can be used from machines on the local network. Those settings are available in the Squid configuration file /etc/squid/squid.conf:

  • First, the port configuration for the proxy is set to bind to any IP address but when the proxy starts, it binds to the IPv6 addresses instead of the IPv4 ones. You need to change the following line:
    http_port 3128

    and add the IP address of the Ethernet port like this:

    http_port 10.16.8.4:3128
  • Second, access to the proxy is enabled only from the localhost. Uncomment the following line to enable access from the localnet:
    http_access allow localnet

    Also, make sure that the localnet is defined as:

    acl localnet src 0.0.0.1-0.255.255.255  # RFC 1122 "this" network (LAN)
    acl localnet src 10.0.0.0/8             # RFC 1918 local private network (LAN)
    acl localnet src 100.64.0.0/10          # RFC 6598 shared address space (CGN)
    acl localnet src 169.254.0.0/16         # RFC 3927 link-local (directly plugged) machines
    acl localnet src 172.16.0.0/12          # RFC 1918 local private network (LAN)
    acl localnet src 192.168.0.0/16         # RFC 1918 local private network (LAN)
    acl localnet src fc00::/7               # RFC 4193 local private network range
    acl localnet src fe80::/10              # RFC 4291 link-local (directly plugged) machines

    The latter should already be done later in the file.

To make sure that the proxy is properly configured, I changed the networking configuration in my Firefox browser on my Mac as follows:

I also turned off my Mac’s WiFi to make sure that the traffic goes through the wired interface and uses the proxy. The test was successful and now I have the top layer of the hierarchy of IoT Edge network configured.

In the next post, I will go over configuring the L5 of the Purdue network architecture and installing Azure IoT Edge runtime on it.