Why OTA is Hard on Embedded Linux
Unlike mobile apps, embedded Linux OTA failures can brick field devices. A robust pipeline must handle interrupted downloads, verification failures, partition corruption, and failed boot sequences — all without a technician on-site.
Architecture Overview
Our production stack uses AWS Greengrass v2 for orchestration, S3 for artifact storage, SWUpdate for on-device updates with A/B partition support, and U-Boot with rollback trigger on failed boot counter.
A/B Partition Layout
/dev/mmcblk0p1 — bootloader (U-Boot, read-only)
/dev/mmcblk0p2 — rootfs A (active)
/dev/mmcblk0p3 — rootfs B (update target)
/dev/mmcblk0p4 — data partition (persistent)Signature Verification
Never deploy unsigned firmware. SWUpdate supports RSA and CMS signing. Generate your signing key pair and embed the public key in your Yocto image. Use openssl cms to sign the .swu artifact.
Rollback Mechanism
U-Boot boot counter is the safety net. On each boot attempt, U-Boot increments a counter. If the new rootfs boots successfully, the application resets the counter. If it hits 3, U-Boot falls back to the known-good partition.
Production Lessons
- Delta updates — Use bsdiff or casync to generate deltas. We reduced OTA payload from 320MB to 18MB.
- Bandwidth throttling — Use Greengrass deployment timeouts on cellular-connected devices.
- Canary deployments — Deploy to 5% of fleet first via Greengrass thing groups.