Quality RTOS & Embedded Software

LIBRARIES
NOTE: The coreMQTT agent used by the OTA library is functional but not yet part of the coreMQTT library or in the main FreeRTOS.org download because it does not yet meet our code quality criteria and hasn't yet been fully tested. The OTA code is currently available in the Long Term Support release and will be included in the next official FreeRTOS download.
 

Running OTA Integrated with the MQTT Agent
Including Code Signing


Preamble

While this demo uses the AWS IoT OTA update service, FreeRTOS is generic MIT licensed open source software, and can be used with any OTA mechanism that works for you. We do however recommend you digitally sign your firmware whichever OTA method you choose. That way the devices that receive the new executable image can verify it came from an authorized source and has not been been modified. You can use Code Signing for AWS IoT to sign your firmware or you can sign with your own code-signing tools.


Introduction

This example uses the MQTT agent to receive notification of a pending OTA Update Job and to download the new firmware image. Use of the MQTT agent enables the OTA update functionality to run in the background, simultaneously with other tasks that are sharing the same MQTT connection.

The example also demonstrates the use of code signing to verify that the downloaded firmware image has not been tampered with and comes from a trusted sender.

See the comments at the top of each C file in the Source Directory or the Functionality section for additional information.

Instructions

Getting started

Start by connecting one of the simple MQTT agent examples described on the main MQTT agent documentation page to the AWS IoT MQTT broker. That will ensure connectivity to AWS IoT is working correctly before moving on to set up the OTA service.

Once connectivity is working with a non OTA demo the following instructions show you how to:

  • Set up the cloud services to be able to store the firmware image and send it to the device.
  • Configure the device and run the OTA client to receive the update.
  • Prepare and create the OTA Update Job to send the firmware to the device.

Setting up the cloud services

Get started with the AWS console

This demo application uses the AWS console to create an IoT Thing, store a firmware image, and schedule an OTA job. An AWS account is required for you to be authorized to perform these operations. Click here to get started with creating and configuring an account.

Prerequisites for using the OTA Update Manager Service

Create a Thing

An AWS IoT Thing, or simply "Thing", is the cloud representation of a device. By registering your physical device as a Thing, you are able to store security credentials for it, assign permissions to your device (what it can and cannot do or access in the cloud), and select it when sending an OTA update. For this demo, the FreeRTOS Windows Simulator “device” will be represented by an IoT Thing and will be sent an OTA job from the AWS Console.

You will have created a Thing when connecting one of the non OTA demos to AWS IoT, so there are no additional steps here.


Setting up the device (the OTA client)

If, as recommended above, you have already used one of the other MQTT agent examples to connect with AWS IoT then you already have the source files, and have configured the network and AWS IoT connection.

Application Version

For a client to accept an OTA update, the version number of the update it receives must be higher than the version of the firmware that it’s currently running.

The application version of the device software is set in the “ota_over_mqtt_demo.c” source file with the “APP_VERSION_MAJOR”, “APP_VERSION_MINOR”, and the “APP_VERSION_BUILD” macros. The default settings of these macros are 0, 9, and 2 respectively. These do not need to be changed for now, but they will be modified in a future step.

OTA Control Protocol

The OTA Control Protocol setting manages the protocol used for AWS IoT Service control operations such as job notifications, updating the job status, and reporting progress. For control operations, only MQTT is supported at this time.

This setting is controlled by the “configENABLED_CONTROL_PROTOCOL” macro that can be found in the “ota_config.h” header file. The default value for this macro is “OTA_CONTROL_OVER_MQTT” and does not need to be modified for this demo.

OTA Data Protocol

The OTA Data Protocol defines the format used to transfer data over-the-air to and from the device. Currently MQTT and HTTPS are supported.

This option is controlled by setting the “configENABLED_DATA_PROTOCOLS” macro that can be found in the "ota_config.h" header file. The default setting for this macro is "OTA_DATA_OVER_MQTT" and does not need to be changed for this demo.

Code Signing

Code signing is used to confirm the author and guarantee that the code was not tampered with or corrupted since it was signed. As part of the OTA process, the signature of the file is verified after being completely downloaded.

The certificate used to verify the file needs to be set before performing an over-the-air update. To configure the code signing certificate, set the "signingcredentialSIGNING_CERTIFICATE_PEM" variable in the "aws_ota_codesigner_certificate.h" header file.

Enable the OTA Demo

Set the "democonfigENABLE_OTA_UPDATE_DEMO" macro to "1" in the "demo_config.h" file to enable the OTA demo.

Enable Mutual Authentication

Set the "democonfigUSE_TLS" macro to "1" in the "demo_config.h" file to enable mutual authentication.

Verify that the project builds and runs successfully

Before you continue, verify that you are able to build and run the project. You can do this by pressing F5 in the Visual Studio demo project, or by navigating to the “Debug” tab and clicking “Start Debugging”.


Prepare for creating the OTA update job

To send an OTA job, there needs to be an updated firmware image stored in an S3 bucket. The AWS IoT OTA Manager service will read the image out of this bucket and send it to the device. For the device to accept the image the received image must have a higher version number than the image already being executed on the device. The FreeRTOS Windows Simulator is being used for both building and running the demo, which cannot be done at the same time. Due to this, we need the following workflow (documented below the bullet list):

  1. Set the application version number and build the project executable.
  2. Verify that the executable can build and run correctly.
  3. Upload the executable /build/VisualStudio/Demo/RTOSDemo.exe to S3.
  4. Set the application version number to something lower than that used in step 1.
  5. Build the demo with the lowered version number.
  6. Run the updated demo and let it continue to run while waiting for an OTA Job.
Note: An example of a production workflow would be:
  1. Write the MCU firmware, integrating the OTA Client library source code.
  2. Program the device hardware (the MCU) with the initial firmware.
  3. Make changes to and test the firmware locally.
  4. Generate the binary for the new version of the firmware.
  5. Upload the new version to S3 and send it to the MCU with the an OTA Job.
Set the application version

To simulate having a “new” firmware image, increment the version number. For this demo, update the following macro values that can be found in the “ota_over_mqtt_demo.c” source file as shown here:

#define APP_VERSION_MAJOR 0
#define APP_VERSION_MINOR 9
#define APP_VERSION_BUILD 2

Build the “new” firmware image

Generate the firmware image by building the project. To do this, press “Ctrl+Shift+b” or navigate to the build tab and press build within the Visual Studio project. This will generate the following executable:

/build/VisualStudio/Demo/RTOSDemo.exe

Because the demo runs on the FreeRTOS Windows Simulator, the “firmware image” in this case is the Windows executable.

Verify that the project builds and runs successfully

It’s best practice to verify firmware locally before you send it via an OTA update. See the “Verify that the project builds and runs successfully” section.

Upload the firmware image to the S3 bucket

  1. Sign in to the Amazon S3 console at https://console.aws.amazon.com/s3/.
  2. Click on the bucket created in the previous steps.
  3. Click on the “Upload” button that is under the “Overview” tab.
  4. Drag and drop “RTOSDemo.exe” to the bucket.
  5. Click “Upload” to add the executable to the bucket.

Lower the application version

Lower the application version. For this demo, update the following macro values that can be found in the “ota_over_mqtt_demo.c” source file as shown here:

#define APP_VERSION_MAJOR 0
#define APP_VERSION_MINOR 9
#define APP_VERSION_BUILD 1

Build and run the OTA client

Press the “Local Windows Debugger” button to build and run the demo. Allow the client to continue to run as it waits to receive an OTA Job from the AWS IoT OTA Manager service.


Creating the OTA update job using the AWS IoT console

At this point, you should have:

  • Created an AWS IoT Thing with the AWS IoT Service.
  • Setup the S3 bucket and managed permissions for the various services.
  • Uploaded a “newer” firmware image to the S3 bucket.
  • Completed the setup required for code signing.
  • Configured the OTA client running on your device.

Create the OTA update job

With the OTA Client running and the cloud services set up, the next step is to send the device a new firmware image by creating an OTA job. Start by going to the AWS IoT console.

  1. In the navigation pane of the AWS IoT console, choose Manage, and then choose Jobs. Then press “Create a Job”.

    Create a job


  2. Under Create a FreeRTOS OTA update job, choose Create OTA update job.

    Create a job


  3. You can deploy an OTA update to a single device or a group of devices. Under Select devices to update, choose Select. Choose the Things tab to update a single device. Select the check box next to the IoT Thing associated with your device and press Next to continue.

    Create a job


  4. Under “Select the protocol for file transfer" , choose MQTT

    Create a job


  5. Under the “Select and sign your file” section, leave the default option of “Sign a new file for me” selected. Under the “Code signing profile” section, press the “Create button”. 



  6. In the “Profile name” section, enter “winsim_codesigning”. In the “Device hardware platform” section, select “Windows Simulator”. Under the “Code signing certificate” section select import and select the certificate and certificate private keys that you generated earlier. If you were using the suggested command, they will be called “ecdsasigner.crt” and “ecdsasigner.key”. After choosing these, select import. Write the path to the “ecdsasigner.crt” certificate that was just imported into the section called “Pathname of code signing certificate on device”. Then press “Create” to make the code signing profile. After doing these steps, it should look like the following image:



  7. Press select in the “Select your file in S3 or upload it” section and navigate to the executable uploaded during the previous steps. Type “./NewRTOSDemo.exe” in the “Pathname of file on device” section. This path is where the file downloaded during the OTA update will be saved. Select the IAM role created for the OTA process and then press “Next” to continue.




  8. Enter a unique job ID.

    Create a job


  9. Leave the Job Type as the default option (snapshot) and click Create to finish creating the OTA Update Job.



  10. You can monitor that status of the job by pressing the “View Job” pop-up or by navigating to Manage > Jobs in the AWS IoT console. The job will be shown as “IN PROGRESS” until the device has successfully rebooted. 



Receive and activate the update

After the job has been created, the device should begin downloading the update. The progress of the download can be monitored by viewing the Visual Studio console. Below is an example of a progress update:

Current State=[WaitingForFileBlock], Event=[RequestFileBlock], New state=[WaitingForFileBlock]
Received valid file block: Block index=96, Size=2048
Number of blocks remaining: 3

The following message will print to the monitor after receiving the final block of the download:

Received final block of the update.

At this point, the demo will verify the code signing signature of the download. If it's successful, the following message will be printed to the console:

sig-sha256-ecdsa signature verification passed.

The demo will attempt to activate the new image after successfully downloading the update. For this demo, the update needs to be activated by manually running the downloaded executable. This is indicated by the following message:

Failed to activate new image: activateNewImage returned error: Manual reset required: OtaPalStatus_t=OtaPalSuccess

The demo will exit with the following message after successfully validating the update.

OTA agent task stopped. Exiting OTA demo.

If the image was successfully downloaded then it is ready to be activated. Stop the debugging session and close the window. Check the directory that you specified earlier for the new executable that was downloaded over-the-air. If you typed in “./NewRTOSDemo.exe” when you created the job, then the new executable will be located in the same directory as the Visual Studio solution. Run this executable. The device will begin to validate the image after the new executable has started. If this process is successful, then the following message will be printed to the console:

New image validation succeeded in self test mode.

When the job process is complete the state of the job will change from IN PROGRESS to COMPLETED in the IoT console. If the device has successfully passed the self-test process, the status of the job will change to Succeeded.


Functionality

This demo illustrates how to build an application with the OTA library which uses the MQTT agent. Utilizing the MQTT agent enables the OTA agent to share the same MQTT broker connection with other tasks.

Demo setup

This example application includes the resources and interface implementations required by the OTA library. See the OTA library design documentation for more additional information about these interfaces.

OTA PAL implementation

The OTA PAL interface is a set of APIs that are used to store, manage, and authenticate downloads. This demo provides an example implementation for the Windows Simulator platform.

Details of the port can be found in Github.

OTA OS Functional interface implementation

The OTA library calls the OS Functional interface implementations to perform actions that are typically provided by an operating system. This includes managing events, timers, and memory allocation. This demo uses FreeRTOS implementations of these interfaces.

Details of the port can be found in Github.

OTA MQTT interface implementation

The OTA MQTT interface is used by the OTA library to connect to AWS IoT Core and manage notification and request data. The OTA library uses the MQTT protocol to inform AWS IoT Core about job status, receive notifications of pending jobs, and to receive data blocks. The OTA MQTT interface implementation in this example allows multiple tasks to use the same MQTT broker connection.

Details of the port can be found in Github .

OTA application callback implementation

The OTA library provides a way to call a user defined function after receiving a job. This function is called when the OTA agent has completed an update job and when the device reboots after activating the image and is performing a self-test. Typically if the update job was successful, then the device will want to reset to boot into the new image. If the job was rejected, this callback will typically return without doing anything and wait for another job.

Details of the implementation can be found in Github.

Memory allocation setup

The OTA agent requires memory to download, decode, and store the job and file. The OTA agent will use the buffers provided by the user unless they are insufficient. In this case, it will dynamically allocate the memory required with the memory abstraction functions defined by the OTA OS Functional interface implementations.

User configuration

The demo requires some configuration that is specific to the device running it. The details of this can be seen in the earlier demo steps.

Entry point of the demo

The OTA Demo source code can be found on Github . This function is the primary thread for managing the over-the-air download process. It performs the following steps:

Initialize the OTA agent

In order to use the OTA agent, you'll need to provide interface implementations, buffers, and an app callback function to the “OTA_Init” API before starting the event processing task.

You can see an example of this in Github .

Create the OTA agent event processing task

The OTA agent operates by managing a state machine that tracks the current status of the download process. The state machine is influenced by receiving events that are sourced by either internal calls or the main application. The OTA agent requires a loop to be running to receive and process these incoming events before it can start. The demo runs this processing loop in a separate thread that calls the “OTA_EventProcessingTask” API that is defined by the OTA library.

You can see an example of this in Github .

Start the OTA agent

Once the OTA agent has been configured and the event processing task has been started, the OTA agent can begin the download process. This is done by sending an event with the “OtaAgentEventStart” ID to the OTA agent with the “OTA_SignalEvent” API (/Documentation/api-ref/ota-for-aws-iot-embedded-sdk/docs/doxygen/output/html/ota_signalevent_function.html).

You can see an example of this in Github.

Monitor the OTA agent statistics

After the OTA agent has been started, it will continue to operate the download process in the background until it is shutdown. The OTA library provides the “OTA_GetStatistics” API for receiving statistics related to the download progress. These statistics include how many packets have been received, queued, processed, and dropped.

You can see an example of polling and printing these statistics in Github.

Shutting down the OTA agent

The OTA agent will continue to run in the background until it receives an event to shutdown. The OTA library provides an API for sending this event called “OTA_Shutdown”. The demo calls this API after receiving and activating a job to shutdown the OTA Agent.

You can see an example of this in Github.


Troubleshooting

The following sections contain information to help you troubleshoot issues with OTA updates.  

Topics

Copyright (C) Amazon Web Services, Inc. or its affiliates. All rights reserved.