-
Notifications
You must be signed in to change notification settings - Fork 143
Setup a ci pipeline for firmware by leoribg #285
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
leoribg
wants to merge
7
commits into
memfault:master
Choose a base branch
from
leoribg:setup-ci-pipeline-git-actions-firmware
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
7c8ab47
Setup a ci pipeline for firmware by leoribg
leoribg eecf4b8
Apply suggestions from code review
tyhoff a96eca1
Resolve suggestions from code review #1
leoribg 9095f51
Remove unnecessary image
leoribg 5fc25be
Specify Linux distribution and link ssv5 and SDKs
leoribg 6898734
Replace PowerShell to bash code fence syntax
leoribg 30b713f
Accept spelling correction suggestion
leoribg File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
237 changes: 237 additions & 0 deletions
237
_posts/2022-11-01-setup-a-ci-pipeline-using-github-actions-and-docker.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,237 @@ | ||
--- | ||
title: Setup a CI pipeline using GitHub Actions and Docker | ||
description: A step by step guide on how to setup continuous integration for firmware projects with GitHub Actions. | ||
author: leoribg | ||
tags: [firmware-ci] | ||
--- | ||
|
||
# Setup a CI pipeline using GitHub Actions and Docker | ||
|
||
# Introduction | ||
|
||
Continuous Integration (CI) practices are essential to bringing security, quality, and automation to your firmware development team. | ||
|
||
In this post, I will show how to set up a Continuous Integration pipeline using modern tools to help your team integrate the features into a mainline firmware project step by step. | ||
|
||
# Continuous Integration Tools | ||
|
||
Continuous Integration is a practice of validating the features integrated into the main project. | ||
|
||
For that, you will need to perform at least these steps: | ||
|
||
- **Git flow** - Rules to handle branches in a git repository. | ||
- **Automatic compliance** - Check if the project is still compiling before merging a feature. | ||
- **Code quality** - Check if there are vulnerabilities in the code. | ||
|
||
So, to complete these steps, we will use the following tools: | ||
|
||
- **GitHub Actions** - It kicks off a compilation and static code analysis on the cloud. It allows us to run build scripts on every commit or pull request to make sure everything still works with the new changes inserted. ([https://docs.github.com/en/actions](https://docs.github.com/en/actions)) | ||
- **Docker** - The Linux container will compile the project and run static analysis in the code. ([https://www.docker.com/](https://www.docker.com/)) | ||
- **Cpp Check** - Perform the static analysis of the code and generate a log from it. ([https://cppcheck.sourceforge.io/](https://cppcheck.sourceforge.io/)) | ||
|
||
# Project creation | ||
|
||
## Board and IDE | ||
|
||
This tutorial is specific to the Simplicity Studio build system, but the general approach should apply to other build systems. I will use the [Silicon Labs Thunderboard Sense Board](https://www.silabs.com/development-tools/thunderboard/thunderboard-sense-two-kit?tab=overview) to run a simple project on [Simplicity Studio IDE](https://www.silabs.com/developers/simplicity-studio) that we will use to build and perform static analysis. | ||
|
||
 | ||
|
||
## Create the project and compile it on the Local Machine | ||
|
||
If you have the Thunderboard Sense board, you can connect it to the PC and check if the Simplicity Studio detects it. | ||
|
||
 | ||
|
||
If you do not have the board, you can add it to my products tab and create the project anyway. It is not impeditive to continue. You can follow the steps in the next images. | ||
|
||
 | ||
|
||
 | ||
|
||
After you connect or add the board, go to the Launcher perspective on Simplicity Studio, click on the board, and next click on Create New Project button. | ||
|
||
 | ||
|
||
Select the Empty C Project and Finish! | ||
|
||
 | ||
|
||
To generate the project Makefile, open the empty.slcp file and do the following steps. | ||
|
||
 | ||
|
||
Select the GCC Makefile option on Change Project Generators, save, and Force Generation. | ||
|
||
 | ||
|
||
After that, the empty.Makefile and empty.project.mak files will be created in the project root directory. | ||
|
||
 | ||
|
||
Now you can build the project in the terminal since Simplicity Studio installed the toolchain. | ||
|
||
If you are running it on Linux, you can go directly to the terminal, but if you are using Windows, you will need to install [Cygwin](https://www.cygwin.com/) and execute the command. | ||
|
||
```bash | ||
make -f empty.Makefile | ||
``` | ||
|
||
Now the project can be built from the terminal, which is necessary for building in CI. | ||
|
||
# Test the compilation on a Docker Container | ||
|
||
We will use a docker image based on Ubuntu 18.04 with [Simplicity Studio v5](https://www.silabs.com/documents/login/software/SimplicityStudio-5.tgz) and the necessary [SDKs](https://github.com/SiliconLabs/gecko_sdk) installed to build our project on a container. It will allow us to configure GitHub Actions to use the docker image to do the same. | ||
|
||
So let's pull the image to our computer: | ||
|
||
```bash | ||
docker pull leoribg/simplicity-studio-5:latest | ||
``` | ||
|
||
Start the container from the image: | ||
|
||
```bash | ||
docker run -t -d leoribg/simplicity-studio-5 | ||
``` | ||
|
||
Copy the project files to the container: | ||
|
||
```bash | ||
docker cp C:\Users\leonardo\SimplicityStudio\v5_workspace\empty 1027ffa9c954:/project | ||
``` | ||
|
||
Compile the project and run the `cppcheck` static code analysis tool: | ||
|
||
```bash | ||
cd /project/ | ||
make -f empty.Makefile -j8 | ||
cppcheck . --output-file=cpplog.txt | ||
``` | ||
|
||
## Create the Git Repository | ||
|
||
Since the project was created, let's create a git repository. | ||
|
||
For this tutorial, this is the repository I've used for my project: | ||
[https://github.com/leoribg/memfault-empty-example](https://github.com/leoribg/memfault-empty-example) | ||
|
||
[To start and add your project to a git repository](https://docs.github.com/en/get-started/quickstart/create-a-repo), run the following commands on your project directory: | ||
|
||
```bash | ||
git init | ||
git add . | ||
git commit -m "first commit" | ||
git remote add origin [email protected]:leoribg/memfault-empty-example.git | ||
git push -u origin master | ||
``` | ||
|
||
# Configure the GitHub Actions | ||
|
||
You can start to configure the GitHub Actions for your project by creating the .yml file with your pipeline inside the “.github/workflows” folder. Or, you can create at your repository page by creating the file following a template provided by GitHub. Let's do the second option, that's easier for those who are starting. | ||
|
||
So, go to your repository page and click on the “Actions” tab: | ||
|
||
 | ||
|
||
Choose the right template: | ||
|
||
 | ||
|
||
This tutorial will use a single workflow to execute our jobs. So, it will only have a single .yml file. | ||
|
||
The c-cpp.yml file starts like this, we will modify it. | ||
|
||
```yaml | ||
name: C/C++ CI | ||
|
||
on: #These are the triggers that will run our jobs | ||
push: | ||
branches: [ "master" ] | ||
pull_request: | ||
branches: [ "master" ] | ||
|
||
jobs: #Jobs are set of steps that will execute our tasks | ||
build: | ||
|
||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- uses: actions/checkout@v3 | ||
- name: configure | ||
run: ./configure | ||
- name: make | ||
run: make | ||
- name: make check | ||
run: make check | ||
- name: make distcheck | ||
run: make distcheck | ||
``` | ||
|
||
Let's modify the content of our .yml file to execute the build and the static analysis of the project. | ||
|
||
We will indicate the Docker Image we use to execute the workflow's steps. After we will build the project, upload the image as an artifact. Finally, we will make a static analysis in the code and upload the artifact with the results. | ||
|
||
```yaml | ||
name: C/C++ CI | ||
|
||
on: | ||
push: | ||
branches: [ "master" ] | ||
pull_request: | ||
branches: [ "master" ] | ||
|
||
jobs: | ||
build: | ||
|
||
runs-on: ubuntu-latest | ||
container: | ||
image: leoribg/simplicity-studio-5:latest #Docker Image | ||
|
||
steps: | ||
- uses: actions/checkout@v3 | ||
- name: make #Build | ||
run: make -f empty.Makefile | ||
- name: binary artifact #Upload the image artifact | ||
uses: actions/upload-artifact@v3 | ||
with: | ||
name: empty.s37 | ||
path: ./build/debug/empty.s37 | ||
- name: static analisys #Lint | ||
run: cppcheck . --output-file=cpplog.txt | ||
- name: lint artifact #Upload the lint artifact | ||
uses: actions/upload-artifact@v3 | ||
with: | ||
name: lint_output.txt | ||
path: ./cpplog.txt | ||
``` | ||
|
||
# Trigger our GitHub Action | ||
|
||
As we have configured the workflow to run every commit or pull-requests on the master branch, when you commit the changes, it will trigger the GitHub Actions on the project. | ||
|
||
 | ||
|
||
After all the tests have been completed and passed, the workflow run is shown like this: | ||
|
||
 | ||
|
||
Also, the artifacts were being uploaded and are available for download. | ||
|
||
 | ||
|
||
# All done! | ||
|
||
You have configured your first CI workflow using GitHub Actions. | ||
|
||
I hope this can help you to start using GitHub Actions on your future projects! | ||
|
||
## Reference & Links | ||
|
||
[^1]: [Git Flow](https://www.atlassian.com/br/git/tutorials/comparing-workflows/gitflow-workflow). | ||
[^2]: [GitHub Actions](https://docs.github.com/en/actions) | ||
[^3]: [Docker](https://www.docker.com/) | ||
[^4]: [CppCheck](https://cppcheck.sourceforge.io/) | ||
[^5]: [Silicon Labs Thunderboard Sense Board](https://www.silabs.com/development-tools/thunderboard/thunderboard-sense-two-kit?tab=overview) | ||
[^6]: [Simplicity Studio IDE](https://www.silabs.com/developers/simplicity-studio) | ||
[^7]: [Cygwin](https://www.cygwin.com/) |
Binary file added
BIN
+137 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/0-thunderboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+18.4 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/1-board-selection.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+132 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/10-actions.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+128 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/11-actions-template.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+204 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/12-cpp-workflow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+191 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/13-cpp-editing.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+157 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/14-commit-workflow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+79.3 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/15-workflow-run.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+96.1 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/16-workflow-artifacts.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+156 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/2-add-new-board.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+80.7 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/3-add-board.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+178 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/4-new-project.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+232 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/6-generators.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+179 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/7-gcc-makefile-gen.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+29.6 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/8-makefile.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+51.5 KB
img/setup-a-ci-pipeline-using-github-actions-and-docker/9-compile.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's include the Dockerfile source with an explanation of what goes into the image- that's a crucial part of this setup and deserves a detailed explanation!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree. The Docker image is also 6GB. Would love to understand more about what's in there and what we're tell ing people they need to put in their GitHub Action CI image.
Would also be a good chance to update the Docker file to adhere to best practices
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, but I was trying to provide the readers a "ready to build" docker image. So they can run and build the example in minutes. The image consists of a ubuntu image with Simplicity Studio and the SDKs installed. I had to install Simplicity Studio on an Ubuntu Machine and copy the installation contents to the container to generate the image. So this is not the focus of the article. But I can try to create a docker file for it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, Simplicity Studio requires login 😕 always a bit tedious when vendors do that for free toolchains.
I'd still like to have the Dockerfile detailed here; providing a pre-built one is great if someone wants to try it, but the heart of the article is about setting up such a system, rather than the actual result (eg, if someone's not using Simplicity Studio, having the Dockerfile reference is much more valuable than the pre-built image, IMO).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, understood. I will do it.