GitHub actions code coverage — Without third parties
As it turns out. The way to integrate code coverage into your build pipeline with GitHub actions is to use a third-party solution, like codcov.io and others. Those solutions are fantastic but can cost up to 20$ / month per user. And they come with many advanced features that not everybody needs. For a basic code coverage check on pull requests and a code coverage badge in the README.md I don’t want to pay a monthly subscription.
After digging the Internet for a free solution I found nothing useful. And decided to hack my way around.
In this post, ill show how to use GitHub actions and some cloud storage ( like s3 ) to create a code coverage badge for your repository, and a GitHub status which can be used to protect the master branch, so if a pull request dropped the code coverage it will be blocked for merging.
A GitHub badge is simply a small SVG embedded in the repo README.md. The simplest way to create one is to use shields.io API.
curl https://img.shields.io/badge/coavrege-$total%-$COLOR > badge.svg
Where $COLOR is a bash variable containing a CSS color like red, green, orange. And $total is another bash variable with the percentile between 0 and 100. shields.io are awesome for providing this free utility.
This simple one-liner can create a badge, what’s left is to upload it to a public storage with cache disabled and embed it in a README.md
BADGE GENERATION
A GitHub actions workflow can run the above on a push/merge to master branch and upload the badge, notice the “Cache-Control” header, this is important because at least in GCP but probably also on AWS if you upload an object to a public storage its cached by default and then the README.md of your project will always contain an outdated version.
README.md
GitHub actions workflow file
A GitHub Status
This is the hacky part of this post. It is common to fail the CI if code coverage was dropped, the way to achieve this with GitHub actions is using GitHub Statuses
A GitHub status is an entity connected to a commit, any commit can have many statuses associated with it. A status can have a state (error, failure, pending, or success). The most common example of a status is your project test suite, while tests are running the status is pending, if they fail it becomes failure, and if they pass then success. This comes handy for pull requests because it is possible to “protect a branch”, allow merging pull requests only if all statuses pass.
When you run a GitHub workflow any job in that workflow will create a status (and also a badge ), but the name of the status is static ( the name of your job, like CodeCov from the example above — line 9 ). If you want your status to contain dynamic information, like the percentile of code coverage you will have to work a bit harder, so let's just dig into the code and explain it later.
Code Coverage Status GitHub workflow
This piece is a bit longer, it creates a pending status for the current commit, runs the tests, downloads the code coverage of the master branch ( which we uploaded in the previous workflow when publishing the badge), and compares to the code coverage of the current run. If the code coverage drops it fails the status else it marks it as successful.
Notice the two curl commands that call api.github.com. They create and then update the code coverage status.
And this is how it looks
That’s it :)
care for your code. Test it and don’t ignore code coverage. Its not the best quality metric ( 90% coverage says nothing about the quality of your tests ) but its better then nothing . Cheers.
