Running pytest as a Github Action

I recently rediscovered my love for Python by adding testing to an existing project. The project needed to be migrated from Python 2 to 3 which made testing a crucial part of the exercise.

I settled on using pytest for implementing the tests.

To be productive, one of the first steps I did was to automate the tests with a Github action so that as soon as I push code to the repository, the tests would run automatically.

Let's first start with the setup of the setup of the project.

There are two very important files which should be checked in into the repository to make this work: requirements.txt and packages.list. The packages.list file contains the list of packages which need to be installed via apt (the action will be based on an Ubuntu Linux container). The contents is something like:

packages.list

1mupdf-tools
2poppler-utils

The second one (requirements.txt) is the list of Python packages we need for the project:

requirements.txt

1pytest
2pytest-cov
3pytest-github-actions-annotate-failures

You might have noticed that there are 2 different pytest packages we install:

• pytest: needed to run pytest
• pytest-cov: adds the option for code coverage to pytest
• pytest-github-actions-annotate-failures: turns pytest failures into action annotations

Speaking of code coverage, you might also want to add a config file for that as well, which should be called .coveragerc. The contents in my case is:

.coveragerc

1[run]
2omit =
3    *test.py

When you want to run the tests manually, you can execute:

1pytest --exitfirst --verbose --failed-first --cov=. --cov-report html

Now, let's combine this into an action. We start with creating a file called .github/workflows/tests.yaml in the root of the repository. This is the file which describes how the action should work. It's contents is as follows:

.github/workflows/tests.yaml

 1name: Tests
 2on: [push]
 3
 4jobs:
 5  build:
 6    name: Run Python Tests
 7    runs-on: ubuntu-latest
 8
 9    steps:
10    
11    - uses: actions/checkout@v2
12
13    - name: Setup timezone
14      uses: zcong1993/setup-timezone@master
15      with:
16        timezone: UTC
17
18    - name: Set up Python 3.8
19      uses: actions/setup-python@v2
20      with:
21        python-version: 3.8
22    
23    - name: Install Python dependencies
24      run: |
25        sudo apt install -y $(grep -o ^[^#][[:alnum:]-]* "packages.list")
26        python3 -m pip install --upgrade pip
27        pip3 install -r requirements.txt        
28
29    - name: Test with pytest
30      run: |
31        pytest --exitfirst --verbose --failed-first \
32        --cov=. --cov-report html        

The action is configured to run each time you push to the repository. The workflow is called Test. You can also see that it runs on the latest version of Ubuntu as specified by runs-on: ubuntu-latest.

The first step is to checkout the source code by using the actions/checkout@v2 action.

The next step is to configure the timezone. This is important if you happen to do time-related stuff and you want to ensure you are in a known timezone. You can use an action zcong1993/setup-timezone@master for doing this.

After that, we install the correct version of Python using the actions/setup-python@v2 action. We are using version 3.8 for this example.

Then, we install all the Linux and Python dependencies by running a couple of terminal commands:

1sudo apt install -y $(grep -o ^[^#][[:alnum:]-]* "packages.list")
2python3 -m pip install --upgrade pip
3pip3 install -r requirements.txt

The last step is to run the tests themselves by running the pytest command.

A sample of the output of the GitHub action can be seen in this example. When you click on a failed run, you'll see the failures in the list of annotations.

A full repository showing this setup can be found here.