Welcome to the Programming Cafe!

Plan for today

Walk in 15 min
Programming Cafe 10 min
GitHub Actions 35 min
Exercises or Work on your own code 50 min
Wrap-up 10 min
Drinks!

Programming Cafe

  • R-Cafe
  • Format
  • Community event
  • Open to UU, HU, UMCU
  • Researchers, students, support staff
  • It is OK to work on your own code

Programming Cafe

  • Ideas for next year
  • Core team members
  • Feedback welcome!

tinyurl.com/uucafe

Basics of GitHub Actions

Research Data Management Support

What is GitHub?


An online repository, based on Git software, for:

  • storing and managing versions of code and documents
  • collaboration
  • publishing
  • project management
  • and more!



GitHub @UtrechtUniversity





https://github.com/UtrechtUniversity

What is GitHub Actions

  • GitHub feature to do magic stuff
  • Free (for public repositories)
  • Perform various tasks on the servers of GitHub
  • Automate things
  • Events/scheduled times

Why using GitHub Actions?

Automation of:

  • Running checks and tests
  • Packaging/publishing
  • Parsing/rendering
    • GitHub Pages
    • Websites
    • Documentation
  • Other stuff

Cool features:

  • Test on several Operating systems (e.g. Windows, macOS, Ubuntu)
  • Simultaneous testing across multiple operating systems and (python) versions
  • Multi-container testing (Docker-compose)
  • Creating and reusing actions from GitHub Marketplace
  • Interact with GitHub API (e.g. comment on pull request, create issues)

Example:



Example

Terminology

Action

An action is a custom application for the GitHub Actions platform that performs a complex but frequently repeated task (e.g. installing Python). Use an action to help reduce the amount of repetitive code that you write in your workflow files.



Event

An event is a specific activity in a repository that triggers GitHub Actions to start. For example, activity can originate from GitHub when someone creates a pull request, opens an issue, or pushes a commit to a repository. You can also trigger GitHub Actions on a schedule.

Terminology


Jobs

A job is a set of steps in a workflow that execute on the same runner (or server). Each step is either a shell script/command that will be executed, or an Action that will be run. Steps are executed in order and are dependent on each other.



Workflow

A workflow is one (or a set of) job(s) that are defined in a script (YAML file) and triggered by events

Terminology


Runner

A runner is a server that runs your workflows when they’re triggered. Each runner can run a single job at a time. GitHub provides Ubuntu Linux, Microsoft Windows, and macOS runners to run your workflows; each workflow run executes in a fresh, newly-provisioned virtual machine.



Further reading

Summary

Use cases for first time using GitHub Actions

When to start with github actions? NOW!

  • Automate checks and tests
  • Automate website building
  • Automate documentation building

Exercise!

Check out https://github.com/UtrechtUniversity/programming-cafe

Part 2: Save the planet, one GitHub Action at a time

Unlimited free resources?

  • Running any code will have an environmental impact
  • Data centers account for 2–3% of global electricity use (IEA, 2022)
  • Microsoft increased its CO2 emissions by nearly 15% from 2020 to 2022.

My typical GitHub Actions Workflow

name: python-package

on: [push, pull_request]

jobs:

  build:
    name: Build for (${{ matrix.python-version }}, ${{ matrix.os }})
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: ['ubuntu-latest', 'macos-latest', 'windows-latest']
        python-version: ['3.6', '3.7', '3.8', '3.9']

    steps:
    - uses: actions/checkout@v2
    - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
    - name: Lint with flake8
      run: |
        pip install flake8
        flake8 mypackage tests
    - name: Test with pytest
      run: |
        pip install pytest
        pytest tests

Reduce! Reuse! Recycle!



It will save you time and the planet!

Reduce

Restrict branch and event types

  • Work with draft pull requests!
on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
    types:
      - ready_for_review

Filter by changed files

project-to-save-the-planet/
├── docs/
│   └── the_plan.md
│   └── ...
├── src/
│   ├── save_the_planet.py
│   ├── supporting_documentation.md
│   └── ...
├── tests/
│   └── ...
├── ...
└── README.md

Filter by changed files

# docs.yml
on:
  push:
    ...
    paths:
      - 'docs/**'

jobs:
  ...

Filter by changed files

# tests.yml
on:
  push:
    ...
    paths:
      - 'src/**'
      - '!src/supporting_documentation.md'
      - 'tests/**'

jobs:
  ...

Reduce

Use CPU versions of tensorflow and pytorch

Reuse

  • Caching and reusing stuff

When?

  • When there are things that do not change between runs most of the times
    • Setting up environment
    • pre compute something
    • compiling

See: Cache action

Reuse

# tests.yaml
jobs:
  test:
    steps:
      ...
      
      - uses: actions/cache@v3
        id: cache-python-env
        with:
          path: ${{ env.pythonLocation }}
          key: ${{ env.pythonLocation }}-${{ hashFiles('pyproject.toml') }}

      - name: Install dependencies
        if: steps.cache-python-env.outputs.cache-hit != 'true'
        run: |
          python -m pip install -e .

Recycle

Restart tests from last failed test with the Pytest --last-failed option

  • Cache test results

Exising GitHub action:

tests.yaml
jobs:
  test:
    steps:
      # create python environment and install pytest    
      ...  

    - name: Run pytest
      uses: sjvrijn/pytest-last-failed@v2
      with:
        pytest-args: '--my --pytest --args'

Summary

Reduce:

  • Work with draft pull requests
  • Use conditional workflows

Reuse:

  • Cache dependencies

Recycle:

  • Recycle test results

Questions?

Check out https://github.com/UtrechtUniversity/programming-cafe if you want to try exercises.

Check out https://blog.esciencecenter.nl/ for the original blog post

Jelle Treep