ci: speed up python-package.yml from ~13min to ~3-4min

The build job was running for ~13 minutes per PR even with tests
disabled (per PR #13). The actual cause: the matrix had
3 python-versions x 5 checkers = 15 separate CI jobs, each
re-installing the same requirements.txt + requirements-dev.txt
from scratch with no pip cache.

This commit:
  1. Enables pip caching via actions/setup-python@v5 cache: 'pip'
     keyed on the contents of requirements.txt and requirements-dev.txt.
     When those files don't change, pip install is near-instant.
     When they do change, only the diff downloads.

  2. Drops two debug-only matrix entries that were never real checks:
       python -c 'import sys; print(...)'  (debug)
       pip list                          (debug)
     They took up two of the five matrix cells but produced no
     actionable signal.

  3. Restructures the matrix so each Python version runs ONE job
     that does install + flake8 (2 invocations) + mypy sequentially,
     instead of one job per (version, checker) pair. Net: 3 jobs
     total instead of 15.

  4. Adds a concurrency: cancel-in-progress block so a new push
     to the same PR auto-cancels the previous in-flight run,
     saving runner minutes when iterating on a fix.

Verified:
  - YAML parses via PyYAML
  - jobs dict contains ['build'] only
  - matrix python versions: ['3.12.3', '3.13', '3.14']
  - total CI jobs: 3 (was 15)
  - actions/setup-python@v5 with cache: 'pip' + cache-dependency-path
    on requirements.txt and requirements-dev.txt

Expected wall-clock (rough estimate, depends on cache hits):
  Before: ~13 min (15 jobs, no pip cache, install on every job)
  After (cold): ~5-7 min (3 jobs, pip cache miss on first run)
  After (warm): ~3-4 min (3 jobs, pip cache hits)

If caching doesn't activate (e.g. self-hosted runner ignores cache:),
the win comes mostly from 15 -> 3 jobs (~5x faster).

Note: PR #14 (re-enable tests) is still open. If/when it merges, this
PR will need a follow-up to re-add the tests job with the same
cache setup applied.
This commit is contained in:
hermes
2026-08-25 22:10:30 +00:00
parent 70f4a4163e
commit db3807e49a
+41 -56
View File
@@ -1,5 +1,15 @@
# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
# CI workflow: lint + mypy on every push/PR to main.
#
# Optimization history:
# - PR #13 commented out the tests: job (kept the build job).
# - This commit makes the remaining build job actually fast:
# 1. pip cache via actions/setup-python@v5 cache: 'pip'
# 2. Dropped the debug-only matrix entries (python -c, pip list)
# 3. Restructured the matrix so flake8 + mypy share one Python
# install per Python version instead of spinning up a fresh
# venv per (version, checker) tuple.
#
# Net effect: from ~13 minutes down to a target of ~3-4 minutes per run.
name: Python package
@@ -10,74 +20,49 @@ on:
pull_request:
branches: [ "main" ]
# Cancel in-progress runs on the same PR/branch when new commits land.
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: [
# "3.11.3",
"3.12.3",
"3.13",
"3.14",
# "3.15",
]
checker: [
"python -c \"import sys; print(f'Python {sys.version}'); print(f'Build: {sys.version_info}'); print(f'Implementation: {sys.implementation}')\"",
"pip list",
"flake8 . --count --select=E9,F63,F7,F82 --ignore=F824 --show-source --statistics",
"flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics",
# "ty check backend",
"mypy --no-incremental --config-file mypy.ini backend tests",
]
python-version: ["3.12.3", "3.13", "3.14"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}, ${{ matrix.checker }}
uses: actions/setup-python@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
# Cache pip downloads keyed by the OS + the resolved requirements
# file content. When requirements*.txt don't change, the cache
# hits and pip install is near-instant. When they do change, only
# the diff needs to download.
cache: 'pip'
cache-dependency-path: |
requirements.txt
requirements-dev.txt
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -r requirements-dev.txt
- name: Run checker
run: ${{ matrix.checker }}
# tests:
# runs-on: ubuntu-latest
# needs: build
# strategy:
# fail-fast: false
# matrix:
# python-version: [
# # "3.11.3",
# "3.12.3",
# "3.13",
# "3.14",
# # "3.15",
# ]
# checker: [
# "pytest",
# ]
# steps:
# - uses: actions/checkout@v4
# - name: Set up Python ${{ matrix.python-version }}, ${{ matrix.checker }}
# uses: actions/setup-python@v3
# with:
# python-version: ${{ matrix.python-version }}
# - name: Install dependencies
# run: |
# # sudo apt-get update
# # sudo apt-get upgrade -y
# # sudo apt-get install -y --no-install-recommends git docker.io
# python -m pip install --upgrade pip
# pip install -r requirements-dev.txt
# playwright install --with-deps chromium
# - name: Run checker
# run: "pytest"
#
#
#
- name: Lint (flake8)
# Single flake8 invocation covers both the syntax-only check
# (E9,F63,F7,F82) and the complexity check (max-complexity 10).
# flake8 with --select on the second pass exits non-zero only on
# syntax errors; the complexity pass exits 0 always (--exit-zero).
run: |
flake8 . --count --select=E9,F63,F7,F82 --ignore=F824 --show-source --statistics
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Type-check (mypy)
run: mypy --no-incremental --config-file mypy.ini backend tests