Build PyPI package

.github/workflows/deploy_pypi.yaml

@@ -0,0 +1,39 @@
+name: "Deploy PyPI workflow"
+
+on:
+  push:
+    branches:
+      - main
+
+jobs:
+  # Build and lint release distribution
+  release-build:
+    name: Build and verify release
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v5
+        with:
+          fetch-depth: 0
+          python-version: ["3.8", "3.9", "3.10"]
+
+      - uses: hynek/build-and-inspect-python-package@v2
+
+  # Publish to PyPI
+  pypi-publish:
+    name: Upload release to PyPI
+    runs-on: ubuntu-latest
+    needs: release-build
+    permissions:
+      # IMPORTANT: this permission is mandatory for trusted publishing
+      id-token: write
+    environment:
+      name: pypi
+    steps:
+      - name: Retrieve release distributions
+        uses: actions/download-artifact@v5
+        with:
+          name: release-dists
+          path: dist/
+
+      - name: Publish release distributions to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

.github/workflows/increment_version_workflow.yaml → .github/workflows/increment_version.yaml

@@ -6,19 +6,20 @@ on:
       - main
 
 jobs:
-  deploy:
+  # Increments repo version and pushes tag
+  increment-version:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v5
+      - uses: actions/checkout@v4
 
-      - name: Extract package version
-        id: package_version
+      - name: Extract repo version
+        id: repo_version
         run: |
-          echo "PACKAGE_VERSION=$(python setup.py --version)" >> $GITHUB_ENV
+          echo "PACKAGE_VERSION=$python setup.py --version)" >> $GITHUB_ENV
 
       - name: Bump version and push tag
         id: tag_version
-        uses: mathieudutour/github-tag-action@v6.1
+        uses: mathieudutour/github-tag-action@v6.2
         with:
           github_token: ${{ secrets.GITHUB_TOKEN }}
           custom_tag: ${{ env.PACKAGE_VERSION }}

CHANGELOG.md

@@ -18,6 +18,7 @@ and this project adheres to [semantic versioning](https://semver.org/spec/v2.0.0
   - `create_spark_session` in `tests/utils/test_utils_create_spark_session.py`.
   - `get_input_variables` in `tests/utils/utils.py`.
   - `get_deltas` in `tests/indicator_matrix/test_indicator_matrix.py`.
+- GitHub Action workflow to deploy repo as package in PyPI.
 
 ### Changed
 

docs/branch_and_deploy_guide.md

@@ -24,11 +24,12 @@ Our repository has two permanent branches:
 1. **Feature branches:**
   - All new features and bugfixes are developed in separate branches created from the `develop` branch.
   - Any hotfixes are developed in separate branches created from the `main` branch.
-  - Branch naming conventions:
-    - `docs/<documentation-description>` - for updates to documentation only.
+  - [Conventional branch][branches] naming conventions:
     - `feat/<feature-description>` - feature branches, for introducing new features.
     - `fix/<bug-description>` - bugfixes, for resolving bugs.
     - `hotfix/<issue-description>` - hotfixes, for urgent fixes that go straight to production.
+    - `release/<release-number>` - for preparing a release.
+    - `chore/<chore-description>` - for non-code tasks, e.g. dependency or documentation updates.
   - [Conventional commit][commits] messages, including the following types:
     - `build` - for changes that affect the build system or external dependencies.
     - `ci` - for changes to CI configuration files and scripts, e.g. GitHub Actions, Dependabot.
@@ -65,44 +66,65 @@ Our repository has two permanent branches:
 
 ### Overview
 
-[GitHub Actions][github-actions] are triggered on pull request from any branch, including feature branches. This CI/CD pipeline
-ensures code does not enter the `develop` or `main` branches unless it has had certain checks. The repository is set up so
-that pull requests cannot be merged if these GitHub Actions fail.
+Certain [GitHub Actions][github-actions] are triggered on merging to any branch. This CI/CD pipeline ensures code does not enter
+any parent branches unless it has had certain checks.
 
-### Steps in the pull request pipeline
+### Pull request workflow steps
 
 1. **Trigger:**
-   - The pipeline is triggered when a `merge` is detected.
+   - When a `merge` is detected.
 
-2. **Branch check:**
-   - Checks that the pull request is only into `main` if coming from `develop` or a branch whose name starts with `hotfix`.
+2. **Check branch:**
+   - Check the base branch for the pull request.
+   - If the base branch is `main`, check if the branch is `develop` or has a name starting with `hotfix`.
 
-3. **Changelog check:**
-   - The changelog is checked for updates.
+3. **Check changelog:**
+   - Check that `CHANGELOG.md` has been updated.
 
-4. **Pre-commit hooks:**
-   - All pre-commit hooks are run.
+4. **Pre-commit:**
+   - Run all pre-commit hooks.
 
-5. **Unit tests:**
-   - All unit tests are run.
+5. **Test:**
+   - Run all unit tests on all versions of Python supported by the repo.
 
 ## Deployment process using GitHub Actions
 
 ### Overview
 
-The deployment process is automated using [GitHub Actions][github-actions]. This CI/CD pipeline is triggered upon merging
-changes into the `main` branch.
+The deployment process is automated using [GitHub Actions][github-actions]. This CI/CD pipeline is triggered upon pushing
+changes into the `main` branch. It is separated into two workflows: one for incrementing the version tag in GitHub and the other
+for deploying to PyPI.
 
-### Steps in the deployment pipeline
+### Increment version workflow steps
 
 1. **Trigger:**
-   - The pipeline is triggered when a `merge` into `main` is detected.
+   - When a `push` to `main` is detected.
 
-2. **Increment version:**
-   - The version of the package is incremented.
-   - The new version tag is pushed.
+2. **Extract repo version:**
+   - Extract the version of the repo from the location specified in `setup.py`.
+
+3. **Bump version and push tag:**
+   - Bump the repository version by pushing a new tag.
 
-Further deployment steps will be added in due course.
+### Deploy PyPI workflow steps
+
+1. **Trigger:**
+   - When a `push` to `main` is detected.
+
+2. **Build and verify package:**
+   - Use `uv` via `hynek/build-and-inspect-python-package` to:
+      - Build the package.
+      - Upload the built wheel and the source distribution as GitHub Actions artifacts.
+      - Lint the wheel contents using `check-wheel-contents`.
+      - Lint the PyPI README using `Twine` and upload it as a GitHub Actions artifact.
+      - Print the tree of both SDist and `wheel`, allowing manual checking of the content list.
+      - Print and upload the packaging metadata as a GitHub Actions artifact.
+
+3. **Download built package:**
+   - Download the built package from GitHub Actions artifacts to `dist`.
+
+4. **Upload package to PyPI:**
+   - Upload the package from `dist` to PyPI.
 
 ## Merging develop to main: A guide for maintainers
 
@@ -140,15 +162,15 @@ As `scalelink` maintainers, ensuring a seamless transition from `develop` to `ma
     This step ensures users and developers can easily find and access the specific versions of `scalelink` through their respective release pages and download links, maintaining comprehensive and navigable documentation.
 
 - **Final review and push:**
- - Review the changes one more time, ensuring that the version bump and `CHANGELOG.md` updates are correctly applied.
- - Push the commit(s) to the `develop` branch. This action updates the branch with the version change and changelog updates.
+  - Review the changes one more time, ensuring that the version bump and `CHANGELOG.md` updates are correctly applied.
+  - Push the commit(s) to the `develop` branch. This action updates the branch with the version change and changelog updates.
 
 ### Merging and deployment
 
 - **Merge to main:**
   - With all preparations complete and changes reviewed, proceed to merge the `develop` branch into the `main` branch.
   - This action can be done through the GitHub UI by completing the pull request initiated in the Preparation section of this guide.
-  - Merging to `main` automatically triggers the GitHub Actions workflow for deployment. **Note: this currently only includes creating a GitHub Release with the new version tag.**
+  - Merging to `main` automatically triggers the GitHub Actions workflow for deployment.
 
 ### Synchronising develop branch post-merge
 
@@ -175,39 +197,86 @@ Below is a visual representation of our Git workflow, illustrating the process f
 
 ```mermaid
 graph TD
-    A([Start feature development])
-    B[Create feature branch from develop branch]
-    C{Feature complete and tested?}
-    D[Raise pull request to merge feature branch into develop branch]
-    E[Trigger automated checks via GitHub Actions]
-    F[Review and approve pull request]
-    G{Develop branch: Ready for release?}
-    H[Update package version -- semver]
-    I[Raise pull request to merge develop branch into main branch]
-    J[Trigger automated checks via GitHub Actions]
-    K[Review and approve pull request]
-    L[Trigger automated deployment via GitHub Actions]
-    M[Create GitHub Release with version tag]
-    N[Update develop branch with main]
-
-    A --> B
-    B --> C
-    C -- No --> B
-    C -- Yes --> D
-    D --> E
-    E --> F
-    F --> G
-    G -- No --> A
-    G -- Yes --> H
-    H --> I
-    I --> J
-    J --> K
-    K --> L
-    L --> M
-    M --> N
-    N --> A
+    Start1([Start or continue feature development or bugfix])
+    Start2([Start or continue hotfix])
+
+    Feat1[Create feature branch from develop branch]
+    Feat2[Develop feature or bugfix in feature branch]
+    Feat3{Feature branch: complete and tested?}
+    Feat4[Raise pull request to merge feature branch into develop branch]
+    Feat5[Trigger automated checks via GitHub Actions]
+    Feat6[Review pull request]
+    Feat7{Feature branch: approve pull request?}
+    Feat8[Merge pull request]
+
+    Dev1{Develop branch: Ready for release?}
+    Dev2[Update package version - semver major or minor update]
+    Dev3[Raise pull request to merge develop branch into main branch]
+    Dev4[Trigger automated checks via GitHub Actions]
+    Dev5[Review pull request]
+    Dev6{Develop branch: approve pull request?}
+    Dev7[Merge pull request]
+
+    Deploy1[Trigger automated deployment via GitHub Actions]
+    Deploy2[Create GitHub Release with version tag]
+    Deploy3[Update develop branch with main]
+    Deploy4[Build and test scalelink package]
+    Deploy5[Publish to PyPI]
+
+    Hotfix1[Create hotfix branch from main branch]
+    Hotfix2[Develop hotfix in hotfix branch]
+    Hotfix3{Hotfix branch: complete and tested?}
+    Hotfix4[Raise pull request to merge hotfix branch into main branch]
+    Hotfix5[Trigger automated checks via GitHub Actions]
+    Hotfix6[Review pull request]
+    Hotfix7{Hotfix branch: approve pull request?}
+    Hotfix8[Update package version - semver patch update]
+    Hotfix9[Merge pull request]
+
+    Start1 --> Feat1
+    Start2 --> Hotfix1
+
+    Feat1 --> Feat2
+    Feat2 --> Feat3
+    Feat3 -- No --> Feat2
+    Feat3 -- Yes --> Feat4
+    Feat4 --> Feat5
+    Feat5 --> Feat6
+    Feat6 --> Feat7
+    Feat7 -- No --> Feat2
+    Feat7 -- Yes ---> Feat8
+    Feat8 --> Dev1
+
+    Dev1 -- No --> Start1
+    Dev1 -- Yes --> Dev2
+    Dev2 --> Dev3
+    Dev3 --> Dev4
+    Dev4 --> Dev5
+    Dev5 --> Dev6
+    Dev6 -- No --> Start1
+    Dev6 -- Yes ---> Dev7
+    Dev7 --> Deploy1
+
+    Deploy1 --> Deploy2
+    Deploy2 --> Deploy3
+    Deploy3 --> Deploy4
+    Deploy4 --> Deploy5
+    Deploy5 ------------> Start1
+
+    Hotfix1 --> Hotfix2
+    Hotfix2 --> Hotfix3
+    Hotfix3 -- No --> Hotfix2
+    Hotfix3 -- Yes --> Hotfix4
+    Hotfix4 --> Hotfix5
+    Hotfix5 --> Hotfix6
+    Hotfix6 --> Hotfix7
+    Hotfix7 -- No --> Hotfix2
+    Hotfix7 -- Yes --> Hotfix8
+    Hotfix8 --> Hotfix9
+    Hotfix9 ----------> Deploy1
 ```
 
+[branches]: https://conventional-branch.github.io/
 [commits]: https://www.markdownguide.org/basic-syntax/#links
 [sem-ver]: https://semver.org/
 [github-actions]: https://github.com/features/actions