Skip to content

Simplify base_deps cache management #862

@kelson42

Description

@kelson42

It is quite complicated to know what exactly happens around base_deps cache files. Most of our modifications of kiwix-build code base impact them, but they are never newly compiled as long as this does not have been requested explicitly. The way they are stored (only one place at the root of http://tmp.kiwix.og/ci/ is also not optimal as it is quite error and side-effect prone.

This issue is about making the base_deps caching optional (so an optim) instead of having it per default.

Here is a proposal to improve the solution:

  • We remove the versioning of the base_deps
  • Each time we push/merge on main branch the base deps are newly compiled and uploaded (erasing older version)
  • In a PR
    • base deps cache files are located in a dedicated sub-directory (like we do for deps cache files)
    • Per default, there is no cache, everything is newly compiled (in any case the first CI run in the PR)
    • We will have a text file describing the list of base_deps files (around 40 for the moment) which are allowed to be used as cache (so no new recompilation). This can be used by the dev to speed-up the kiwix-build run.
    • The kiwix-build PR template will propose the list of these base deps files with for each a check-box. Per default nothing is checked, but if the dev wants, he can check here the things he does not want to recompile each time.
    • To validate a PR all the checkboxes should unchecked (before last CI run)
    • A "contrib" script will be created to create automatically the code of the PR template based on the base deps files uploaded to http://tmp.kiwix.org/ci (this script should be run time to time to verify we have not forgotten to update the PR template)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions