Github actions use cached images

[bn222]

What happened?

While working on #414, we have noticed that a build error was hidden (locally and) in Github actions. We should ensure that Github actions don't rely on cached versions.

What did you expect to happen?

Github actions should fail if the build fails on a clean system.

What are the minimal steps needed to reproduce the bug?

n/a

Anything else we need to know?