Ask HN: Tooling to skip/speed up CI steps?

2 points by qznc 6y ago ↗ HN

Our repository is so large/active that our CI is painful. You could also say our build/test process is so slow that our CI is painful. I’m looking for tooling to reduce the load on our CI.

For example, if only documentation has been changed, there is no need to run the tests. If no documentation is changed there is no need to rebuild the Doxygen website. If only a one component is changed, the unit tests for the others need to be run. If only tests have been changed there is no need to deploy the software. If only build config is changed, there is no need to run the unit tests.

I’m not sure what to google for. Is there a name for such mechanisms? Buildpipe seems roughly the right idea but is not really applicable without Buildkite.

The configuration for such a tool is so complex that a Python library might make more sense than an executable? We are certainly not the first ones. Does everybody write this from scratch? It makes sense for monorepos but the awesome list does not contain anything.

https://github.com/korfuri/awesome-monorepo

1 comment

[ 3.7 ms ] story [ 13.9 ms ] thread

eberkund 6y ago ↗

Depends the CI software obviously but for Azure DevOps/VSTS, they have just what you describe and seem to call it "triggers": https://docs.microsoft.com/en-us/azure/devops/pipelines/buil...

I have seen similar things in other CI providers but basically you have two options:

1) Use a monorepo and filter out which steps or when to run the CI based on the conditions you describe using the features of the CI software. 2) Split up the repo and give each repo its own CI which runs when files in the subrepo change. For example the frontend CI would run the frontend tests and compile a static file which could just simply be downloaded by the main build.

Both will speed up your CI process, which one is best depends a lot on your application architecture and build and deployment process.