Reproducible C++ builds by logging Git hashes (jgarby.uk)

Here's a short writeup of a bit of my build system for a project I'm working on. It's pretty simple, and is just a relatively clean way of recording the repository state when code was compiled, so I can reproduce results later on. Just thought the interaction between git, cmake, and C++ was a bit nice!

Give Nix a look sometime, it takes this to a whole new level by including all of the build dependencies in the hash, and their build dependencies and so on. The standard flake workflow even includes the warning about having uncommitted files.

A simpler way to do this, especially if you do tagging in your repositories, is to use `git describe`. For example:

    $ git describe --dirty
    v1.4.1-1-gde18fe90-dirty

The format is <the most recent tag>-<the number of commits since that tag>-g<the short git hash>-<dirty, but only if the repo is dirty>.

If the repo isn't dirty, then the hash you get excludes that part:

    $ git describe --dirty
    v1.4.1-1-gde18fe90

If you're using lightweight tags (the default) and not annotated tags (with messages and signatures and etc) you may want to add `--tags` because otherwise it'll skip over any lightweight tags.

The other nice thing about this is that, if the repo is not -dirty, you can use the output from `git describe` in other git commands to reference that commit:

    $ git show -s v1.4.1-1-gde18fe90
    commit de18fe907edda2f2854e9813fcfbda9df902d8f1 (HEAD -> 1.4.1-release, origin/HEAD, origin/1.4.1-release)
    Author: rockowitz <rockowitz@minsoft.com>
    Date:   Sun May 28 17:09:46 2023 -0400

        Create codacy.yml

Logging Git hashes makes C++ builds reproducible and easy to track."

This is many useful things, but it's far from a reproducible C++ build. That'd require you ensure bit-for-bit identic builds when you reproduce, and logging the repository state is just a tiny first step to get there.

https://nikhilism.com/post/2020/windows-deterministic-builds... is a good resource on some of the other steps needed. It's... a non-trivial journey :)

nix fixes this

had to be said

That barely scratches the surface when it comes to reproducible c and c++ builds. In fact the topic of reproducible builds assumes your sources are the same, as in that's really not the problem here.

You need to control every single library header version you are using outside your source like stdlibs, os headers, third party, and have a strategy to deal with rand/datetime variables that can be part of the binary.

Git hashes have nothing whatsoever to do with whether you can do a clean build of the same tree twice with the same results, bit for bit.

Git hashes or tags can help identify what was built: the inputs.

You only need to know that for traceability: when you hold the released outputs, but do not hold (or are not sure you hold) the matching inputs.

If builds are reproducible, the traceability becomes more meaningful.

In the TXR project, have a ./configure option called --build-id. This sets an ID that is appended to the version, which is in the executable. It is nothing by default; not used. It is meant to be useful for people who interact with the code, so they can check what they are running (things can get confusing when you are going back and forth among versions, or making local changes).

If you set the build ID it to the word "git", then it is calculated using:

  git describe --tags --dirty

that's probably what this author should be using. It gives you a meaningful ID that is related to the most recent release tag, and whether the repo was dirty.

  $ git describe --tags --dirty
  txr-302-20-g77c99b74e-dirty

We are (sadly, only) 20 commits after 302, at a commit whose short hash is 77c99b74e, and the repo is in a modified state.

I have it rigged in the Makefile that it actually keeps track of the most recent build ID in a little .build_id file. If the build ID changes relative to what is in that file, the Makefile will force a rebuild of the .o files which incorporate the build ID.

Also, there is no need to be generating dynamic #include material just for this. A simple -Dsymbol=var option in the CFLAGS will define a preprocessor symbol:

  CFLAGS += -DMY_BUILD_ID=\"$(my_build_id)\"

For those of you using CMake, have a look at the module below:

https://github.com/xrootd/xrootd/blob/master/cmake/XRootDVer...

and also the genversion.sh script at the top of the repo.

I use these plus #cmakedefine and git tags to manage the project version without having to do it via commits.

As others have commented, this trick alone cannot ensure truly "reproducible" builds.

We used the same trick (git hash + git diff to monitor uncommitted changes) in a Python simulation framework we are developing for the JAXA/EU space mission "LiteBIRD." [1]

[1] https://iopscience.iop.org/article/10.1088/1475-7516/2025/11...