Preparing a distribution

These instructions are primarily for our own benefit. Lest we forget.

When creating a new release, ensure the following have been done:

  • Python dependencies and Python maximum version need to be updated

    • in cwipc_util/python/pyproject.toml remove the dependency specifiers,

    • build on all platforms, ensure everything works, possibly lowering the versions of some dependencies, and possibly lowering the maximum Python version

    • Especially watch out for opencv and open3d which can some times lag 2 Python versions

      • Add the Python package dependency specifiers again for the currently selected versions.

      • Update the Python version range in the toplevel CMakeLists.txt.

      • Update scripts/install-3rdparty-windows.ps1 with the best Python version.

      • Update scripts/install-3rdparty-macos.sh with the best Python version.

      • Check the Ubuntu install-3rdparty scripts for which Python they install.

      • Check .github/workflows/build.yml for the Python versions used.

  • Check whether nlohman_json can be updated (CMakeLists.txt)

  • Check whether nsis can be updated (.github/workflows/build.yml)

  • Check whether the slurped and modified cpack config files for nsis (in CMakeFiles) need to be updated.

  • Dependencies for the .deb installer for apt/Ubuntu need to be updated. There may be better ways to do this, but this works

    • On the targeted Ubuntu, check out and edit CMakeFiles/CwipcInstallers.cmake

    • Comment out the definitions for CPACK_DEBIAN_PACKAGE_DEPENDS and CPACK_DEBIAN_PACKAGE_RECOMMENDS.

    • Un-comment-out CPACK_DEBIAN_PACKAGE_SHLIBDEPS YES.

    • Build, run cpack with:

      cpack --config build/CPackConfig.cmake -D CPACK_DEBIAN_FILE_NAME="cwipc_test_ubuntu2404.deb"

    • Extract the resulting debian package with ar x.

    • Unpack the control.tar.gz file.

    • Inspect the dependencies that cpack auto-generated.

    • Fix the dependencies and recommendations based on what cpack found.

  • scripts/install-3rdparty-windows.ps1 should be updated to download the most recent compatible packages. Go through each of the packages, determine the current version. Uninstall old versions from your build machine. Run the powershell script to test it installs the new packages. Do the build, to ensure it works with the new packages. Test the build to ensure it runs with the new packages.

    • Note: the only package that is important here nowadays is Python, because the other other two left here, k4a and k4abt, are no longer maintained.

  • For Windows and Android, the vcpkg dependent packages should all be updated to the most recent version:

    cd .\vcpkg
git checkout master
git pull
.\bootstrap-vcpkg.bat
cd ..
.\vcpkg\vcpkg x-update-baseline
.\vcpkg\vcpkg.exe install
git commit -a -m "Vcpkg packages updated to most recent version"

  • The toplevel vcpkg.json has a version string.

  • setup.py in cwipc_util/python and every other package has a default version string that is only used when installing with pip install -e (because usually it is dynamically determined at build time. For good measure update these default version strings when doing a major release.

  • CWIPC_API_VERSION incremented if there are any API changes (additions only).

  • CWIPC_API_VERSION_OLD incremented if there are API changes that are not backward compatible.

    • Both these need to be changed in api.h and cwipc/util.py.

  • CHANGELOG.md updated.

Version numbers for the release no longer need to be updated manually, but note the exceptions above.

After making all these changes push to github. Ensure the CI/CD build passes (easiest is by running ./scripts/nightly.sh which does a nightly build). This build will take a looooong time, most likely, because the vcpkg dependencies have been updated and the Windows runner will have to rebuild the world.

After that tag all submodules and the main module with v_X_._Y_._Z_.

If one of the next steps fails just fix the issue and do another micro-release. Has happened to me every single release, I think:-)

Push the tag to github, this will build the release.

After the release is built copy the relevant new section of CHANGELOG.md to the release notes.

After that, update the brew formula at <https://github.com/cwi-dis/homebrew-cwipc>. Use:

  • brew edit cwipc and change the URL and version (and possibly Python or other dependencies),

  • brew fetch cwipc to get the error about the SHA mismatch, fix the SHA,

  • brew install to install the new version,

  • then push the changes (easy from within vscode),

  • then brew upgrade cwipc on another machine to test.

Finally, when you are happy that everything works, edit the release on the github web interface and clear the prerelease flag.

Continuous integration

The project uses GitHub Actions (.github/workflows/build.yml) to build and package on every push. Tags matching v* trigger a release job which uploads artifacts to GitHub Releases and updates the Homebrew formula.

Building and publishing the documentation

Documentation is generated from reStructuredText files in docs/ using Sphinx with the ReadTheDocs theme. You will need the packages listed in docs/requirements.txt (sphinx, myst-parser, sphinx-rtd-theme).

To build locally:

cd docs
python -m pip install -r requirements.txt
sphinx-build -b html . _build/html

The HTML output appears in docs/_build/html. Run the same commands in a CI job to verify formatting or catch broken links.

On ReadTheDocs the project https://cwipc.readthedocs.io is configured to use this docs/ folder as the source. Currently you have to manually trigger a build there. The project page is at https://app.readthedocs.org/projects/cwipc/ and it may be that only Jack can do this at the moment.