Skip to content

Build system

Jump to bottom
Franz Miltz edited this page Aug 10, 2021 · 4 revisions

Quick start

To build the project, do the following:

  1. Create an empty directory buid/ inside the repository folder.
  2. Enter build/ and run cmake .., this will configure the hyped project inside the empty folder. The options you can specify are
    • -DRELEASE=ON (or OFF)
    • -DPEDANTIC=ON (or OFF)
    • -DCOVERAGE=ON (or OFF)
  3. Run make -j4 <target>. This will build the target using four CPU cores. The targets are:
    • hyped: the main binary
    • testrunner: the binary to run the tests
    • test: builds testrunner, configures and runs it
    • coverage: creates a coverage report in <build-dir>/test/coverage/html; only if -DCOVERAGE=ON was set

More details

Why not GNU Make?

The build system was revamped in 2022. Here's what was wrong with the old setup:

  1. Hand-written Make files are hard to understand which is the opposite of what we want to achieve in HYPED.
  2. Making changes was difficult because the dependencies were unclear and the non-standard structure stopped members to get help online.
  3. Most projects and companies use CMake which is why not having it prevents members from getting accustomed to it.

That being said, I encourage anyone who decides to make something in C or C++ to set up some Make files by hand. It's a valuable experience, but it limits productivity in way that is not justifiable for the HYPED code base.

Why CMake?

As of today, CMake is the king in the industry and it's hard to see it go away. That in itself would be enough of a reason to adopt it. However, there are a few more:

  1. Easy to reason about: Instead of specifying every step that has to be executed to build a project, CMake allows you to say what the outcome is supposed to be.
  2. The libraries we use, like rapidjson, Eigen and GoogleTest all support CMake and are thus very easy to integrate into our project.
  3. CMake manages compilers and compile flags by design.
  4. It achieves everything that GNU Make does.

How does CMake work?

As opposed to GNU Make, CMake is strictly speaking not a build system. Rather, it generates build files, in our case Makefiles, that build the project in the way we specified. This is reflected in the way you use it:

The old approach to compiling the hyped binary was to run

$ make

in the top level of the directory.

Now, we have to configure CMake first by running

$ mkdir build
$ cd build
$ cmake ..

or if you prefer a single command

$ cmake -S . -B build

Then you can run your make commands inside the build/ directory as usual. The final binary will be created at run/hyped.

Targets

CMake allows you to specify three types of build targets:

  1. libraries: binaries that don't run by themselves but are linked against, i.e. used by other targets. For example, we compile each subdirectory of src/ into a separate library that are then included by the final executable.
  2. executables: binaries that can be run. We have two of those, namely hyped (the main program) and testrunner (the binary that contains all the tests).
  3. custom targets: instead of specifying a C++ binary, we tell CMake what it has to do to build the target. This can be whatever we want, really. We use this to format all the source code before compilation and to run the test runner after compilation.

Setting up the compiler

As of right now, the system only supports 64-bit linux builds. This is obviously a problem as the software is designed to be run on the BBB which is has an ARM CPU. This will be fixed in the beginning of the semester.

Here is how we set the compile flags at the moment:

set(CMAKE_CXX_COMPILER "clang++")
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

option(COVERAGE "Makes the binary produce coverage information")
if(COVERAGE)
  message("coverage:  ON")
  set(COVERAGE_FLAGS "-fprofile-instr-generate -fcoverage-mapping")
else()
  message("coverage:  OFF")
  set(COVERAGE_FLAGS "")
endif()

option(RELEASE "Configures the binary for release")
if (RELEASE)
  message("release:   ON")
  set(OPTIMISATION_FLAGS "-O2")
else()
  message("release:   OFF")
  set(OPTIMISATION_FLAGS "-Og")
endif()

option(PEDANTIC "Enable pedantic warnings" ON)
if(PEDANTIC)
  message("pedantic:  ON")
  set(WARN_FLAGS "-Wall -Wextra -Wpedantic")
else()
  message("pedantic:  OFF")
  set(WARN_FLAGS "")
endif()

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -pthread ${COVERAGE_FLAGS} ${OPTIMISATION_FLAGS} ${WARN_FLAGS}")
set(CMAKE_CXX_LINKER_FLAGS "${CMAKE_CXX_LINKER_FLAGS} -stdlib=libc++ -lpthread ${WARN_FLAGS}")

message("compile flags: ${CMAKE_CXX_FLAGS}")
message("link flags:    ${CMAKE_CXX_LINKER_FLAGS}")

add_compile_definitions(ARCH_64)
add_compile_definitions(LINUX)

As you can see, there are a few options:

  1. RELEASE to specify a release build. This should be used if the binary is supposed to be run on the BBB.
  2. COVERAGE to generate coverage information. This is used in CI.
  3. PEDANTIC to generate warnings. This is on by default but can be turned off, for example for CI.

You can configure cmake with the desired values as follows:

$ cmake .. -DRELEASE=ON -DCOVERAGE=OFF -DPEDANTIC=ON

The default setup should be fine in most cases, though.

Additionally you can see that we add some more flags, namely -pthread, -lpthread and -stdlib=libc++. These tell the compiler and linker that we want to use the C++ standard library and POSIX threads.

Finally, we use CMake to insert macro definitions into the code. As of right now, these are required for successful compilation.

Building the libraries

Eigen & rapidjson

The first libraries that are being built are rapidjson and Eigen. Both of these are header only, i.e. no source code has to be compiled, so we only need to download them and then tell CMake where it can find the header files. We do this by using the ExternalProject package:

include(ExternalProject)

ExternalProject_Add(
    rapidjson
    PREFIX "lib/rapidjson"
    GIT_REPOSITORY "https://github.com/Tencent/rapidjson.git"
    TIMEOUT 10
    CMAKE_ARGS
        -DRAPIDJSON_BUILD_TESTS=OFF
        -DRAPIDJSON_BUILD_DOC=OFF
        -DRAPIDJSON_BUILD_EXAMPLES=OFF
    CONFIGURE_COMMAND ""
    BUILD_COMMAND ""
    INSTALL_COMMAND ""
    UPDATE_COMMAND ""
)
ExternalProject_Get_Property(rapidjson source_dir)
set(RAPIDJSON_INCLUDE_DIR ${source_dir}/include)
set_target_properties(rapidjson PROPERTIES EXCLUDE_FROM_ALL TRUE)

This clones a git repository and then we define the ${RAPIDJSON_INCLUDE_DIR} to add it to the include dirs later on.

GoogleTest

While the testing library is installed similarly, it requires a bit more work. In order to use it, we need to compile it and link our testrunner binary to it.

include(ExternalProject)

ExternalProject_Add(
    googletest
    PREFIX "lib/gtest"
    GIT_REPOSITORY "https://github.com/google/googletest"
    TIMEOUT 10
    INSTALL_COMMAND ""
)

ExternalProject_Get_Property(googletest source_dir)
set(GTEST_INCLUDE_DIR ${source_dir}/googletest/include)
set_target_properties(googletest PROPERTIES EXCLUDE_FROM_ALL TRUE)

ExternalProject_Get_Property(googletest binary_dir)
set(GTEST_LIBRARY_PATH ${binary_dir}/lib/libgtest.a)
set(GTEST_LIBRARY gtest)
add_library(${GTEST_LIBRARY} UNKNOWN IMPORTED)
set_property(TARGET ${GTEST_LIBRARY} PROPERTY IMPORTED_LOCATION
                ${GTEST_LIBRARY_PATH} )
add_dependencies(${GTEST_LIBRARY} googletest)
set_target_properties(gtest PROPERTIES EXCLUDE_FROM_ALL TRUE)

We have to create a library target that we can later link against. Of course, we don't build the library explicitly so we have to tell CMake "Trust us, it will be there once you've built the external project."

Internal libraries

Each subfolder of the src/ directory is build into a separate library binary. The way to do this is specified in the CMakeLists.txt file in each of those directories. They all look very similar. In particular, most of the work is done inside this function:

set(ALL_LIBS "data;embrakes;navigation;propulsion;propulsion_can;sensors;state_machine;telemetry;utils;utils_concurrent;utils_io;utils_math")
function(make_lib target, include_path)
    file(GLOB headers "${CMAKE_CURRENT_SOURCE_DIR}/*.hpp")
    file(GLOB code "${CMAKE_CURRENT_SOURCE_DIR}/*.cpp")
    add_library(${target} STATIC ${headers} ${code})
    target_include_directories(${target}
        INTERFACE ${include_path}
    )
    set(link_libs ${ALL_LIBS})
    list(REMOVE_ITEM link_libs "${target}")
    target_link_libraries(${target} ${link_libs})
    add_custom_target("${target}-format"
        COMMAND clang-format -i -style=file ${code} ${headers}
    )
    add_dependencies(${target} "${target}-format")
endfunction()

We first find all the *.cpp and *.hpp files and specify a static library with them. Then we specify the include directories, i.e. the place where other binaries that are linked to this one can find the header files. This is usually the parent directory of the library itself, so that we can include src/state_machine/main.hpp with #include <state_machine/main.hpp>. There are some exception, as we want to be able to use #include <utils/concurrent/thread.hpp>.

Then we create a list called link_libs which contains all other libraries except the target (because we can't link a binary with itself).

Finally we specify a format target which runs clang-format for all the source files as a dependency of the library. This forces all source code to be formatted in place before compilation.

Building the executables

Building the hyped executable is now a very simple task. We simply need to add the run/main.cpp file to the executable and then link all the libraries.

set(target "hyped")
add_executable(${target} ${CMAKE_SOURCE_DIR}/run/main.cpp)
target_link_libraries(${target}
    data
    embrakes
    navigation
    propulsion
    propulsion_can
    sensors
    state_machine
    telemetry
    utils
    utils_concurrent
    utils_io
    utils_math
)

The same goes for the testrunner. This is a bit more complicated because we need to find more than one source file and also do the formatting manually.

Test and coverage targets

Once we have built the binaries, we can specify some less straight forward targets. Firstly, there's the test target which runs the testrunner.

add_custom_target(test
    COMMAND cp -r ${HYPED_CONFIG_DIR} ${CMAKE_BINARY_DIR}/test/
    COMMAND ${TEST_BINARY}
    DEPENDS testrunner
)

Then there is the coverage target which runs the tests and generates coverage which can then be inspected or even uploaded to Codecov. This essentially involves running a sequence of shell commands which generate the data and then convert it to a usable format:

set(LLVM_COV_DIR "${CMAKE_CURRENT_BINARY_DIR}/coverage")
set(LLVM_COV_RAW "${LLVM_COV_DIR}/hyped.profraw")
set(LLVM_COV_DATA "${LLVM_COV_DIR}/hyped.profdata")
set(LCOV_DATA "${LLVM_COV_DIR}/hyped.covdata")

add_custom_target(coverage
    COMMAND mkdir -p ${LLVM_COV_DIR}
    COMMAND sh -c "LLVM_PROFILE_FILE=\"${LLVM_COV_RAW}\" ${TEST_BINARY}"
    COMMAND llvm-profdata merge -sparse ${LLVM_COV_RAW} -o ${LLVM_COV_DATA}
    COMMAND llvm-cov export -format=lcov -instr-profile=${LLVM_COV_DATA} ${TEST_BINARY} > ${LCOV_DATA}
    COMMAND genhtml --quiet --output-directory ${LLVM_COV_DIR}/html ${LCOV_DATA}
    DEPENDS testrunner
)

The coverage report can be found under build/test/coverage/html. However, the information uploaded to Codecov is probably more interesting.

HYPED Logo

Clone this wiki locally