Contact
Evil Genius SocietyHomeLearnForumLeaderboardSocialsMessages
← Back to module

Lesson 10 of 10

Everything in Module 17 has been a fragment. This lesson assembles one project that does all of it — and the test of whether the layout is right is that somebody else can use the result, three different ways, without editing anything.

The shape

egs-tokenizer/
├── CMakeLists.txt          top level: options and subdirectories, nothing else
├── cmake/
│   └── egsConfig.cmake.in  template for the installed package config
├── include/egs/            the PUBLIC interface. Consumers see exactly this.
│   └── tokenizer.hpp
├── src/                    implementation + PRIVATE headers
│   ├── CMakeLists.txt
│   ├── tokenizer.cpp
│   └── internal/
│       ├── charclass.hpp
│       └── charclass.cpp
├── apps/                   executables
│   ├── CMakeLists.txt
│   └── tokcat.cpp
└── tests/
    ├── CMakeLists.txt
    └── test_tokenizer.cpp

Four directories, and each one is a decision from an earlier lesson:

  • include/egs/ — the published headers, nested under a project directory so a consumer writes #include "egs/tokenizer.hpp" and cannot collide with anybody else's tokenizer.hpp.
  • src/internal/ — private headers. Lesson 17.8 established why this must be a separate directory from apps/: a quoted #include searches the including file's own directory first, so adjacency defeats any include-path rule you write.
  • apps/ and tests/ — consumers of the library, built the same way an outside project would build against it.

The code

// include/egs/tokenizer.hpp
#pragma once
#include <string>
#include <string_view>
#include <vector>

namespace egs {
struct Token { std::string text; std::size_t column; };

std::vector<Token> tokenize(std::string_view line);
}
// src/internal/charclass.hpp
#pragma once

namespace egs::internal { bool is_separator(char c); }
// src/internal/charclass.cpp
#include "internal/charclass.hpp"

namespace egs::internal {
bool is_separator(char c) { return c == ' ' || c == '\t' || c == ',' || c == ';'; }
}
// src/tokenizer.cpp
#include "egs/tokenizer.hpp"
#include "internal/charclass.hpp"

namespace egs {
std::vector<Token> tokenize(std::string_view line) {
    std::vector<Token> out;
    std::size_t i = 0;
    while (i < line.size()) {
        while (i < line.size() && internal::is_separator(line[i])) ++i;
        if (i >= line.size()) break;
        const std::size_t start = i;
        while (i < line.size() && !internal::is_separator(line[i])) ++i;
        out.push_back({std::string(line.substr(start, i - start)), start + 1});
    }
    return out;
}
}
// apps/tokcat.cpp
#include <iostream>
#include "egs/tokenizer.hpp"

int main(int argc, char** argv) {
    const std::string line = argc > 1 ? argv[1] : "hello, world;  again";
    for (const egs::Token& t : egs::tokenize(line)) {
        std::cout << "  col " << t.column << ": " << t.text << "\n";
    }
    return 0;
}
// tests/test_tokenizer.cpp
#include <iostream>
#include "egs/tokenizer.hpp"

int failures = 0;

void expect_tokens(std::string_view in, std::vector<std::string> want) {
    auto got = egs::tokenize(in);
    bool ok = got.size() == want.size();
    for (std::size_t i = 0; ok && i < want.size(); ++i) ok = got[i].text == want[i];
    if (!ok) {
        ++failures;
        std::cout << "  FAIL tokenize(\"" << in << "\") gave " << got.size() << " tokens:";
        for (const auto& t : got) std::cout << " '" << t.text << "'";
        std::cout << "\n";
    }
}

int main() {
    expect_tokens("", {});
    expect_tokens("one", {"one"});
    expect_tokens("hello, world", {"hello", "world"});
    expect_tokens("  a;;b  c ", {"a", "b", "c"});
    std::cout << (failures ? "  tests failed\n" : "  all tokenizer tests passed\n");
    return failures ? 1 : 0;
}

The top level does almost nothing

# CMakeLists.txt
cmake_minimum_required(VERSION 3.20)
project(egs_tokenizer VERSION 0.1.0 LANGUAGES CXX)

# Only offer testing when this is the top-level project. A parent that
# FetchContents us does not want our tests in their build.
if(PROJECT_IS_TOP_LEVEL)
  enable_testing()
endif()

add_subdirectory(src)
add_subdirectory(apps)
if(PROJECT_IS_TOP_LEVEL)
  add_subdirectory(tests)
endif()

PROJECT_IS_TOP_LEVEL (CMake 3.21; CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR before that) is the polite version of lesson 17.9's TINYLOG_BUILD_TESTS option — it does the right thing with no configuration from the consumer at all.

The library

# src/CMakeLists.txt
add_library(tokenizer
  tokenizer.cpp
  internal/charclass.cpp
)
add_library(egs::tokenizer ALIAS tokenizer)

include(GNUInstallDirs)

# An exported target cannot advertise a path inside the source tree, because
# that path will not exist on the consumer's machine. BUILD_INTERFACE applies
# while building here; INSTALL_INTERFACE applies to an installed consumer.
target_include_directories(tokenizer
  PUBLIC
    $<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/include>
    $<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
  PRIVATE
    ${CMAKE_CURRENT_SOURCE_DIR}
)
target_compile_features(tokenizer PUBLIC cxx_std_20)
target_compile_options(tokenizer PRIVATE -Wall -Wextra -Wpedantic)

# ---- Installation, so consumers can find_package(egs) ----
install(TARGETS tokenizer EXPORT egsTargets)
install(DIRECTORY ${PROJECT_SOURCE_DIR}/include/ TYPE INCLUDE)

install(EXPORT egsTargets
  FILE        egsTargets.cmake
  NAMESPACE   egs::
  DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/egs
)

include(CMakePackageConfigHelpers)
write_basic_package_version_file(
  ${CMAKE_CURRENT_BINARY_DIR}/egsConfigVersion.cmake
  COMPATIBILITY SameMajorVersion
)
configure_package_config_file(
  ${PROJECT_SOURCE_DIR}/cmake/egsConfig.cmake.in
  ${CMAKE_CURRENT_BINARY_DIR}/egsConfig.cmake
  INSTALL_DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/egs
)
install(FILES
  ${CMAKE_CURRENT_BINARY_DIR}/egsConfig.cmake
  ${CMAKE_CURRENT_BINARY_DIR}/egsConfigVersion.cmake
  DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/egs
)
# cmake/egsConfig.cmake.in
@PACKAGE_INIT@

include("${CMAKE_CURRENT_LIST_DIR}/egsTargets.cmake")

check_required_components(egs)
# apps/CMakeLists.txt
add_executable(tokcat tokcat.cpp)
target_link_libraries(tokcat PRIVATE egs::tokenizer)
# tests/CMakeLists.txt
add_executable(test_tokenizer test_tokenizer.cpp)
target_link_libraries(test_tokenizer PRIVATE egs::tokenizer)
add_test(NAME tokenizer COMMAND test_tokenizer)

Note that apps/ and tests/ link egs::tokenizer — the alias, not the raw tokenizer target. Using the namespaced name everywhere means these files are identical to what an outside consumer writes, so if the alias ever breaks you find out in your own build.

Build, test, install

cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_INSTALL_PREFIX=$PWD/stage > /dev/null
cmake --build build > /dev/null && echo "built"
./build/apps/tokcat "Evil Genius, Society;  Cambodia"
echo "--- ctest:"
(cd build && ctest 2>&1 | tail -3)
built
  col 1: Evil
  col 6: Genius
  col 14: Society
  col 24: Cambodia
--- ctest:
100% tests passed, 0 tests failed out of 1

Total Test time (real) =   <time>
cmake --install build > /dev/null
echo "=== what got installed:"
find stage -type f | sed 's|^stage/||' | sort
=== what got installed:
include/egs/tokenizer.hpp
lib/cmake/egs/egsConfig.cmake
lib/cmake/egs/egsConfigVersion.cmake
lib/cmake/egs/egsTargets-relwithdebinfo.cmake
lib/cmake/egs/egsTargets.cmake
lib/libtokenizer.a

Six files, and that is a complete, self-describing package: the header, the archive, and the CMake config that tells a consumer how to use them.

The error you will hit

The first version of this src/CMakeLists.txt said simply target_include_directories(tokenizer PUBLIC ${PROJECT_SOURCE_DIR}/include). It built and tested perfectly, and then failed at configure time as soon as install(EXPORT ...) was added:

CMake Error in src/CMakeLists.txt:
  Target "tokenizer" INTERFACE_INCLUDE_DIRECTORIES property contains path:

    ".../egs-tokenizer/include"

  which is prefixed in the source directory.

CMake is refusing to export a promise it knows is false. INTERFACE_INCLUDE_DIRECTORIES is what consumers inherit, and a path inside your source tree will not exist on their machine. The $<BUILD_INTERFACE:...> / $<INSTALL_INTERFACE:...> pair is the answer: one path for building here, a different one once installed.

It is worth knowing that this error only appears when you add installation. A library that is only ever consumed by add_subdirectory can carry the wrong include directory forever without complaint.

Three ways to consume it

1. find_package, from the installed tree

// main.cpp
#include <iostream>
#include "egs/tokenizer.hpp"

int main() {
    for (const egs::Token& t : egs::tokenize("installed, and found")) {
        std::cout << "  " << t.column << " " << t.text << "\n";
    }
}
# consumer/CMakeLists.txt
cmake_minimum_required(VERSION 3.20)
project(downstream LANGUAGES CXX)

# Nothing here knows where egs lives, only that it is installed.
find_package(egs 0.1 REQUIRED)

add_executable(downstream main.cpp)
target_link_libraries(downstream PRIVATE egs::tokenizer)
mkdir -p consumer && cp main.cpp consumer/
cmake -S consumer -B cb -G Ninja -DCMAKE_PREFIX_PATH=$PWD/stage > /dev/null
cmake --build cb > /dev/null && ./cb/downstream
  1 installed
  12 and
  16 found

No include paths, no library paths, no C++ standard — all of it arrived through egs::tokenizer, which is what an exported target is for. CMAKE_PREFIX_PATH is the only thing that points at the installation.

The version is checked too:

# consumer/CMakeLists.txt
cmake_minimum_required(VERSION 3.20)
project(downstream LANGUAGES CXX)

find_package(egs 2.0 REQUIRED)

add_executable(downstream main.cpp)
target_link_libraries(downstream PRIVATE egs::tokenizer)
cmake -S consumer -B cb2 -G Ninja -DCMAKE_PREFIX_PATH=$PWD/stage 2>&1 | grep -A2 'Could not find' | head -4
  Could not find a configuration file for package "egs" that is compatible
  with requested version "2.0".

COMPATIBILITY SameMajorVersion is doing that: 0.1 satisfies a request for 0.1, and refuses 2.0.

2. FetchContent, from the repository

git init -q && git add -A
git -c user.email=dev@example.com -c user.name=dev commit -qm "egs-tokenizer 0.1.0"
git tag v0.1.0
echo "tagged $(git tag)"
tagged v0.1.0
# fetcher/CMakeLists.txt
cmake_minimum_required(VERSION 3.20)
project(fetching LANGUAGES CXX)

include(FetchContent)
FetchContent_Declare(egs GIT_REPOSITORY ${PARENT_REPO} GIT_TAG v0.1.0)
FetchContent_MakeAvailable(egs)

add_executable(fetching main.cpp)
target_link_libraries(fetching PRIVATE egs::tokenizer)
mkdir -p fetcher && cp main.cpp fetcher/
cmake -S fetcher -B fb -G Ninja -DPARENT_REPO=$PWD > /dev/null
cmake --build fb > /dev/null && ./fb/fetching
echo "--- did the dependency's own tests come along?"
test -d fb/_deps/egs-build/tests && echo "  yes (that would be a nuisance)" || echo "  no"
echo "--- tests ctest knows about in the consumer:"
(cd fb && ctest -N 2>&1 | tail -2)
  1 installed
  12 and
  16 found
--- did the dependency's own tests come along?
  no
--- tests ctest knows about in the consumer:

Total Tests: 0

Total Tests: 0. PROJECT_IS_TOP_LEVEL was false inside the fetched copy, so tests/ was never added and the consumer's ctest is not polluted with somebody else's test suite. The same source tree behaves correctly in both roles.

3. add_subdirectory, vendored

The third way needs no demonstration because it is what FetchContent_MakeAvailable just did — add_subdirectory(third_party/egs-tokenizer EXCLUDE_FROM_ALL) against a checked-in copy behaves identically, minus the clone.

That all three work from one unmodified tree is the actual deliverable of this module.

The checklist

For any C++ library you expect somebody else to use:

  1. Public headers in include/<project>/, so includes are unambiguous.
  2. Private headers in src/, and consumers in a different directory.
  3. Every target declares its own requirements with target_* and the right PUBLIC/PRIVATE/INTERFACE.
  4. $<BUILD_INTERFACE> / $<INSTALL_INTERFACE> on any include directory you export.
  5. An ALIAS target with a namespace, used by your own apps/ and tests/ too.
  6. Tests and apps behind PROJECT_IS_TOP_LEVEL.
  7. install(TARGETS ... EXPORT ...) plus a generated package config.
  8. A VERSION in project() and a version file with a stated compatibility rule.

Try it

Add a second library to the project — one that depends on the first — and export both from the same package. Then prove a consumer gets the transitive dependency automatically, and that the private headers of neither library are reachable.

Checkpoint

  • Public headers in include/<project>/; consumers write #include "egs/tokenizer.hpp".
  • Private headers in src/, with apps and tests in different directories — adjacency beats include paths (lesson 17.8).
  • The top-level CMakeLists.txt should only set options and add subdirectories.
  • PROJECT_IS_TOP_LEVEL gates tests and apps, so a consumer's build is not polluted.
  • Verified: a fetched copy of the project contributed Total Tests: 0 to the consumer's ctest.
  • Always add a namespaced ALIAS, and use it in your own apps/ and tests/ as well.
  • An exported target cannot advertise a source-tree path — CMake errors with prefixed in the source directory.
  • Fix with $<BUILD_INTERFACE:...> and $<INSTALL_INTERFACE:...>.
  • That error only appears once you add installation; an add_subdirectory-only library hides it.
  • install(TARGETS ... EXPORT ...) + install(EXPORT ... NAMESPACE egs::) + configure_package_config_file makes find_package work.
  • Give project() a VERSION and write a version file with a stated COMPATIBILITY.
  • Verified: find_package(egs 0.1) succeeded and find_package(egs 2.0) was refused under SameMajorVersion.
  • Inside an exported target, link namespaced names — a bare name lands in the generated config and breaks consumers.
  • Verified: a consumer linking only egs::counter got libtokenizer.a transitively.
  • Packaging enforces privacy that include paths only advise on: an unshipped header cannot be included.

Questions about this lesson

Log in to ask a question.