Lesson 10 of 10
Everything in Module 17 has been a fragment. This lesson assembles one project that does all of it — and the test of whether the layout is right is that somebody else can use the result, three different ways, without editing anything.
The shape
egs-tokenizer/
├── CMakeLists.txt top level: options and subdirectories, nothing else
├── cmake/
│ └── egsConfig.cmake.in template for the installed package config
├── include/egs/ the PUBLIC interface. Consumers see exactly this.
│ └── tokenizer.hpp
├── src/ implementation + PRIVATE headers
│ ├── CMakeLists.txt
│ ├── tokenizer.cpp
│ └── internal/
│ ├── charclass.hpp
│ └── charclass.cpp
├── apps/ executables
│ ├── CMakeLists.txt
│ └── tokcat.cpp
└── tests/
├── CMakeLists.txt
└── test_tokenizer.cpp
Four directories, and each one is a decision from an earlier lesson:
include/egs/— the published headers, nested under a project directory so a consumer writes#include "egs/tokenizer.hpp"and cannot collide with anybody else'stokenizer.hpp.src/internal/— private headers. Lesson 17.8 established why this must be a separate directory fromapps/: a quoted#includesearches the including file's own directory first, so adjacency defeats any include-path rule you write.apps/andtests/— consumers of the library, built the same way an outside project would build against it.
The code
// include/egs/tokenizer.hpp
#pragma once
#include <string>
#include <string_view>
#include <vector>
namespace egs {
struct Token { std::string text; std::size_t column; };
std::vector<Token> tokenize(std::string_view line);
}
// src/internal/charclass.hpp
#pragma once
namespace egs::internal { bool is_separator(char c); }
// src/internal/charclass.cpp
#include "internal/charclass.hpp"
namespace egs::internal {
bool is_separator(char c) { return c == ' ' || c == '\t' || c == ',' || c == ';'; }
}
// src/tokenizer.cpp
#include "egs/tokenizer.hpp"
#include "internal/charclass.hpp"
namespace egs {
std::vector<Token> tokenize(std::string_view line) {
std::vector<Token> out;
std::size_t i = 0;
while (i < line.size()) {
while (i < line.size() && internal::is_separator(line[i])) ++i;
if (i >= line.size()) break;
const std::size_t start = i;
while (i < line.size() && !internal::is_separator(line[i])) ++i;
out.push_back({std::string(line.substr(start, i - start)), start + 1});
}
return out;
}
}
// apps/tokcat.cpp
#include <iostream>
#include "egs/tokenizer.hpp"
int main(int argc, char** argv) {
const std::string line = argc > 1 ? argv[1] : "hello, world; again";
for (const egs::Token& t : egs::tokenize(line)) {
std::cout << " col " << t.column << ": " << t.text << "\n";
}
return 0;
}
// tests/test_tokenizer.cpp
#include <iostream>
#include "egs/tokenizer.hpp"
int failures = 0;
void expect_tokens(std::string_view in, std::vector<std::string> want) {
auto got = egs::tokenize(in);
bool ok = got.size() == want.size();
for (std::size_t i = 0; ok && i < want.size(); ++i) ok = got[i].text == want[i];
if (!ok) {
++failures;
std::cout << " FAIL tokenize(\"" << in << "\") gave " << got.size() << " tokens:";
for (const auto& t : got) std::cout << " '" << t.text << "'";
std::cout << "\n";
}
}
int main() {
expect_tokens("", {});
expect_tokens("one", {"one"});
expect_tokens("hello, world", {"hello", "world"});
expect_tokens(" a;;b c ", {"a", "b", "c"});
std::cout << (failures ? " tests failed\n" : " all tokenizer tests passed\n");
return failures ? 1 : 0;
}
The top level does almost nothing
# CMakeLists.txt
cmake_minimum_required(VERSION 3.20)
project(egs_tokenizer VERSION 0.1.0 LANGUAGES CXX)
# Only offer testing when this is the top-level project. A parent that
# FetchContents us does not want our tests in their build.
if(PROJECT_IS_TOP_LEVEL)
enable_testing()
endif()
add_subdirectory(src)
add_subdirectory(apps)
if(PROJECT_IS_TOP_LEVEL)
add_subdirectory(tests)
endif()
PROJECT_IS_TOP_LEVEL (CMake 3.21; CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR before
that) is the polite version of lesson 17.9's TINYLOG_BUILD_TESTS option — it does the right thing
with no configuration from the consumer at all.
The library
# src/CMakeLists.txt
add_library(tokenizer
tokenizer.cpp
internal/charclass.cpp
)
add_library(egs::tokenizer ALIAS tokenizer)
include(GNUInstallDirs)
# An exported target cannot advertise a path inside the source tree, because
# that path will not exist on the consumer's machine. BUILD_INTERFACE applies
# while building here; INSTALL_INTERFACE applies to an installed consumer.
target_include_directories(tokenizer
PUBLIC
$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
PRIVATE
${CMAKE_CURRENT_SOURCE_DIR}
)
target_compile_features(tokenizer PUBLIC cxx_std_20)
target_compile_options(tokenizer PRIVATE -Wall -Wextra -Wpedantic)
# ---- Installation, so consumers can find_package(egs) ----
install(TARGETS tokenizer EXPORT egsTargets)
install(DIRECTORY ${PROJECT_SOURCE_DIR}/include/ TYPE INCLUDE)
install(EXPORT egsTargets
FILE egsTargets.cmake
NAMESPACE egs::
DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/egs
)
include(CMakePackageConfigHelpers)
write_basic_package_version_file(
${CMAKE_CURRENT_BINARY_DIR}/egsConfigVersion.cmake
COMPATIBILITY SameMajorVersion
)
configure_package_config_file(
${PROJECT_SOURCE_DIR}/cmake/egsConfig.cmake.in
${CMAKE_CURRENT_BINARY_DIR}/egsConfig.cmake
INSTALL_DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/egs
)
install(FILES
${CMAKE_CURRENT_BINARY_DIR}/egsConfig.cmake
${CMAKE_CURRENT_BINARY_DIR}/egsConfigVersion.cmake
DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/egs
)
# cmake/egsConfig.cmake.in
@PACKAGE_INIT@
include("${CMAKE_CURRENT_LIST_DIR}/egsTargets.cmake")
check_required_components(egs)
# apps/CMakeLists.txt
add_executable(tokcat tokcat.cpp)
target_link_libraries(tokcat PRIVATE egs::tokenizer)
# tests/CMakeLists.txt
add_executable(test_tokenizer test_tokenizer.cpp)
target_link_libraries(test_tokenizer PRIVATE egs::tokenizer)
add_test(NAME tokenizer COMMAND test_tokenizer)
Note that apps/ and tests/ link egs::tokenizer — the alias, not the raw tokenizer
target. Using the namespaced name everywhere means these files are identical to what an outside
consumer writes, so if the alias ever breaks you find out in your own build.
Build, test, install
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_INSTALL_PREFIX=$PWD/stage > /dev/null
cmake --build build > /dev/null && echo "built"
./build/apps/tokcat "Evil Genius, Society; Cambodia"
echo "--- ctest:"
(cd build && ctest 2>&1 | tail -3)
built
col 1: Evil
col 6: Genius
col 14: Society
col 24: Cambodia
--- ctest:
100% tests passed, 0 tests failed out of 1
Total Test time (real) = <time>
cmake --install build > /dev/null
echo "=== what got installed:"
find stage -type f | sed 's|^stage/||' | sort
=== what got installed:
include/egs/tokenizer.hpp
lib/cmake/egs/egsConfig.cmake
lib/cmake/egs/egsConfigVersion.cmake
lib/cmake/egs/egsTargets-relwithdebinfo.cmake
lib/cmake/egs/egsTargets.cmake
lib/libtokenizer.a
Six files, and that is a complete, self-describing package: the header, the archive, and the CMake config that tells a consumer how to use them.
The error you will hit
The first version of this src/CMakeLists.txt said simply
target_include_directories(tokenizer PUBLIC ${PROJECT_SOURCE_DIR}/include). It built and tested
perfectly, and then failed at configure time as soon as install(EXPORT ...) was added:
CMake Error in src/CMakeLists.txt:
Target "tokenizer" INTERFACE_INCLUDE_DIRECTORIES property contains path:
".../egs-tokenizer/include"
which is prefixed in the source directory.
CMake is refusing to export a promise it knows is false. INTERFACE_INCLUDE_DIRECTORIES is what
consumers inherit, and a path inside your source tree will not exist on their machine. The
$<BUILD_INTERFACE:...> / $<INSTALL_INTERFACE:...> pair is the answer: one path for building here,
a different one once installed.
It is worth knowing that this error only appears when you add installation. A library that is only
ever consumed by add_subdirectory can carry the wrong include directory forever without complaint.
Three ways to consume it
1. find_package, from the installed tree
// main.cpp
#include <iostream>
#include "egs/tokenizer.hpp"
int main() {
for (const egs::Token& t : egs::tokenize("installed, and found")) {
std::cout << " " << t.column << " " << t.text << "\n";
}
}
# consumer/CMakeLists.txt
cmake_minimum_required(VERSION 3.20)
project(downstream LANGUAGES CXX)
# Nothing here knows where egs lives, only that it is installed.
find_package(egs 0.1 REQUIRED)
add_executable(downstream main.cpp)
target_link_libraries(downstream PRIVATE egs::tokenizer)
mkdir -p consumer && cp main.cpp consumer/
cmake -S consumer -B cb -G Ninja -DCMAKE_PREFIX_PATH=$PWD/stage > /dev/null
cmake --build cb > /dev/null && ./cb/downstream
1 installed
12 and
16 found
No include paths, no library paths, no C++ standard — all of it arrived through egs::tokenizer,
which is what an exported target is for. CMAKE_PREFIX_PATH is the only thing that points at the
installation.
The version is checked too:
# consumer/CMakeLists.txt
cmake_minimum_required(VERSION 3.20)
project(downstream LANGUAGES CXX)
find_package(egs 2.0 REQUIRED)
add_executable(downstream main.cpp)
target_link_libraries(downstream PRIVATE egs::tokenizer)
cmake -S consumer -B cb2 -G Ninja -DCMAKE_PREFIX_PATH=$PWD/stage 2>&1 | grep -A2 'Could not find' | head -4
Could not find a configuration file for package "egs" that is compatible
with requested version "2.0".
COMPATIBILITY SameMajorVersion is doing that: 0.1 satisfies a request for 0.1, and refuses 2.0.
2. FetchContent, from the repository
git init -q && git add -A
git -c user.email=dev@example.com -c user.name=dev commit -qm "egs-tokenizer 0.1.0"
git tag v0.1.0
echo "tagged $(git tag)"
tagged v0.1.0
# fetcher/CMakeLists.txt
cmake_minimum_required(VERSION 3.20)
project(fetching LANGUAGES CXX)
include(FetchContent)
FetchContent_Declare(egs GIT_REPOSITORY ${PARENT_REPO} GIT_TAG v0.1.0)
FetchContent_MakeAvailable(egs)
add_executable(fetching main.cpp)
target_link_libraries(fetching PRIVATE egs::tokenizer)
mkdir -p fetcher && cp main.cpp fetcher/
cmake -S fetcher -B fb -G Ninja -DPARENT_REPO=$PWD > /dev/null
cmake --build fb > /dev/null && ./fb/fetching
echo "--- did the dependency's own tests come along?"
test -d fb/_deps/egs-build/tests && echo " yes (that would be a nuisance)" || echo " no"
echo "--- tests ctest knows about in the consumer:"
(cd fb && ctest -N 2>&1 | tail -2)
1 installed
12 and
16 found
--- did the dependency's own tests come along?
no
--- tests ctest knows about in the consumer:
Total Tests: 0
Total Tests: 0. PROJECT_IS_TOP_LEVEL was false inside the fetched copy, so tests/ was never
added and the consumer's ctest is not polluted with somebody else's test suite. The same source
tree behaves correctly in both roles.
3. add_subdirectory, vendored
The third way needs no demonstration because it is what FetchContent_MakeAvailable just did —
add_subdirectory(third_party/egs-tokenizer EXCLUDE_FROM_ALL) against a checked-in copy behaves
identically, minus the clone.
That all three work from one unmodified tree is the actual deliverable of this module.
The checklist
For any C++ library you expect somebody else to use:
- Public headers in
include/<project>/, so includes are unambiguous. - Private headers in
src/, and consumers in a different directory. - Every target declares its own requirements with
target_*and the rightPUBLIC/PRIVATE/INTERFACE. $<BUILD_INTERFACE>/$<INSTALL_INTERFACE>on any include directory you export.- An
ALIAStarget with a namespace, used by your ownapps/andtests/too. - Tests and apps behind
PROJECT_IS_TOP_LEVEL. install(TARGETS ... EXPORT ...)plus a generated package config.- A
VERSIONinproject()and a version file with a stated compatibility rule.
Try it
Add a second library to the project — one that depends on the first — and export both from the same package. Then prove a consumer gets the transitive dependency automatically, and that the private headers of neither library are reachable.
Checkpoint
- Public headers in
include/<project>/; consumers write#include "egs/tokenizer.hpp". - Private headers in
src/, with apps and tests in different directories — adjacency beats include paths (lesson 17.8). - The top-level
CMakeLists.txtshould only set options and add subdirectories. PROJECT_IS_TOP_LEVELgates tests and apps, so a consumer's build is not polluted.- Verified: a fetched copy of the project contributed
Total Tests: 0to the consumer'sctest. - Always add a namespaced
ALIAS, and use it in your ownapps/andtests/as well. - An exported target cannot advertise a source-tree path — CMake errors with
prefixed in the source directory. - Fix with
$<BUILD_INTERFACE:...>and$<INSTALL_INTERFACE:...>. - That error only appears once you add installation; an
add_subdirectory-only library hides it. install(TARGETS ... EXPORT ...)+install(EXPORT ... NAMESPACE egs::)+configure_package_config_filemakesfind_packagework.- Give
project()aVERSIONand write a version file with a statedCOMPATIBILITY. - Verified:
find_package(egs 0.1)succeeded andfind_package(egs 2.0)was refused underSameMajorVersion. - Inside an exported target, link namespaced names — a bare name lands in the generated config and breaks consumers.
- Verified: a consumer linking only
egs::countergotlibtokenizer.atransitively. - Packaging enforces privacy that include paths only advise on: an unshipped header cannot be included.
Questions about this lesson
Log in to ask a question.