Lesson 9 of 9
Part III has given you templates, the standard library, lambdas, error handling, and class design. This project uses all five to build something you would otherwise reach for a library to do: a JSON parser and serializer that round-trips a document exactly.
It is about 250 lines. Every line below has been compiled with -Wall -Wextra, run, and checked
under AddressSanitizer and UndefinedBehaviorSanitizer.
The specification
Parse the whole of JSON — objects, arrays, strings with escapes, numbers, true, false, null —
into a value type you can inspect, and write it back out compact or pretty-printed. Specifically:
- Round-trip.
dump(parse(dump(v))) == dump(v)for every document. - Reject bad input with a position. Not just "parse error" — line and column.
- Survive hostile input. A megabyte of
[must not crash the process. - Handle Khmer.
យើងmust decode toយើង, and UTF-8 must pass back out intact. - No leaks, no undefined behaviour, proven with sanitizers.
Requirement 5 is not decoration. A recursive parser that allocates on every level is exactly the
shape of program where a missed delete hides for years.
The value type
A JSON value is one of six things. That is the definition of std::variant (lesson 13.10).
// ASSEMBLE json 1 -- the whole project's includes, once
#include <charconv>
#include <cctype>
#include <cmath>
#include <cstdio>
#include <iostream>
#include <stdexcept>
#include <string>
#include <string_view>
#include <type_traits>
#include <utility>
#include <variant>
#include <vector>
Then the value type itself.
// ASSEMBLE json 2
class Value;
using Null = std::monostate;
using Bool = bool;
using Number = double;
using String = std::string;
using Array = std::vector<Value>;
// An object is an ORDERED list of pairs, not a std::map. Two reasons, both in
// the lesson: std::map is not required to accept an incomplete value type, and
// a vector round-trips the document with its keys in the order they appeared.
using Object = std::vector<std::pair<std::string, Value>>;
class Value {
public:
using Storage = std::variant<Null, Bool, Number, String, Array, Object>;
Value() = default; // null
Value(Bool b) : v_(b) {}
Value(Number n) : v_(n) {}
Value(int n) : v_(static_cast<Number>(n)) {}
Value(const char* s) : v_(String(s)) {}
Value(String s) : v_(std::move(s)) {}
Value(Array a) : v_(std::move(a)) {}
Value(Object o) : v_(std::move(o)) {}
bool is_null() const { return std::holds_alternative<Null>(v_); }
bool is_bool() const { return std::holds_alternative<Bool>(v_); }
bool is_number() const { return std::holds_alternative<Number>(v_); }
bool is_string() const { return std::holds_alternative<String>(v_); }
bool is_array() const { return std::holds_alternative<Array>(v_); }
bool is_object() const { return std::holds_alternative<Object>(v_); }
// Typed access. Throws std::bad_variant_access if you ask for the wrong one,
// which is the right failure: it is a bug in your code, not bad input.
Bool as_bool() const { return std::get<Bool>(v_); }
Number as_number() const { return std::get<Number>(v_); }
const String& as_string() const { return std::get<String>(v_); }
const Array& as_array() const { return std::get<Array>(v_); }
const Object& as_object() const { return std::get<Object>(v_); }
// Object lookup. Returns nullptr for a missing key, so callers can branch
// without exceptions on the ordinary "field is absent" path.
const Value* find(std::string_view key) const {
if (!is_object()) return nullptr;
for (const auto& [k, v] : as_object())
if (k == key) return &v;
return nullptr;
}
const Storage& storage() const { return v_; }
private:
Storage v_;
};
Three decisions in there are worth defending.
Object is a vector of pairs, not a std::map. Two reasons. The standard only guarantees that
std::vector, std::list and std::forward_list accept an incomplete element type, and at the
point Object is declared, Value is still incomplete — it is the class currently being defined.
std::map<std::string, Value> there is undefined behaviour, even though libstdc++ happens to compile
it. The second reason is better: a vector keeps the keys in the order the document had them, so
a round-trip does not silently reorder someone's config file. The cost is O(n) lookup, which for
objects with a handful of keys is faster than a map anyway.
as_number() throws, find() returns nullptr. These are different failures.
A missing key is ordinary — the document just did not have that field — so find reports it as a
value you can test. Asking a string for as_number() is a bug in your code, so it throws
std::bad_variant_access. Lesson 15.1's rule, applied: exceptions for programmer errors, return
values for expected outcomes.
Value() = default is null. std::monostate is the first alternative, so a default-constructed
Value is JSON null, which is the behaviour you want and costs nothing to arrange.
The parser
Recursive descent: one function per grammar rule, each calling the others. JSON's grammar is small enough that the parser reads like the specification.
// ASSEMBLE json 3
struct ParseError : std::runtime_error {
std::size_t line, column;
ParseError(std::string msg, std::size_t l, std::size_t c)
: std::runtime_error(std::move(msg)), line(l), column(c) {}
};
class Parser {
public:
explicit Parser(std::string_view text) : s_(text) {}
Value parse() {
skip_ws();
Value v = parse_value(0);
skip_ws();
if (pos_ != s_.size()) fail("trailing characters after value");
return v;
}
private:
static constexpr int kMaxDepth = 200;
std::string_view s_;
std::size_t pos_ = 0;
std::size_t line_ = 1;
std::size_t col_ = 1;
[[noreturn]] void fail(const std::string& msg) const {
throw ParseError(msg, line_, col_);
}
bool eof() const { return pos_ >= s_.size(); }
char peek() const { return eof() ? '\0' : s_[pos_]; }
char advance() {
char c = s_[pos_++];
if (c == '\n') { ++line_; col_ = 1; } else { ++col_; }
return c;
}
void expect(char c) {
if (peek() != c) fail(std::string("expected '") + c + "'");
advance();
}
void skip_ws() {
while (!eof()) {
char c = peek();
if (c == ' ' || c == '\t' || c == '\n' || c == '\r') advance();
else break;
}
}
Value parse_value(int depth) {
if (depth > kMaxDepth) fail("nesting too deep");
if (eof()) fail("unexpected end of input");
switch (peek()) {
case '{': return parse_object(depth);
case '[': return parse_array(depth);
case '"': return Value(parse_string());
case 't': return parse_literal("true", Value(true));
case 'f': return parse_literal("false", Value(false));
case 'n': return parse_literal("null", Value());
default: return parse_number();
}
}
Value parse_literal(std::string_view word, Value result) {
if (s_.substr(pos_, word.size()) != word)
fail("invalid literal");
for (std::size_t i = 0; i < word.size(); ++i) advance();
return result;
}
Value parse_object(int depth) {
expect('{');
Object obj;
skip_ws();
if (peek() == '}') { advance(); return Value(std::move(obj)); }
for (;;) {
skip_ws();
if (peek() != '"') fail("object key must be a string");
std::string key = parse_string();
skip_ws();
expect(':');
skip_ws();
obj.emplace_back(std::move(key), parse_value(depth + 1));
skip_ws();
if (peek() == ',') { advance(); continue; }
expect('}');
return Value(std::move(obj));
}
}
Value parse_array(int depth) {
expect('[');
Array arr;
skip_ws();
if (peek() == ']') { advance(); return Value(std::move(arr)); }
for (;;) {
skip_ws();
arr.push_back(parse_value(depth + 1));
skip_ws();
if (peek() == ',') { advance(); continue; }
expect(']');
return Value(std::move(arr));
}
}
std::string parse_string() {
expect('"');
std::string out;
for (;;) {
if (eof()) fail("unterminated string");
char c = advance();
if (c == '"') return out;
if (c != '\\') {
if (static_cast<unsigned char>(c) < 0x20) fail("control character in string");
out += c;
continue;
}
if (eof()) fail("unterminated escape");
switch (advance()) {
case '"': out += '"'; break;
case '\\': out += '\\'; break;
case '/': out += '/'; break;
case 'b': out += '\b'; break;
case 'f': out += '\f'; break;
case 'n': out += '\n'; break;
case 'r': out += '\r'; break;
case 't': out += '\t'; break;
case 'u': out += parse_unicode_escape(); break;
default: fail("invalid escape");
}
}
}
// \uXXXX, encoded as UTF-8. Khmer lives in the U+1780 block, which is three
// bytes, so this path is not optional for this site's data.
std::string parse_unicode_escape() {
unsigned cp = 0;
for (int i = 0; i < 4; ++i) {
if (eof()) fail("truncated \\u escape");
char c = advance();
unsigned d;
if (c >= '0' && c <= '9') d = static_cast<unsigned>(c - '0');
else if (c >= 'a' && c <= 'f') d = static_cast<unsigned>(c - 'a' + 10);
else if (c >= 'A' && c <= 'F') d = static_cast<unsigned>(c - 'A' + 10);
else fail("invalid hex digit in \\u escape");
cp = cp * 16 + d;
}
std::string out;
if (cp < 0x80) {
out += static_cast<char>(cp);
} else if (cp < 0x800) {
out += static_cast<char>(0xC0 | (cp >> 6));
out += static_cast<char>(0x80 | (cp & 0x3F));
} else {
out += static_cast<char>(0xE0 | (cp >> 12));
out += static_cast<char>(0x80 | ((cp >> 6) & 0x3F));
out += static_cast<char>(0x80 | (cp & 0x3F));
}
return out;
}
Value parse_number() {
std::size_t start = pos_;
if (peek() == '-') advance();
if (!std::isdigit(static_cast<unsigned char>(peek()))) fail("invalid number");
// JSON forbids leading zeros: 0 and 0.5 are fine, 01 is not.
if (peek() == '0') {
advance();
if (std::isdigit(static_cast<unsigned char>(peek()))) fail("leading zero in number");
} else {
while (std::isdigit(static_cast<unsigned char>(peek()))) advance();
}
if (peek() == '.') {
advance();
if (!std::isdigit(static_cast<unsigned char>(peek()))) fail("digit expected after '.'");
while (std::isdigit(static_cast<unsigned char>(peek()))) advance();
}
if (peek() == 'e' || peek() == 'E') {
advance();
if (peek() == '+' || peek() == '-') advance();
if (!std::isdigit(static_cast<unsigned char>(peek()))) fail("digit expected in exponent");
while (std::isdigit(static_cast<unsigned char>(peek()))) advance();
}
double out = 0;
std::string_view text = s_.substr(start, pos_ - start);
auto [ptr, ec] = std::from_chars(text.data(), text.data() + text.size(), out);
if (ec != std::errc{} || ptr != text.data() + text.size()) fail("number out of range");
return Value(out);
}
};
Value parse(std::string_view text) { return Parser(text).parse(); }
The details that matter
advance() tracks line and column. Every character goes through it, so fail() always knows
where it is. That is the entire cost of good error messages — one branch per character.
kMaxDepth is not optional. parse_value calls parse_array calls parse_value. A document
of 100,000 open brackets is 100,000 stack frames. Section "Hostile input" below shows what happens
without the guard.
parse_string builds the string once. It appends to out and returns it; the compiler moves it
out (lesson 9.4). No intermediate std::string per character.
The \u escape emits UTF-8 by hand. Khmer sits in U+1780–U+17FF, which is the three-byte range,
so the 0xE0 branch is the one this site's data exercises. An escape is the only way a Khmer string
can travel through a strictly-ASCII JSON file, so this path is not optional here.
from_chars for numbers, not std::stod. stod is locale-dependent — in a locale where the
decimal separator is a comma it parses 1.5 as 1. from_chars is locale-independent by
specification, does not allocate, and does not throw; it reports failure in its return value.
The serializer
Writing is easier than reading, and std::visit over the variant makes it nearly mechanical.
// ASSEMBLE json 4
void write_escaped(std::string& out, std::string_view s) {
out += '"';
for (char c : s) {
switch (c) {
case '"': out += "\\\""; break;
case '\\': out += "\\\\"; break;
case '\b': out += "\\b"; break;
case '\f': out += "\\f"; break;
case '\n': out += "\\n"; break;
case '\r': out += "\\r"; break;
case '\t': out += "\\t"; break;
default:
if (static_cast<unsigned char>(c) < 0x20) {
char buf[7];
std::snprintf(buf, sizeof buf, "\\u%04x", static_cast<unsigned char>(c));
out += buf;
} else {
// UTF-8 bytes pass through unchanged; \u escaping them is
// legal but makes Khmer unreadable in the output file.
out += c;
}
}
}
out += '"';
}
void write_number(std::string& out, double n) {
if (n == static_cast<double>(static_cast<long long>(n)) &&
std::abs(n) < 1e15) {
out += std::to_string(static_cast<long long>(n));
return;
}
char buf[32];
auto [ptr, ec] = std::to_chars(buf, buf + sizeof buf, n);
out.append(buf, (ec == std::errc{}) ? ptr : buf);
}
void write_value(std::string& out, const Value& v, int indent, int depth) {
const bool pretty = indent > 0;
const std::string pad = pretty ? std::string(static_cast<std::size_t>(indent * (depth + 1)), ' ') : "";
const std::string padEnd = pretty ? std::string(static_cast<std::size_t>(indent * depth), ' ') : "";
const char* nl = pretty ? "\n" : "";
const char* sep = pretty ? ": " : ":";
std::visit([&](const auto& x) {
using T = std::decay_t<decltype(x)>;
if constexpr (std::is_same_v<T, Null>) {
out += "null";
} else if constexpr (std::is_same_v<T, Bool>) {
out += x ? "true" : "false";
} else if constexpr (std::is_same_v<T, Number>) {
write_number(out, x);
} else if constexpr (std::is_same_v<T, String>) {
write_escaped(out, x);
} else if constexpr (std::is_same_v<T, Array>) {
if (x.empty()) { out += "[]"; return; }
out += '['; out += nl;
for (std::size_t i = 0; i < x.size(); ++i) {
out += pad;
write_value(out, x[i], indent, depth + 1);
if (i + 1 < x.size()) out += ',';
out += nl;
}
out += padEnd; out += ']';
} else {
if (x.empty()) { out += "{}"; return; }
out += '{'; out += nl;
for (std::size_t i = 0; i < x.size(); ++i) {
out += pad;
write_escaped(out, x[i].first);
out += sep;
write_value(out, x[i].second, indent, depth + 1);
if (i + 1 < x.size()) out += ',';
out += nl;
}
out += padEnd; out += '}';
}
}, v.storage());
}
std::string dump(const Value& v, int indent = 0) {
std::string out;
write_value(out, v, indent, 0);
return out;
}
The if constexpr chain inside the lambda is the generic-visitor pattern from lesson 13.10: one
lambda, auto parameter, and the compiler generates a branch per alternative — with a compile error
if you forget one, because std::visit requires the visitor to be callable for every type.
write_number exists because std::to_string(1.0) gives "1.000000" and to_chars gives "1".
JSON has one number type, so an integer-valued double must print without a decimal point or every
round-trip grows.
Running it
// ASSEMBLE json 5
const char* kDoc = R"JSON({
"site": "Evil Genius Society",
"languages": ["en", "km"],
"free": true,
"learners": 0,
"motto": {"km": "\u1799\u17be\u1784", "en": "we"},
"version": 1.5,
"sponsor": null
})JSON";
int main() {
Value doc = parse(kDoc);
std::cout << "1. round-trip, pretty:\n" << dump(doc, 2) << "\n\n";
std::cout << "2. compact:\n" << dump(doc) << "\n\n";
std::cout << "3. typed access:\n";
std::cout << " site " << doc.find("site")->as_string() << "\n";
std::cout << " languages " << doc.find("languages")->as_array().size() << " entries\n";
std::cout << " free " << std::boolalpha << doc.find("free")->as_bool() << "\n";
std::cout << " khmer " << doc.find("motto")->find("km")->as_string() << "\n";
std::cout << " missing " << (doc.find("nope") == nullptr ? "nullptr" : "?") << "\n";
std::cout << "\n4. idempotence: dump(parse(dump(doc))) == dump(doc)? "
<< (dump(parse(dump(doc))) == dump(doc) ? "yes" : "no") << "\n";
std::cout << "\n5. errors:\n";
const std::vector<std::string> bad = {
R"({"a": 1,})",
R"({"a" 1})",
R"([1, 2)",
R"({"a": tru})",
R"( {"a": 01})",
"\"unterminated",
R"({"a": "bad \q escape"})",
};
for (const std::string& b : bad) {
try {
parse(b);
std::cout << " ACCEPTED (bug): " << b << "\n";
} catch (const ParseError& e) {
std::cout << " line " << e.line << " col " << e.column
<< ": " << e.what() << "\n";
}
}
std::cout << "\n6. building a document in code:\n";
Object out;
out.emplace_back("module", Value(16));
out.emplace_back("lesson", Value("16.9"));
out.emplace_back("tags", Value(Array{Value("variant"), Value("recursion"), Value("raii")}));
std::cout << dump(Value(std::move(out)), 2) << "\n";
return 0;
}
1. round-trip, pretty:
{
"site": "Evil Genius Society",
"languages": [
"en",
"km"
],
"free": true,
"learners": 0,
"motto": {
"km": "យើង",
"en": "we"
},
"version": 1.5,
"sponsor": null
}
2. compact:
{"site":"Evil Genius Society","languages":["en","km"],"free":true,"learners":0,"motto":{"km":"យើង","en":"we"},"version":1.5,"sponsor":null}
3. typed access:
site Evil Genius Society
languages 2 entries
free true
khmer យើង
missing nullptr
4. idempotence: dump(parse(dump(doc))) == dump(doc)? yes
5. errors:
line 1 col 9: object key must be a string
line 1 col 6: expected ':'
line 1 col 6: expected ']'
line 1 col 7: invalid literal
line 1 col 10: leading zero in number
line 1 col 14: unterminated string
line 1 col 14: invalid escape
6. building a document in code:
{
"module": 16,
"lesson": "16.9",
"tags": [
"variant",
"recursion",
"raii"
]
}
Every one of the seven bad documents was rejected with a position. Two are worth calling out.
{"a": 01} is invalid JSON — the grammar forbids leading zeros — and the first version of this
parser accepted it. The bad-input list found the bug, not a review. The fix is the if (peek() == '0') branch in parse_number: after a zero, the integer part is over.
{"a": 1,} reports "object key must be a string" rather than "trailing comma". The parser consumed
the comma, looped, and found } where a key belongs. It is accurate but it is not the message a
human wants. Improving it is the first exercise below.
Hostile input
Requirement 3, tested directly.
std::string deep(100000, '[');
deep += std::string(100000, ']');
Value v = parse(deep); // 100,000 recursive calls
With the depth guard removed:
$ g++ -std=c++20 -Wall -Wextra -O2 -o ng noguard.cpp
$ ./ng
parsing 100000 levels with no depth guard...
Segmentation fault
With it:
depth 150: parsed
depth 300: nesting too deep
depth 100000: nesting too deep
A crash on untrusted input is a denial of service. Any recursive parser reading data you did not write needs a depth limit, and 200 is generous — real JSON is rarely nested past 20.
Verifying it
Clean under sanitizers, which is the requirement:
$ g++ -std=c++20 -Wall -Wextra -g -fsanitize=address,undefined json.cpp -o json_san
$ ./json_san > san.out
$ ./json | diff - san.out && echo identical
identical
No ASan report, no UBSan report, exit code 0, and byte-identical output to the optimised build.
Note what carried this: no new and no delete appear anywhere in the project. std::string,
std::vector and std::variant own everything, and the recursion unwinds through destructors. When
fail() throws from twelve levels deep, every partially-built Array and Object on the way out is
destroyed by RAII (lesson 15.4) with nothing to write.
Break it deliberately
| Change | What happens |
|---|---|
| Remove the kMaxDepth check | Segfault on deeply nested input |
| Use std::stod instead of from_chars | Passes here; wrong under a comma-decimal locale |
| std::map<std::string, Value> for Object | Compiles on libstdc++; UB, and reorders keys |
| Drop the < 0x20 check in parse_string | Accepts raw newlines inside strings |
| Return out by std::move from parse_string | Warning: -Wpessimizing-move |
| Let find() throw instead of returning nullptr | Every optional field now needs a try |
What you have built
A real parser. Not a toy one — it has correct escape handling, locale-independent numbers, positional errors, a depth guard, and order-preserving objects, which is more than several widely-used implementations manage.
What it does not have: surrogate pairs (😀 for emoji outside the BMP), streaming input,
and the aggressive optimisation a production parser uses. Those are the next three things to add, and
the first is the exercise.
Try it
Three additions, in order of difficulty.
- Make
{"a": 1,}saytrailing comma in objectinstead ofobject key must be a string. - Add a
Value::type_name()returning"null","bool","number","string","array"or"object", and use it to produce a real error when typed access fails. - Handle surrogate pairs, so
"😀"decodes to the four UTF-8 bytes of U+1F600.
Checkpoint
- A JSON value is a sum type;
std::variantis the direct expression of that. Objectas a vector of pairs:std::mapis not guaranteed to accept an incomplete type.- It also preserves key order, so a round-trip does not reorder someone's file.
- Recursive descent gives one function per grammar rule; the parser reads like the spec.
advance()tracking line and column costs one branch and buys every error message.- Measured: without a depth guard, 100,000 nested brackets segfault. With it, a clean error.
- Any recursive parser on untrusted input needs a depth limit.
from_charsoverstd::stod: locale-independent, no allocation, no throw.\uescapes decode to UTF-8 by hand; Khmer is the three-byte branch, emoji the four-byte one.- Surrogate pairs:
0x10000 + ((hi - 0xD800) << 10) + (lo - 0xDC00). std::visitwith a generic lambda andif constexprwrites one branch per alternative — and fails to compile if you miss one.- No
new, nodeletein 250 lines. RAII unwinds a partially-built tree on throw for free. - Verified: ASan and UBSan clean, output byte-identical to the optimised build.
- Exceptions for programmer errors (
as_number()on a string), return values for expected ones (find()on a missing key). - The bad-input list found a real bug —
01was accepted. Write the rejection tests first.
Questions about this lesson
Log in to ask a question.