Knowledge baseExam Center
ENEnglish
Choose language13

CKTIS / C++

Strings and text: code units, code points, and grapheme clusters

A C++ string stores elements, not ready-made user-visible characters. Learn the distinction between UTF-8 code units, Unicode code points, and grapheme clusters, and understand the roles of std::u8string and a u8 string literal.

Version: C++23; repository compiler flag, production runtime not attested

Syntax

std::string text = "abc";
std::u8string utf8 = u8"e\u0301";

std::string owns a contiguous sequence of char elements; its type alone does not guarantee UTF-8. std::u8string stores char8_t elements intended for UTF-8 code units. Neither container automatically validates Unicode text. size() counts stored elements, excluding the trailing zero terminator.

Distinguish three levels: UTF-8 code units are 8-bit encoding units; Unicode code points are numbered Unicode positions, such as U+0065; grapheme clusters approximate user-perceived characters. One Unicode scalar value occupies one to four UTF-8 code units. One cluster can contain several code points: e followed by U+0301, combining acute accent, forms one extended grapheme cluster but two code points and three UTF-8 code units.

In C++23, a u8 string literal, such as u8"e\u0301", has an array type with const char8_t elements and UTF-8 encoding. The literal includes a trailing zero element; a std::u8string constructed from this literal does not count it in size(). char8_t is distinct from char: this literal cannot directly initialize a std::string.

Indexing, iteration, and substr() operate on elements, not grapheme clusters. Cutting inside a multi-unit UTF-8 sequence can produce invalid encoding, while cutting between code points can split a cluster. Operations on user-perceived characters require Unicode decoding and segmentation; size() alone performs neither. The linked document is an evolving working draft, not a frozen C++23 publication; the example uses features available in C++23.

Examples

#include <iostream>
#include <string>

int main() {
    const std::u8string text = u8"e\u0301";
    std::cout << "UTF-8 code units: " << text.size() << '\n';
    std::cout << "Values:";
    for (char8_t unit : text) {
        std::cout << ' ' << static_cast<unsigned int>(unit);
    }
    std::cout << '\n';
}

Result

UTF-8 code units: 3
Values: 101 204 129

The displayed code and declared result were verified in an isolated environment using the method recorded in the verification evidence for this exact version.

Sources