CKTIS / Java
Encoding Java strings as UTF-8 and UTF-16BE
Use `String.getBytes(Charset)` to encode text as bytes. See why `Euro €` takes 8 bytes in UTF-8 and 12 in UTF-16BE, and why UTF-16BE does not always use two bytes per code point.
Version: Java SE 21
Syntax
import java.nio.charset.StandardCharsets;
byte[] utf8 = text.getBytes(StandardCharsets.UTF_8);
byte[] utf16be = text.getBytes(StandardCharsets.UTF_16BE);StandardCharsets.UTF_8/UTF_16BE refers to two separate Charset constants, StandardCharsets.UTF_8 and StandardCharsets.UTF_16BE, in java.nio.charset. Both are available on every Java SE 21 implementation. The String.getBytes overload that takes a Charset encodes the text into a new byte[]. At a file or protocol boundary, pass the required encoding explicitly. No-argument getBytes() uses the JVM default charset, which is UTF-8 in Java SE 21 unless changed in an implementation-specific way.
UTF-8 uses 1 to 4 bytes per Unicode scalar value, that is, per Unicode code point outside the surrogate range U+D800–U+DFFF. UTF-16BE writes each 16-bit code unit as two bytes, most significant byte first. It does not add a byte-order mark (BOM). By contrast, Java's StandardCharsets.UTF_16 writes a BOM and uses big-endian order when encoding.
Euro € contains five ASCII characters, including the space, plus €. That gives 5 × 1 + 3 = 8 bytes in UTF-8 and 6 × 2 = 12 in UTF-16BE. A supplementary code point such as 😀 (U+1F600) requires a surrogate pair: two code units and 4 bytes in UTF-16BE. It also uses 4 bytes in UTF-8. String.length() counts UTF-16 code units, while bytes.length counts bytes.
The getBytes(String) overload takes an encoding name and throws the checked UnsupportedEncodingException if that encoding is unsupported. When text cannot be encoded, the behavior of getBytes(String) and getBytes() is unspecified. getBytes(Charset) substitutes the charset's default replacement bytes for malformed or unmappable input sequences. To report errors instead of replacing input, use a CharsetEncoder with CodingErrorAction.REPORT for both error types.
Examples
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.util.HexFormat;
public class EncodingDemo {
public static void main(String[] args) {
String euro = "Euro \u20AC";
String emoji = "\uD83D\uDE00";
show(euro, StandardCharsets.UTF_8);
show(euro, StandardCharsets.UTF_16BE);
show(emoji, StandardCharsets.UTF_8);
show(emoji, StandardCharsets.UTF_16BE);
}
private static void show(String text, Charset charset) {
byte[] bytes = text.getBytes(charset);
String hex = HexFormat.ofDelimiter(" ").formatHex(bytes);
System.out.println(charset.name() + " " + bytes.length + ": " + hex);
}
}Result
UTF-8 8: 45 75 72 6f 20 e2 82 ac UTF-16BE 12: 00 45 00 75 00 72 00 6f 00 20 20 ac UTF-8 4: f0 9f 98 80 UTF-16BE 4: d8 3d de 00
The first two lines are for `Euro €`, including the space; the last two are for `😀`. The program prints the array length and its bytes in hexadecimal, not the input characters.
Related lessons
- Java · Lesson 50 · Advanced · Version: 2