3 UTF, Unicode, ASCII, rune \- character set and format
5 The Plan 9 character set and representation are
6 based on the Unicode Standard and on the ISO multibyte
8 encoding (Universal Character
9 Set Transformation Format, 8 bits wide).
10 The Unicode Standard represents its characters in 21
14 values in an 8-bit byte stream.
15 Throughout this manual,
22 is a 32-bit quantity representing a Unicode character.
23 Internally, programs may store characters as runes.
24 However, any external manifestation of textual information,
25 in files or at the interface between programs, uses a
26 machine-independent, byte-stream encoding called
30 is designed so the 7-bit
32 set (values hexadecimal 00 to 7F),
33 appear only as themselves
35 Runes with values above 7F appear as sequences of two or more
36 bytes with values only from 80 to FF.
40 encoding of the Unicode Standard is backward compatible with
43 programs presented only with
46 even if not written to deal with
49 programs that deal with uninterpreted byte streams.
50 However, programs that perform semantic processing on
53 characters must convert from
56 in order to work properly with non-\c
62 Letting numbers be binary,
63 a rune x is converted to a multibyte
68 001. x in [00000000.00000000.0bbbbbbb] → 0bbbbbbb
70 010. x in [00000000.00000bbb.bbbbbbbb] → 110bbbbb, 10bbbbbb
72 011. x in [00000000.bbbbbbbb.bbbbbbbb] → 1110bbbb, 10bbbbbb, 10bbbbbb
74 100. x in [000bbbbb.bbbbbbbb.bbbbbbbb] → 11110bbb, 10bbbbbb, 10bbbbbb, 10bbbbbb
77 Conversion 001 provides a one-byte sequence that spans the
79 character set in a compatible way.
80 Conversions 010, 011 and 100 represent higher-valued characters
81 as sequences of two, three or four bytes with the high bit set.
82 Plan 9 does not support the 5 and 6 byte sequences proposed by X-Open.
83 When there are multiple ways to encode a value, for example rune 0,
84 the shortest encoding is used.
86 In the inverse mapping,
87 any sequence except those described above
88 is incorrect and is converted to rune hexadecimal FFFD.
93 table of characters and descriptions, suitable for
100 .IR "The Unicode Standard" .