Shift JIS

The most popular extension is Windows code page 932 (a CCSID also used for IBM's extension to Shift JIS), which is registered with the IANA as "Windows-31J",[1] separately from Shift JIS. This was popularized by Microsoft, although Microsoft itself does not recognize the Windows-31J name and instead calls that variation "shift_jis".[8][9] IBM's code page 943 includes the same double-byte codes as Microsoft's code page 932, while IBM's code page 932 includes fewer extensions (excluding those which Microsoft incorporates from NEC), and retains the character order from the 1978 edition of JIS X 0208, rather than implementing the character variant swaps from the 1983 standard.[10]

Windows-31J assigns 0x5C to U+005C REVERSE SOLIDUS (the backslash), and 0x7E to U+007E TILDE, following US-ASCII.[11] However, most localised fonts on Windows display U+005C as a Yen sign for JIS X 0201 compatibility.[12][13] It includes several extensions, namely "NEC special characters (Row 13), NEC selection of IBM extensions (Rows 89 to 92), and IBM extensions (Rows 115 to 119)",[1] in addition to setting some encoding space aside for end user definition.[14]

Windows codepage 932 is the version used in the W3C/WHATWG encoding standard used by HTML5, which includes the "formerly proprietary extensions from IBM and NEC" from Windows-31J in its table for JIS X 0208,[15] and also treats the label "shift_jis" interchangeably with "windows-31j" with the intent of being "compatible with deployed content".[16]

The version of Shift-JIS originating from the classic Mac OS (known as x-mac-japanese, Code page 10001[8] or MacJapanese) assigned the tilde to 0x7E (following US-ASCII, not JIS X 0201 which assigns the overline here), but the Yen sign to 0x5C (as in JIS X 0201 and standard Shift JIS). It also extended JIS X 0201 by assigning the backslash to 0x80 (corresponding to 0x5C in US-ASCII), the non-breaking space to 0xA0, the copyright sign to 0xFD, the trademark symbol to 0xFE and the half-width horizontal ellipsis to 0xFF. It also added extended double byte characters; including 53 vertical presentation forms in the Shift_JIS range 0xEB41–0xED96, at 84 JIS rows down from their canonical forms, and 260 special characters in the Shift_JIS range 0x8540–0x886D.[17] This variant was introduced in KanjiTalk version 7.[18]

However, certain Mac OS typefaces used other variants. Sai Mincho and Chu Gothic use a "PostScript" variant of MacJapanese, which included additional vertical presentation forms and a different set of extended special characters, based on the NEC special characters, some of which were only available in the printer versions of the fonts.[17] Older versions of Maru Gothic and Hon Mincho from System 7.1 encoded vertical presentation forms at 10 (not 84) JIS rows down from their canonical forms, and did not include the special character extensions, this was subsequently changed.[17][19] The typical variant used with KanjiTalk version 6 placed the vertical presentation forms 10 rows down, and also used the NEC extension layout for row 13.[20]

The newer JIS X 0213 standard defines an extended variant of Shift_JIS referred to as Shift_JISx0213 (in a previous version of the standard) or Shift_JIS-2004. It is a superset of standard Shift JIS.[21]

In order to represent the allocated rows on both planes of JIS X 0213, Shift_JIS-2004 uses the following method of mapping codepoints.[22]

${\displaystyle s_{1}={\begin{cases}\left\lfloor {\frac {k+257}{2}}\right\rfloor &{\mbox{if }}m=1{\mbox{ and }}1\leq k\leq 62\\\left\lfloor {\frac {k+385}{2}}\right\rfloor &{\mbox{if }}m=1{\mbox{ and }}63\leq k\leq 94\\\left\lfloor {\frac {k+479}{2}}\right\rfloor -\left\lfloor {\frac {k}{8}}\right\rfloor \times 3&{\mbox{if }}m=2{\mbox{ and }}k=1,3,4,5,8,12,13,14,15\\\left\lfloor {\frac {k+411}{2}}\right\rfloor &{\mbox{if }}m=2{\mbox{ and }}78\leq k\leq 94\end{cases}}}$

${\displaystyle s_{2}={\begin{cases}t+63&{\mbox{if }}k{\mbox{ is odd and }}1\leq t\leq 63\\t+64&{\mbox{if }}k{\mbox{ is odd and }}64\leq t\leq 94\\t+158&{\mbox{if }}k{\mbox{ is even }}\end{cases}}}$

In the above, ${\displaystyle s_{1}s_{2}}$ is a two-byte Shift_JIS-2004 sequence, ${\displaystyle m}$ is the plane (面, men, surface) number (1 or 2), ${\displaystyle k}$ is the row (区, ku, ward) number (1-94) and ${\displaystyle t}$ is the cell (点, ten, point) number (1-94). The ku and ten numbers are equivalent to ${\displaystyle j_{1}-32}$ and ${\displaystyle j_{2}-32}$ respectively, where ${\displaystyle j_{1}j_{2}}$ is a two-byte JIS sequence referencing a given plane.

The same set of characters can represented by EUC-JIS-2004, the EUC-JP based counterpart.

Some of the additions collide with popular Shift JIS extensions, including Windows codepage 932 which is used in web standards (see above). For example, compare plane 1 row 89 in JIS X 0213 (beginning 硃, 硎, 硏...)[23] to row 89 in the JIS X 0208 variant defined in web standards (beginning 纊, 褜, 鍈...).[24] In addition, some of the characters map to Unicode characters beyond the BMP.

The space with lead bytes 0xF5 to 0xF9 (beyond the region used for JIS X 0208) is used by Japanese mobile phone operators for pictographs for use in E-mail.[25] KDDI goes further and defines hundreds more in the space with lead bytes 0xF3 and 0xF4.[26]

Beyond even this, there have been numerous minor variations made on Shift JIS, with individual characters here and there altered. Most of these extensions and variants have no IANA registration, so there is much scope for confusion, if the extensions are used.

A variant is the one that must be used if wanting to encode Shift JIS in source code strings of C and similar programming languages. This variant doubles the byte 0x5C if it appears as second byte of a two-byte character, but not if it appears as a single "¥" (ASCII: "\") character, because 0x5C is the beginning of an escape sequence. The best way of handling this is a special editor which encodes Shift JIS this way.

The chart below gives the detailed meaning of each byte in a stream encoded in standard Shift JIS (conforming to JIS X 0208:1997).

Some of the bytes which are not used for single-byte codes or initial bytes in JIS X 0208:1997 are used by certain extensions, resulting in the layout detailed in the chart below.