How to encode Japanese and Unicode text with Base64

What Base64 represents

Base64 is an encoding that represents a byte sequence with a limited ASCII alphabet. It does not operate directly on text, so encoding Japanese text requires an explicit character encoding at the intermediate step.

text -> bytes in a character encoding -> Base64 text
Base64 text -> restored bytes -> text in the same character encoding

Base64 is not encryption. Anyone can recover the bytes without a key, so Base64-encoding personal data, passwords, or API keys does not protect them.

Why Japanese text and emoji become mojibake

ASCII characters occupy the same single-byte values in many encodings, which can hide a faulty implementation. Japanese characters and emoji require multiple bytes. If the encoder uses UTF-8 while the decoder interprets the restored bytes with another encoding, text can become mojibake such as 日本語.

The browser's older btoa() interface treats each input character as one byte, so passing a Unicode string directly, as in btoa("日本語"), can throw an exception. Convert the text to UTF-8 bytes first.

A safe UTF-8 round trip

Encoding 日本語 as UTF-8 produces the expected Base64 value 5pel5pys6Kqe. For the emoji 😀, the value is 8J+YgA==.

Browser code can use TextEncoder and TextDecoder to make the text-to-UTF-8-byte boundary explicit.

function bytesToBase64(bytes) {
  let binary = "";
  for (const byte of bytes) binary += String.fromCharCode(byte);
  return btoa(binary);
}

function base64ToBytes(base64) {
  const binary = atob(base64);
  return Uint8Array.from(binary, (character) => character.charCodeAt(0));
}

const encoded = bytesToBase64(new TextEncoder().encode("日本語"));
// expected: "5pel5pys6Kqe"

const decoded = new TextDecoder("utf-8", { fatal: true }).decode(base64ToBytes(encoded));
// expected: "日本語"

The fatal: true option detects invalid UTF-8 as an exception instead of hiding it behind replacement characters. It does not prove that unknown input was intended to be UTF-8. The sender and receiver still need to agree on a character encoding.

Line breaks, padding, and base64url

• Whether line breaks are accepted depends on the consuming specification. RFC 4648 does not direct an encoder to add them unless another specification requires it.
• Trailing = characters pad the final input block. Some protocols allow omission, but follow the receiving format's rules.
• Base64url replaces the usual Base64 + and / with URL-friendly - and _. It is used for data such as JWT parts. Do not treat the two alphabets as the same encoded string.

Before guessing that line breaks should be removed, = added, or alphabet characters replaced, identify the Base64 variant required by the input contract.

A mojibake troubleshooting checklist

1. Identify the original text and its intended character encoding.
2. Encode it as UTF-8 in the Base64 app.
3. Decode the output immediately and compare it exactly with the original.
4. Round-trip small samples containing ASCII, Japanese text, emoji, and a line break.
5. If only another system fails, compare the character encoding, Base64 versus base64url, padding, and line-break rules.
6. Replace sensitive test data with a harmless sample such as 日本語😀.

The App Museum Base64 app encodes and decodes UTF-8 text entirely in the browser. Use a round trip with a concrete input to confirm the byte-transfer assumptions.

References

• RFC 4648: Base-N Encodings
• Unicode Standard 17.0: Conformance and Encoding Forms
• RFC 3629: UTF-8, a transformation format of ISO 10646