# How can I avoid producing mojibake?

## Understand bytes and text

JavaScript strings are Unicode text. `Uint8Array`, `Buffer`, network bodies,
and file contents may be bytes. Keep that distinction explicit and decode only
at the boundary where the byte encoding is known.

## Assume UTF-8

Assume text is UTF-8 unless there is a specific reason to believe otherwise.
Most mojibake comes from decoding valid UTF-8 using a legacy single-byte
encoding.

```ts
const text = new TextDecoder("utf-8", { fatal: false }).decode(bytes);
const bytesAgain = new TextEncoder().encode(text);
```

Windows APIs may require UTF-16, and a specific external protocol may declare a
different encoding. Those are concrete reasons to choose something else;
guessing from the language is not.

## Avoid lossy spreadsheet exports

Legacy CSV export tools, particularly locale-dependent Excel exports, may pick
a 256-character encoding based on the operating system and default language.
Prefer a UTF-8 CSV export or preserve the `.xlsx` file. A tab-separated UTF-16
export is less universal but at least preserves Unicode.

## Do not trust probabilistic encoding detection

Encoding detection on raw bytes is unreliable for modern multilingual text and
emoji. Many older detectors were built when “encoding detection” largely meant
“language detection” and can misidentify valid UTF-8 as a regional Windows
encoding.

If no encoding is supplied, UTF-8 is normally the safest first attempt. ftfy's
`guess_bytes` exists for unavoidable cases but intentionally uses only strong
signals and makes no promise of universal accuracy.

## “Extended ASCII” is not an encoding

ASCII contains 128 character codes. Many incompatible encodings assign
different meanings to bytes 128 through 255: Windows-1252, Windows-1250,
Latin-1, cp437, MacRoman, and others.

Name the actual encoding. “Extended ASCII” usually means “the 256-character
encoding I learned first,” which depends on a person's country and operating
system and cannot identify how bytes should be decoded.