Strings look the same but are not equal (JavaScript and Python)

Your test says the two strings are different. Your eyes say they are the same. The computer is counting characters you cannot see, or two different ways of writing the same letter.

Prove they are different

// JavaScript
JSON.stringify(a) === JSON.stringify(b)   // still false? look closer
[...a].map(c => c.codePointAt(0).toString(16))
[...b].map(c => c.codePointAt(0).toString(16))
a.length                   // UTF-16 units; emoji may count as 2
[...a].length              // code points

# Python
print(repr(a), repr(b))
print([hex(ord(c)) for c in a])
print([hex(ord(c)) for c in b])
print(len(a), len(b))      # code points

In the text diff the two pastes will show a change even when the glyphs match. The word and character counter will disagree with the length you count by eye. Both tools run in your browser.

Cause 1: a character you cannot see

Zero-width space (U+200B), zero-width joiner, a soft hyphen or a bidi mark can sit in the middle or at either end. They add to .length / len() and break equality, hashes, cache keys and unique-index lookups.

a.trim() === b.trim() is not enough. JavaScript’s trim() and Python’s str.strip() remove Unicode whitespace (including non-breaking space) but they keep zero-width spaces, because those are format characters, not spaces. JavaScript trim() does also drop a leading BOM (U+FEFF); Python strip() does not. See invisible characters in a copied password for the full table.

// Strip the usual invisible suspects, then map NBSP to a real space
const clean = s => s
  .replace(/[\u200B-\u200D\uFEFF\u00AD\u2060-\u2064\u2066-\u2069\u202A-\u202E]/g, "")
  .replace(/\u00A0/g, " ");

# Python
import re, unicodedata
def clean(s):
    s = unicodedata.normalize("NFC", s)
    s = re.sub(r"[\u200B-\u200D\uFEFF\u00AD\u2060-\u2064\u2066-\u2069\u202A-\u202E]", "", s)
    return s.replace("\u00a0", " ")

Do not blindly strip U+200D (ZWJ) from text that should keep emoji families intact.

Cause 2: the same letter, written two ways

Unicode allows more than one sequence for many accented letters. “café” can be:

FormCode pointsLooks like
NFC (composed)U+0063 U+0061 U+0066 U+00E9café
NFD (decomposed)U+0063 U+0061 U+0066 U+0065 U+0301café

They print the same and they are not equal. macOS file names are often NFD; most web input and Linux are NFC. A string copied from a Mac path into a test written on Linux is a classic flake.

a.normalize("NFC") === b.normalize("NFC")          // JavaScript
unicodedata.normalize("NFC", a) == unicodedata.normalize("NFC", b)

Anvil’s /v1/text/inspect flags not_nfc and can return a normalised form with form=nfc. Do not send secrets to it.

Cause 3: a look-alike from another alphabet

Latin a (U+0061) and Cyrillic а (U+0430) look the same in most fonts and compare as different. That is a homoglyph, not an invisible character: you can see a letter, just not which letter.

A reliable compare

  1. Normalise both sides to NFC.
  2. Replace non-breaking spaces with ordinary spaces.
  3. Strip the zero-width and bidi characters you do not want (keep ZWJ if you need intact emoji).
  4. Then compare, hash, or use the string as a cache key.

For a visual walk-through of hidden marks, see Wirebook’s ghost-footprints lesson.

Related

Invisible zero-width characters · Invisible characters in a password · Non-breaking space vs space · Look-alike (homoglyph) characters · Mojibake · Text diff · Anvil text X-ray · Wirebook: invisible characters

Questions

Why do two strings that look the same fail === in JavaScript?

They contain different code points. Hidden zero-width characters, a non-breaking space, or an accented letter written as two characters instead of one will all make === return false.

Does trim() fix strings that look equal but are not?

Only for characters that count as whitespace. trim() will not remove a zero-width space. Normalise to NFC and strip format characters as well.

What is the difference between NFC and NFD?

They are two Unicode normalisation forms. NFC stores many accented letters as one character; NFD stores the base letter plus a combining accent. They look the same and compare as different until you normalise.

How do I print the real characters in Python?

Use repr(s) or [hex(ord(c)) for c in s]. That shows escapes such as \u200b which the default print hides.

Why does a test pass on my machine and fail in CI?

Different operating systems favour different normalisation (macOS file names are often NFD). Hidden characters can also appear in fixtures copied from chat or docs.

Published .

Try it

Related guides

Newest guides