PRACTICAL CHECKLIST

Invisible Unicode Characters: A Practical Cleaning Checklist

If text looks normal but search, code, or copy-and-paste behaves strangely, inspect the actual code points before rewriting the wording.

Start with the exact text that fails

Keep the original string unchanged. Copy the smallest example that reproduces the problem, then inspect it in a tool that reports Unicode code points and one-based positions. A visual scan is not enough: zero-width characters occupy no visible width, and a byte-order mark can be invisible at the beginning of a file.

Check these characters first

• U+200B ZERO WIDTH SPACE can create an invisible break opportunity or make exact matching fail.
• U+2060 WORD JOINER can prevent a line break while remaining invisible.
• U+00AD SOFT HYPHEN may appear only when a line wraps.
• U+FEFF can be a valid leading byte-order mark, so do not remove it blindly.
• U+200C and U+200D affect joining and emoji sequences; inspect context before removal.
• U+034F COMBINING GRAPHEME JOINER can affect comparisons around combining marks.

Use context before you clean

Record the character name, code point, position, and a short context window. Then decide whether the character is an accidental copy artifact or meaningful text formatting. Removing a joiner can change an emoji or a script’s shaping. Removing direction controls can change how mixed-direction text is displayed. The safest default is selective removal, followed by a comparison with the original.

What this checklist can and cannot tell you

Invisible Unicode inspection can explain many copy, search, and formatting failures. It cannot determine who wrote a passage, prove that text was generated by an AI system, or remove statistical patterns in word choice. Those are different questions. A clean character scan only means that the listed hidden characters were not found or were removed.

Run the check locally

For a position-by-position scan and selective removal, use the zero width space checker. For a broader list of invisible formatting characters, use the invisible Unicode character inspector. Both keep pasted text in your browser tab.