Unicode Normalization Tool
Normalize text characters using NFC, NFD, NFKC, or NFKD forms.
Unicode Normalization Tool
Runs entirely in your browser
This tool executes locally using standard browser APIs (for example window.crypto.getRandomValues or crypto.subtle where applicable). Your input is not transmitted to or stored on a ScriptPulse server. See How Calculations Work.
What This Tool Does
- The Unicode Normalization Tool converts text into one of four standard normalization forms (NFC, NFD, NFKC, NFKD), resolving a subtle but genuinely common problem: the Unicode Standard often allows the same visual character to be represented by more than one distinct sequence of code points. An accented letter like é can be a single precomposed code point (U+00E9), or a base letter e followed by a separate combining acute accent mark (U+0065 U+0301) — both render identically, but they are different underlying data until normalized to the same form.
- This isn't a rare edge case: it affects any text processing pipeline that compares, searches, sorts, or deduplicates strings containing accented characters, combining marks, or certain other Unicode features — two strings that look completely identical on screen can silently fail an equality check, a search match, or a uniqueness constraint if they happen to use different underlying representations of the same visual content.
How It Works
- You choose a target normalization form: NFC (composed, canonical), NFD (decomposed, canonical), NFKC (composed, compatibility), or NFKD (decomposed, compatibility).
- The 'C' forms (NFC, NFKC) combine base characters and combining marks into single precomposed code points wherever a precomposed equivalent exists; the 'D' forms (NFD, NFKD) do the opposite, decomposing precomposed characters into a base character plus separate combining marks.
- The 'K' (compatibility) forms additionally fold in characters that are visually or semantically similar but historically distinct — for example, certain typographic ligatures or full-width variants get mapped to their plain equivalents — which the canonical (non-K) forms deliberately leave untouched.
- The browser's native String.prototype.normalize() implementation performs the actual transformation, ensuring the result matches the Unicode Standard's own defined normalization algorithm rather than a custom approximation.
Usage
- Paste the text you want to normalize.
- Choose the target normalization form (NFC, NFD, NFKC, or NFKD).
- Run the normalization.
- Compare the output's underlying representation against your original input, or use it directly in whatever comparison or storage pipeline needs a consistent form.
Examples
- Normalizing a set of usernames to NFC before checking for duplicates, catching two registrations that would otherwise silently create visually-identical but technically-different accounts.
- Converting decomposed text (base character plus separate combining accents) received from one system into NFC before passing it to another system that expects precomposed characters.
- Using NFKC to fold full-width or typographic-ligature variants of characters into their plain equivalents before a text-search operation that should treat them as equivalent.
- Debugging a failed string comparison in application code by normalizing both sides to the same form and confirming they then match, isolating the bug to a representation mismatch rather than genuinely different content.
Real-World Use Cases
- Normalizing user-submitted text (form input, search queries, usernames) to a single consistent form before storing or comparing it, to prevent visually-identical values from being treated as different.
- Debugging why two strings that look identical fail an equality check or a search match, by normalizing both and confirming the underlying difference was a representation mismatch.
- Preparing text for a system (a database unique constraint, a deduplication pipeline) that needs a single canonical representation per distinct visual value.
- Understanding the practical difference between canonical and compatibility normalization before choosing which form fits a specific search or storage requirement.
Best Practices
- Normalize text to a single consistent form (NFC is the most common default) at the point it enters your system — form input, API payloads, imported data — rather than trying to handle every possible representation downstream.
- Choose canonical forms (NFC/NFD) when visual equivalence should be preserved exactly; choose compatibility forms (NFKC/NFKD) when broader equivalences (like ligatures or full-width variants) should also be folded together, which is common for search but can be too aggressive for exact-match storage.
- Normalize both sides of any string comparison consistently — comparing an NFC string against an un-normalized or NFD string will fail even when the visual content is identical.
- Document which normalization form your system standardizes on, so future code doesn't need to rediscover the choice or introduce an inconsistent second convention.
Common Mistakes
- Assuming visually identical strings are guaranteed to be represented identically, without normalizing before an equality check, search, or uniqueness constraint.
- Applying compatibility normalization (NFKC/NFKD) where canonical normalization (NFC/NFD) was actually needed, unintentionally folding together characters that should have remained visually and semantically distinct.
- Normalizing only one side of a comparison (e.g. normalizing stored data but not incoming search queries), producing inconsistent, hard-to-reproduce matching failures.
- Treating normalization as a one-time fix rather than a consistent policy applied at every entry point where new text enters the system, allowing new un-normalized data to reintroduce the same problem later.
Limitations
- This tool applies the browser's standard Unicode normalization algorithm to text you provide; it does not modify data already stored elsewhere — you'd need to apply the same normalization within your own system's data pipeline.
- Normalization resolves representation differences for equivalent characters; it does not resolve genuinely different characters that merely look similar across different scripts (a separate problem sometimes called 'confusables,' not addressed by normalization forms).
- Very large input is processed in the browser and may affect responsiveness for extremely long text blocks.
Technical Reference Guide
- Unicode Normalization Forms — NFC, NFD, NFKC, and NFKD — are formally defined in Unicode Standard Annex #15 (UAX #15).
- The distinction between canonical equivalence (visually and semantically identical) and compatibility equivalence (similar but historically distinct, such as ligatures) is also defined within UAX #15.
- The underlying combining character mechanism that makes multiple representations of the same visual character possible is part of the core Unicode Standard's text-encoding model, not a separate specification.
FAQ
Why do two identical-looking strings sometimes fail to match?
They may use different underlying Unicode representations of the same visual character — for example, a precomposed accented letter versus a base letter plus a separate combining accent. Normalizing both to the same form resolves this.
Which normalization form should I use by default?
NFC (composed, canonical) is the most common default for general text storage and comparison, since it produces the most compact representation while preserving exact visual and semantic equivalence.
What's the difference between canonical and compatibility normalization?
Canonical forms (NFC/NFD) only fold together representations that are exactly visually and semantically equivalent. Compatibility forms (NFKC/NFKD) go further, also folding in similar-but-historically-distinct variants like typographic ligatures or full-width characters — useful for search, but can be too aggressive for exact-match storage.
Does normalization fix strings that just look similar but use different scripts?
No — that's a different problem, often called 'confusables' (e.g. a Latin 'a' versus a visually similar Cyrillic character), and normalization forms don't address it since those are genuinely different characters, not different representations of the same one.
Where should I apply normalization in my system?
As close to the point text enters your system as possible — form submissions, API payloads, imported data — so everything downstream can rely on a single consistent representation rather than needing to handle every possible variant.
Is NFD ever preferable to NFC?
Yes, in specific cases — some text-processing algorithms (certain search or linguistic analysis techniques) work more naturally with decomposed characters, since it separates base letters from their accents explicitly. Choose based on what your specific downstream processing actually needs.
Part of this Developer Hub
This utility is part of our comprehensive Encoding & Conversion topic workspace.
Explore foundational guidelines, technical specifications, and other interactive utilities related to this workflow.
Related Tools
Explore related utilities inside the Encoding & Conversion Lab workshop for complementary engineering workflows.
View all Encoding & Conversion Lab tools