Encoding & Conversion Lab
A technical reference manual for representing the same data safely across text-only channels and character sets.
Runs entirely in your browser
This tool executes locally using standard browser APIs (for example window.crypto.getRandomValues or crypto.subtle where applicable). Your input is not transmitted to or stored on a ScriptPulse server. See How Calculations Work.
Overview
Encoding & Conversion Lab's 8 tools cover binary-to-text encoding schemes (Base32, Base58, Base85) alongside binary/hex conversion, ASCII reference, and Unicode inspection/normalization/detection — the mechanics of representing data as text safely, compactly, or verifiably across different contexts.
This category has complete, single-hub coverage under Encoding & Conversion, which additionally absorbed 8 previously-homeless Data Workshop tools (number bases, Roman numerals, color notation, timestamps) that shared the same 'representation conversion' scope genuinely, rather than being forced in as a catch-all.
Why Encoding & Conversion Lab Matters
Encoding mismatches and misunderstandings are a genuinely distinct class of bug from the higher-level format problems Data Workshop covers: a character encoding mismatch produces visually garbled ('mojibake') text even when the underlying bytes are perfectly intact, and choosing the wrong binary-to-text scheme (Base64 where Base32's transcription safety was actually needed, for instance) creates real operational friction later.
Common Workflows
- Encoding a shared secret or token in Base32 specifically when it needs to survive manual transcription or verbal read-back, rather than defaulting to Base64.
- Inspecting an unfamiliar character's exact Unicode code point and UTF-8 byte representation with the Unicode Inspector when debugging a display or comparison bug.
- Normalizing text to a consistent Unicode form with the Unicode Normalization Tool before storing or comparing user-submitted strings.
- Converting between binary and hexadecimal representations of the same byte value with the Binary ↔ Hex Converter for low-level debugging.
- Detecting whether a piece of text is likely ASCII or requires broader Unicode-aware handling with the Character Encoding Detector.
Learning Roadmap: Beginner
- Learn why UTF-8 is backward-compatible with ASCII for code points 0-127 — the single most useful fact for understanding why so much legacy text 'just works' under modern encoding.
- Inspect an emoji's actual Unicode code point and observe that it requires 4 UTF-8 bytes, not the single byte a plain ASCII character needs.
- Compare Base32 and Base64 output for the same input side by side to see the concrete size-versus-transcription-safety tradeoff directly.
Learning Roadmap: Professional
- Understand Unicode normalization forms (NFC/NFD/NFKC/NFKD) well enough to choose the right one for a specific comparison or search requirement, not just apply one by convention.
- Know precisely which binary-to-text encoding (Base32, Base58, Base64, Base85) fits a given constraint — human transcription safety, URL safety, or raw compactness — rather than defaulting to Base64 for everything.
- Treat encoding detection as inherently probabilistic, not a certainty — prefer systems that declare their encoding explicitly over ones that require guessing it after the fact.
Tool Selection Strategy
- Choose Base32 when human transcription safety matters most (shared secrets, verbally-read codes); choose Base58 for a middle ground of compactness and transcription safety; choose Base85 when raw compactness matters more than URL-safety; default to Base64 otherwise.
- Use the Unicode Inspector when you need a specific character's properties; use the Unicode Normalization Tool when two visually-identical strings aren't comparing as equal — these are different tools for different symptoms of the same underlying Unicode representation-multiplicity issue.
- Use binary/hex conversion directly when you already have a numeric value in hand; use the Text to Binary/Unicode tools instead when starting from actual text content that needs to be encoded first.
Tools in Encoding & Conversion Lab
Base32 Encoder Decoder
Encode plain text to Base32 and decode it back locally.
Open toolBase58 Encoder Decoder
Convert text or hex payloads to Base58 (Bitcoin style) and back.
Open toolBase85 Encoder Decoder
Encode files or text to Base85/Ascii85 and decode it back.
Open toolBinary ↔ Hex Converter
Convert binary sequences to hexadecimal representation and vice versa.
Open toolASCII Table Explorer
Search and explore the full ASCII table including control codes.
Open toolUnicode Inspector
Inspect unicode code points, plane categories, and block details.
Open toolUnicode Normalization Tool
Normalize text characters using NFC, NFD, NFKC, or NFKD forms.
Open toolCharacter Encoding Detector
Analyze bytes or text to detect character encoding signatures.
Open toolDeveloper Guide
Go deeper than the tool list — these Developer Hubs cover the concepts behind Encoding & Conversion Lab's tools in full depth.
Comparative Guides
Decide between the technologies and formats Encoding & Conversion Lab's tools work with:
Common Mistakes
- Defaulting to Base64 for a value that specifically needs human transcription safety, when Base32 was designed exactly for that requirement.
- Assuming two visually identical strings are guaranteed byte-identical, without checking whether one uses a composed and the other a decomposed Unicode representation.
- Treating 'mojibake' garbled text as data loss, when in most cases the underlying bytes are intact and the issue is an encoding-mismatch during interpretation, not corruption.
- Confusing a color-notation or numeral-base conversion (a coordinate-system transform) with a security-relevant encoding scheme — they solve entirely different problems despite superficial syntactic similarity.
Standards Landscape
- RFC 4648 defines Base16, Base32, and Base64; Base58 and Base85 have no equivalent IETF standard, originating instead from Bitcoin's reference implementation and Adobe's PostScript specification respectively.
- The Unicode Standard and Unicode Standard Annex #15 (Normalization Forms) govern code points and the NFC/NFD/NFKC/NFKD normalization forms.
- RFC 3629 defines UTF-8, including its deliberate byte-for-byte compatibility with 7-bit ASCII for code points 0-127.
Glossary
- Code point
- A character's assigned numeric identity in the Unicode Standard, written in U+XXXX notation, distinct from its specific byte-level encoding (UTF-8, UTF-16).
- Normalization
- Converting text to one consistent Unicode representation form, resolving cases where the same visual character has more than one valid underlying code-point sequence.
- Base-N encoding
- A family of schemes (Base32, Base58, Base64, Base85) representing binary data as text using an N-character alphabet, trading off compactness against transcription and URL safety differently.
- Mojibake
- Garbled text produced when bytes are correctly preserved but interpreted under the wrong character encoding.
Troubleshooting
- If two strings that look identical fail an equality check, normalize both to the same Unicode form and compare again — a composed-versus-decomposed mismatch is the most common cause.
- If text displays as garbled symbols, check the assumed encoding at the point of interpretation before assuming the underlying data was corrupted or lost.
- If a Base-N-encoded value fails to decode, confirm which specific variant (standard Base64 vs. URL-safe, for instance) the source actually used, since related but distinct alphabets can look superficially similar.
References
Frequently Asked Questions
- Which base encoding should I use for a shared secret or token?
- Base32 — it's fully case-insensitive and its alphabet avoids visually similar characters, making it the safest choice for anything a human might need to read aloud, hand-copy, or display in all-caps, which is exactly why TOTP secrets conventionally use it.
- Why do two visually identical strings sometimes fail to match?
- They may use different Unicode representations of the same visual character — for example a precomposed accented letter versus a base letter plus a separate combining accent. Normalize both to the same form to resolve this.
- Is Base64 the same as encryption?
- No — none of the encoding schemes in this category (Base32, Base58, Base64, Base85) provide any confidentiality. They are fully reversible without any secret and should never be relied on to protect sensitive data.
- Why does UTF-8 look identical to plain ASCII for English text?
- UTF-8 was deliberately designed so every ASCII character (code points 0-127) encodes to the exact same single byte it always had in plain ASCII — a genuine, important compatibility guarantee, not a coincidence.
- What causes garbled ("mojibake") text?
- In most cases the original bytes are intact — the issue is that they're being interpreted under the wrong assumed character encoding somewhere in the pipeline, not that data was actually lost.
- Which hub covers this entire category?
- The Encoding & Conversion hub owns all 16 of its supporting tools (this category's 8 plus 8 absorbed Data Workshop tools), treating the whole set as one coherent 'representation conversion' topic.
- Why did some Data Workshop tools move into this category's hub?
- Tools like number-base conversion, color notation, and timestamp conversion are genuinely representation-conversion problems structurally similar to the base-encoding tools here, so they were absorbed into the Encoding & Conversion hub in Milestone 4 rather than left without a topical home.
- Can encoding detection ever be 100% certain?
- No — it's fundamentally a best-effort inference in the general case, since some byte sequences are valid under more than one encoding. Prefer systems that declare their encoding explicitly over relying on detection.
Related Categories
Other categories whose tools share a Developer Hub with Encoding & Conversion Lab: