Unicode and UTF-8 URL Encoding

Unicode and UTF-8 URL Encoding: How é and Emoji Become Percent Escapes

A URL is restricted to ASCII, so any character outside that range needs two steps before it can travel. First, the character is converted to bytes using an encoding, and the WHATWG URL Standard and every modern browser use UTF-8 for this step.¹ Second, each resulting byte is percent-escaped individually. Consequently, a single accented letter like é becomes a two-byte UTF-8 sequence, C3 and A9 in hex, which percent-encoding writes as %C3%A9.² An emoji, which needs four UTF-8 bytes, produces four separate %XX escapes for what looks like one character on screen.³ This two-step process is why non-ASCII text in a URL looks longer and more cryptic than the ASCII case. The CapyToolkit encoder performs both steps automatically using the browser's native UTF-8 handling, so pasting any language or emoji into ENCODE mode shows you the exact byte sequence a server will receive.

Why a character becomes multiple percent escapes

Percent-encoding operates on bytes, not on characters, and that distinction is the source of every multi-escape sequence you see. ASCII characters happen to be one byte each in UTF-8, so an ASCII letter needs at most one escape. Non-ASCII characters need more bytes, and UTF-8 assigns each Unicode code point a variable-length sequence: two bytes for characters like é or ñ, three bytes for many CJK characters, and four bytes for characters outside the Basic Multilingual Plane, which includes most emoji.

Consequently, encodeURIComponent('é') returns %C3%A9, not a single two-digit escape,⁴ because the function encodes each of the two underlying bytes separately. This byte-first behavior is fixed by the JavaScript specification⁵ and mirrored by the WHATWG URL Standard, so it is consistent across every modern browser and JavaScript engine, not an implementation quirk you need to work around. Once you accept that percent-encoding never sees "characters" at all, only the bytes a text encoding produced, the escape counts for accented letters, CJK text, and emoji stop looking arbitrary.

Why percent-encoding sees bytes, not characters

The distinction between bytes and characters is the source of every multi-escape sequence. A JavaScript string is composed of UTF-16 code units, but URL encoding operates on the UTF-8 bytes that result from converting those units. This conversion step is invisible in most code, which is why assuming a one-to-one relationship between characters and escape sequences produces wrong expectations. Once you internalize that every escape represents a byte, not a character, the apparently chaotic sequence lengths become predictable.

The byte math behind common examples

Working through the numbers removes the mystery. The letter é maps to Unicode code point U+00E9, which sits in the first two-byte range of UTF-8. In UTF-8, code points from U+0080 to U+07FF encode as two bytes, and the arithmetic for U+00E9 produces the bytes 0xC3 and 0xA9, which is exactly the %C3%A9 you see in encoded output.

Why emoji produce four escapes

Most emoji, including 😀 at U+1F600, sit above U+FFFF, the range that requires four UTF-8 bytes under the standard's encoding rules. Encoding that single emoji therefore yields four %XX sequences, such as %F0%9F%98%80. Yet from the reader's side it is still one visible character; the byte count only reflects how many bytes UTF-8 needs to represent that code point, not how complex the glyph looks. Once you know a character's UTF-8 byte length, the number of escapes it produces is fully predictable. This is also why a string mixing English letters, accented text, and emoji encodes to wildly different lengths for what looks like a similar amount of visible text.

The fact that emoji need four escapes is a direct consequence of their Unicode range, not a choice by browser makers. Code points above U+FFFF require four bytes in UTF-8³ because the encoding scheme reserves the first two bytes for signaling that a four-byte sequence follows, leaving only six payload bits in each of the remaining bytes. That structural requirement means any character outside the Basic Multilingual Plane produces four escapes regardless of how simple it looks visually.

Verifying non-ASCII round trips correctly

Confirming a language or emoji survives encoding is a matter of testing the exact string you plan to use, including any special characters at the edges of the range. Paste text containing accented letters, CJK characters, or emoji into CapyToolkit's ENCODE mode, and the encodeURIComponent row shows the full percent-escaped byte sequence your server will receive.

Building on this, copy that encoded string into DECODE mode and confirm it returns to the identical original text, character for character. This round trip matters because a mismatched encoding assumption on the receiving end, such as a server expecting Latin-1 instead of UTF-8, can turn correctly encoded bytes into garbled text called mojibake. Testing locally in the browser catches that mismatch before it reaches production, since the encode and decode steps use the same UTF-8 assumption your server should also be using.

Catching mojibake before it reaches a user

The safest time to catch a mojibake mismatch is during integration testing, not after a launch. Send the exact same text through encode in the browser and decode in the server pipeline, then compare the byte sequences for a UTF-8 versus Latin-1 mismatch in decoded text. If the server returns different characters, the pipeline is using the wrong encoding, and you should fix it before any user-generated content flows through that path. Running the same check for every language your application supports, not just the ones you tested during development, is the cheapest way to catch a mojibake bug before a user reports it.

When to use this

Use this whenever a URL must carry non-English text, accented characters, or emoji, such as a search query, a display name, or a file name from an international user. Paste the exact string into CapyToolkit to see its UTF-8 percent-encoded form, and verify the round trip decodes back correctly. This check is especially useful when integrating with an API whose documentation is unclear about which byte encoding it expects for non-ASCII parameters, since a quick round trip settles the question faster than reading through inconsistent documentation.

Examples

café

caf%C3%A9

😀

%F0%9F%98%80

東京

%E6%9D%B1%E4%BA%AC

%C3%A9

é