URL codec

Why URLs need encoding

A URL is both a human-readable address and a structured string parsed by browsers, proxies, and servers. Certain characters separate components: ? starts the query, & separates parameters, # begins the fragment, / divides path segments, and spaces are awkward in many contexts. If a parameter value itself contains & or =, the parser would misread the structure unless those bytes are written in a safe form.

URL encoding—more formally percent-encoding—replaces unsafe or reserved bytes with a % followed by two hexadecimal digits. The byte space often becomes %20 (or + in the application/x-www-form-urlencoded variant). This is not encryption; it is a transport spelling so structure and payload can coexist.

Try encoding a phrase with the text to URL-encoded tool.

Reserved versus unreserved characters

RFC 3986 defines unreserved characters that usually appear as themselves: letters, digits, and -._~. Reserved characters like :/?#[]@!$&'()*+,;= have roles in URL grammar. Whether you encode a reserved character depends on where it appears. A / in the path is a separator; a / inside a single path segment’s data may need encoding if you want a literal slash in that segment’s value.

Practical rule: encode data you inject into a component so that reserved characters lose their special meaning for the surrounding parser.

Percent-encoding mechanics

1. Express the string as bytes (almost always UTF-8 on the modern web).
2. For each byte that must be encoded, write % plus uppercase or lowercase hex (decoders accept both; emitters often use uppercase).
3. Leave safe characters unchanged.

Example: the space in hello world becomes hello%20world. The UTF-8 encoding of é is bytes C3 A9, so the character becomes %C3%A9.

Decoding reverses the mapping. Incomplete sequences like %Z or a trailing % should be rejected or handled per your platform’s rules—do not guess silently in security-sensitive code.

Query strings and application/x-www-form-urlencoded

HTML forms traditionally encode spaces as + and use & / = as delimiters. Libraries named encodeURIComponent versus URLSearchParams differ in edge cases. When debugging:

• Confirm whether + means space or a literal plus.
• Encode values before joining with &.
• Do not encode the entire URL blindly—scheme and host have different rules.

encodeURI and encodeURIComponent in JavaScript encode different character sets; choosing the wrong one is a frequent source of broken links.

Paths, matrices, and fragments

Path segments should encode spaces and most reserved characters that are not intended as separators. Fragments (#...) are handled by the client and not sent to servers in the HTTP request; encoding still matters for correctness inside the client. Userinfo (user:pass@) is obsolete for secrets in URLs—avoid putting credentials in URLs regardless of encoding.

Double encoding and decoding bugs

Double encoding happens when %20 is encoded again into %2520. Symptoms include literal %20 showing up in UI text or files not found because the server looks for the wrong name. Under-encoding leaves & inside a value and truncates subsequent parameters.

A robust pipeline encodes exactly once at the boundary where a raw string becomes part of a URL component, and decodes once when extracting values.

Internationalized domain names and paths

Hosts use IDNA (Punycode) for non-ASCII domain labels—a different mechanism from percent-encoding path and query data. Paths and queries use UTF-8 percent-encoding. Mixing legacy system encodings (Latin-1 assumed on one side, UTF-8 on the other) produces mojibake after decoding.

Always document UTF-8 as the byte encoding for new APIs.

Security angles

URL encoding is not a substitute for HTML escaping, SQL parameterization, or command-argument safety. A string that is correctly percent-encoded for a query parameter can still be dangerous when later placed into HTML without escaping, or into a shell without proper quoting.

Open-redirect and SSRF issues often involve URLs as data. Validate schemes and hosts after decoding—attackers may encode dots, slashes, or credentials to confuse naive filters. Compare decoded forms carefully; better yet, parse with a standard URL library and allowlist properties.

Logging and privacy

Query strings frequently contain search terms, tokens, or personal data. Encoding does not hide them from server logs or browser history. Prefer POST bodies or headers for sensitive values, and strip secrets from access logs.

Testing checklist

1. Encode a string with spaces, &, =, and non-ASCII characters.
2. Decode and compare to the original code points.
3. Place the encoded value in a real query and read it back from your framework’s parameter API.
4. Check that your framework does not double-decode.
5. Verify + versus %20 behavior matches the media type you use.

The URL encoding tool helps with steps 1–2 during manual exploration.

Common scenarios

Building search links. Encode the user query value; do not encode ?q= itself. OAuth redirect URIs. Exact string matching may be required; encoding differences break redirects. Object storage keys in URLs. Encode path segments so nested keys with spaces work. CSV of links. Watch for spreadsheet software re-interpreting % sequences.

Related encodings

Do not confuse percent-encoding with Base64 or HTML entities. Base64 uses a different alphabet and is for arbitrary bytes in text contexts; HTML entities protect document structure in markup. Use the encoder that matches the receiving parser.

Summary

URL encoding lets arbitrary text ride inside URL components without breaking delimiters. Convert to UTF-8 bytes, percent-encode unsafe values once, and decode once on the way out. Choose APIs carefully for query versus full-URI encoding, validate URLs after parsing for security, and remember that encoding is spelling—not confidentiality.