Pikkola Text Studio, tool No. 39

One line of text, spelled the way the next program wants it.

Paste a string and read it as a URL parameter, an HTML or XML attribute, a JSON string, a shell argument or a CSV cell. Both directions, with the code point and the UTF-8 bytes behind every character. Nothing leaves the tab.

6
formats, both ways
0
requests sent
UTF-8
bytes, shown

The studio

One box in, six dialects out

Choose the format on the left of the bar and the direction beside it. The result updates as you type, and a mistake is named instead of guessed at.

encodeURIComponent()
Start from a sample
Input
0
Encoder

Output
encodeURIComponent() 0
    The three URL encodings A space, a colon and a slash are where they disagree. Each line can be copied on its own.
    Variant Result Length Copy

    The inspector

    Every character, with the bytes behind it

    The table follows the text in the studio. When a string arrives as mojibake, the byte column is where the break shows.

    Characters
    0
    Code points
    0
    UTF-16 units
    0
    UTF-8 bytes
    0
    ASCII
    0
    Two or three byte
    0
    Four byte
    0
    Per character
    # Char Code point UTF-16 UTF-8 HTML JS

    The reference

    Three things worth knowing before you paste

    The same letter has a different spelling in each format, and the format that reads it back is not always the one that wrote it.

    01

    Three URL encoders, one choice

    encodeURIComponent writes everything except the unreserved characters, so a value put after ?q= survives. encodeURI keeps the punctuation that gives an address its shape, so it is for a whole address rather than one value. The form encoding writes a space as + and keeps only a few characters bare, which is what a server that parses application/x-www-form-urlencoded expects.

    02

    Named, decimal, hexadecimal

    An HTML entity names one character. & is a name, and it only exists for the characters a browser knows by name. Everything else, Hangul and emoji included, is written by number: 가 in base ten or 가 in base sixteen. Browsers accept both, XML accepts both, and an XML file with no document type declaration accepts only the five predefined names.

    03

    JSON is a subset of JavaScript

    A JSON string and a JavaScript string agree on \n, \t, \", \\ and \uXXXX. They differ at the edges: JavaScript also reads \x41, \0 and \u{1F600}, and JSON reads none of those. Anything above U+FFFF becomes a surrogate pair of two \u escapes in JSON, which is why the same emoji is eight characters wide once escaped.

    One character, five spellings
    Plain Code point URL HTML JSON UTF-8
    가U+AC00%EA%B0%80가\uAC00EA B0 80
    😀U+1F600%F0%9F%98%80😀\uD83D\uDE00F0 9F 98 80
    &U+0026%26&\u002626
    spaceU+0020%20 \u002020
    "U+0022%22"\"22
    €U+20AC%E2%82%AC€\u20ACE2 82 AC
    The escapes a string carries
    Written Means JSON URL HTML
    \nline feed, U+000Ayes%0A

    \ttab, U+0009yes%09	
    \rcarriage return, U+000Dyes%0D
    \\a backslash, U+005Cyes%5C--
    \"a double quote, U+0022yes%22"
    \0null, U+0000no%00--
    \u0041A by code pointyesAA
    \u{1F600}😀, JavaScript onlyno%F0%9F%98%80😀

    The table is a reading aid; the studio computes every value above and can copy any of them.

    Questions

    Why is a space a %20 in one place and a + in another?

    Both are the same space. A query string written as application/x-www-form-urlencoded uses + because a form post once sent spaces as plus signs, and a URL path or a fragment uses %20 because + there is only a plus. encodeURIComponent always writes %20, so a value that goes into a query string in a browser is safe with either form, but a server that parses the path as a form will read + as a space.

    Why does encodeURI leave a colon and a slash alone?

    encodeURI is for encoding a whole address, so it keeps the characters that give an address its structure: the scheme separator, the slashes, the query mark and the fragment marker. encodeURIComponent is for one piece of that address, so it encodes all of those except the unreserved characters, which is why a value meant to sit after ?q= needs encodeURIComponent.

    What is the difference between 가 and 가?

    Nothing but the base of the number. Both name the same character, U+AC00, the first Hangul syllable. The decimal form writes the code point in base ten and the hexadecimal form in base sixteen, and a browser reads both the same way. Named entities such as & exist only for a fixed list of characters, so everything else has to be written numerically.

    Why do emoji need two \u escapes in JSON?

    JSON strings are sequences of UTF-16 code units, and every code point above U+FFFF takes two of them. U+1F600 is written as the pair \uD83D\uDE00, the high and the low surrogate. In JavaScript source the newer form \u{1F600} does the same job in one piece, but it is not valid inside a JSON string and older parsers will not read it.

    Is it safe to paste a token or a password here?

    The page cannot send it anywhere: the content policy sets connect-src to none, the scripts are its own, and there is no server that could receive the text. It is not stored either; only the format and the options are kept in this browser. Still, a page is only as trustworthy as the copy you opened, so check the address before pasting anything secret.

    Why does unescaping sometimes produce a replacement character?

    Because the bytes did not form valid UTF-8. For example %EA%B0 alone is the first two bytes of a three-byte character, and a decoder has to put something in its place. The page shows U+FFFD, the replacement character, and says how many sequences failed, so the break can be found rather than hidden behind a guess.