encoding whitespace to preserve strings and comments

most compilers delete comments immediately

but, for transpilation, we want to keep all unaltered text “as is”, i.e. keep comments

comments containing whitespace pose a small problem with OhmJS Syntactic (space-skipping) rules

use URL-style encoding to encode whitespace

use URL-style decoding to decode comments back to original form

preprocess and postprocess done using Lexical rules (i.e. raw PEG), allowing all other passes to use Syntactic parsing rules.

e.g. abc def becomes abc%20def until postprocessing

preprocess strings, too, reduces later headaches

back

definition of what is a “string” may depend on target language