Previously sequential commas were not matched.

…omments.DO)

Uses tandy-tokenize and tandy-decomment from
https://github.com/hackerb9/tandy-tokenize.

Source code is included in adjunct directory. If the user does not
have it already installed, they can run 'make tokenize'.

Note, this is not the actual flex source code, but the generated C
files.

Instead of requiring people to install it by hand and put it in the
path, just use the copy in adjunct.

Original code: 18K
Tokenized code: 16K
Decommented Tokenized code: 7K
Crunched Decommented Tokenized code: 6K

Steps to update to latest in the future:

wget https://github.com/hackerb9/tokenize/releases/latest/download/tokenize-cfiles.tar.gz
tar -zxvpf tokenize-cfiles.tar.gz
mv tokenize/* adjunct/
rmdir tokenize
rm tokenize-cfiles.tar.gz

The runner has used up 5min of CPU time already and is still going
trying to download and recompile zillions of packages. All I did was:

     brew update
     brew install gawk

Let's try it without the update first.