Make the lexer pass whitespace through (as OTHER tokens) for text lines.

With this change, we can recreate the original text-line input
exactly. Previously we were inserting a space between every pair of
tokens so our output had a lot more whitespace than our input.

With this change, we can drop the "-b" option to diff and match the
input exactly.
This commit is contained in:
Carl Worth
2010-05-25 15:04:32 -07:00
parent 808401fd79
commit 9fb8b7a495
3 changed files with 84 additions and 42 deletions
-2
View File
@@ -517,8 +517,6 @@ _token_list_print (token_list_t *list)
for (node = list->head; node; node = node->next) {
_token_print (node->token);
if (node->next)
printf (" ");
}
}