- Remove 'asciiend' from ascii_string grammar rule (handled by scanner)
- Add scanner logic to skip comment-only lines when measuring indentation
- Update scanner to include 'asciiend' in ASCII_CONTENT token
- Implement external scanner for BLOCK_COMMENT (partial fix)
Results: Reduced parse errors from 156 to 119 (23% improvement)
Allow ］ (U+FF3D) after asciiend in ASCII content scanner.
This enables ASCII strings inside arrays with full-width brackets:
var x = ［ascii...asciiend］
The parser now correctly recognizes this as an array containing
an ASCII string, not as a syntax error.
Fixes 1 additional parsing error.
ASCII blocks can now be used as function arguments and array
elements. The scanner now accepts ) and ] as valid characters
after 'asciiend', allowing constructs like:
- var x = uiAA(ascii...asciiend)
- var y = [ascii...asciiend]
Fixes 12 parsing errors in test scripts.
- Consume \r as part of token instead of skipping it
- Break after consuming \n to avoid processing multiple lines
- Consume leading whitespace separately for indent calculation
- Fix ASCII_CONTENT to return false at EOF without asciiend
This fixes ERROR tokens with CRLF line endings, especially
with trailing blank lines.
- Refactor statement parsing with proper precedence handling
- Improve block structure parsing with indent/dedent support
- Enhance control flow parsing (conditionals, loops)
- Add print command support
- Improve function declaration parsing
- Update scanner for better string and comment handling
- Add comprehensive test corpus
- Better handling of newlines and statement boundaries
