Fix CRLF handling in external scanner

- Consume \r as part of token instead of skipping it
- Break after consuming \n to avoid processing multiple lines
- Consume leading whitespace separately for indent calculation
- Fix ASCII_CONTENT to return false at EOF without asciiend

This fixes ERROR tokens with CRLF line endings, especially
with trailing blank lines.
This commit is contained in:
2025-11-27 01:25:06 +01:00
parent 0b78c43138
commit eaf0963459
5 changed files with 12315 additions and 12389 deletions

7
src/node-types.json generated
View File

@@ -1543,7 +1543,6 @@
{
"type": "source_file",
"named": true,
"root": true,
"fields": {},
"children": {
"multiple": true,
@@ -2054,8 +2053,7 @@
},
{
"type": "block_comment",
"named": true,
"extra": true
"named": true
},
{
"type": "break_statement",
@@ -2067,8 +2065,7 @@
},
{
"type": "comment",
"named": true,
"extra": true
"named": true
},
{
"type": "continue_statement",