Fix CRLF handling in external scanner
- Consume \r as part of token instead of skipping it - Break after consuming \n to avoid processing multiple lines - Consume leading whitespace separately for indent calculation - Fix ASCII_CONTENT to return false at EOF without asciiend This fixes ERROR tokens with CRLF line endings, especially with trailing blank lines.
This commit is contained in:
7
src/node-types.json
generated
7
src/node-types.json
generated
@@ -1543,7 +1543,6 @@
|
||||
{
|
||||
"type": "source_file",
|
||||
"named": true,
|
||||
"root": true,
|
||||
"fields": {},
|
||||
"children": {
|
||||
"multiple": true,
|
||||
@@ -2054,8 +2053,7 @@
|
||||
},
|
||||
{
|
||||
"type": "block_comment",
|
||||
"named": true,
|
||||
"extra": true
|
||||
"named": true
|
||||
},
|
||||
{
|
||||
"type": "break_statement",
|
||||
@@ -2067,8 +2065,7 @@
|
||||
},
|
||||
{
|
||||
"type": "comment",
|
||||
"named": true,
|
||||
"extra": true
|
||||
"named": true
|
||||
},
|
||||
{
|
||||
"type": "continue_statement",
|
||||
|
||||
Reference in New Issue
Block a user