Skip to content

bug: grammar consumes the rest of the source string #14

@char0n

Description

@char0n

Did you check existing issues?

  • I have read all the tree-sitter docs if it relates to using the parser
  • I have searched the existing issues

Tree-Sitter CLI Version, if relevant (output of tree-sitter --version)

tree-sitter 0.24.3 (bdfe32402e85673bbc693216f0a6ef72c98bb665)

Describe the bug

Hi,

I've been playing with how Errors and tree-sitter recovery mechanism work in this grammar and found some interesting cases that I'm not sure how to handle.

---
openapi: 3.1.0
x-top-level: value
info:
   title: Sample API
  unknownFixedField: value
  description: Optional multiline or single-line description in [CommonMark](http://commonmark.org/help/)
    or HTML.
  summary: example summary
  termsOfService: Terms of service
  version::: 0.1.9
  x-version: 0.1.9-beta
  license:
    name: Apache License 2.0
    x-fullName: Apache License 2.0
    identifier::: Apache License 2.0
    url: https://www.apache.org/licenses/LICENSE-2.0 

produces

[ERROR] [0, 0] - [4, 20]
  [block_mapping_pair] [1, 0] - [1, 14]
    key: [flow_node] [1, 0] - [1, 7]
      [plain_scalar] [1, 0] - [1, 7]
        [string_scalar] [1, 0] - [1, 7]
    value: [flow_node] [1, 9] - [1, 14]
      [plain_scalar] [1, 9] - [1, 14]
        [string_scalar] [1, 9] - [1, 14]
  [block_mapping_pair] [2, 0] - [2, 18]
    key: [flow_node] [2, 0] - [2, 11]
      [plain_scalar] [2, 0] - [2, 11]
        [string_scalar] [2, 0] - [2, 11]
    value: [flow_node] [2, 13] - [2, 18]
      [plain_scalar] [2, 13] - [2, 18]
        [string_scalar] [2, 13] - [2, 18]
  [flow_node] [3, 0] - [3, 4]
    [plain_scalar] [3, 0] - [3, 4]
      [string_scalar] [3, 0] - [3, 4]
  [block_mapping_pair] [4, 3] - [4, 20]
    key: [flow_node] [4, 3] - [4, 8]
      [plain_scalar] [4, 3] - [4, 8]
        [string_scalar] [4, 3] - [4, 8]
    value: [flow_node] [4, 10] - [4, 20]
      [plain_scalar] [4, 10] - [4, 20]
        [string_scalar] [4, 10] - [4, 20]

What is happening here is that parser doesn't parse after title: Sample API. This makes it pretty unusable in for example Editor cases. Only part of the source string is parsed and the rest is either ignored or consumed by parser. I would expect CST to contain at least another Error object with additional children nodes.

Did you ever bumped into this issue?

Thanks for any answer

Steps To Reproduce/Bad Parse Tree

https://tree-sitter.github.io/tree-sitter/playground

Expected Behavior/Parse Tree

Parse the entire source string and don't consume it

Repro

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions