How to define complex syntax definitions?

[brandonchinn178]

I’m trying to write a syntax definition for Haskell, but it’s turning out to be fairly difficult since it’s an indentation-based syntax (like Python) but most operations can go on the next line, provided they are indented more, and the syntax ends on the next line that starts at the original indentation. For example:

foo
  :: Int
  -> String
foo x = "foo" ++ show x

This example is equivalent to

foo :: Int -> String
foo x = "foo" ++ show x

or even

foo :: Int -> String; foo x = "foo" ++ show x

Some syntaxes I’ve had trouble writing definitions for include:

-- how to tell the `:: Int -> String` group that `x y = ...`
-- ends the type annotation?
let x :: Int
      -> String
    x y = show y ++ " hello "
in x 3 ++ "world"

-- matching `^(\s*){{ident}}` for `myFoo` lets me write
-- ```
-- match: '^\1\S'
-- pop: true
-- ```
-- on the top level, but when i push another context:
-- ```
-- match: '::'
-- push:
--   - match: '^\1\S'
--     pop: true
-- ```
-- `\1` no longer refers to the captured indentation
class Foo f where
  myFoo :: f -> Int
  myBar :: f -> Bool

If anyone could solve these syntax problems, that’d be great; otherwise if there’s some way to hook into Sublime Text to attach scopes manually in Python, please point me to such a reference. Thanks!

Note: the only relevant topic I found was: Indent-block syntax highlighting, but it doesn’t address this more complicated syntax.

[kingkeith]

have you considered putting the:

- match: '^(?=\1\S)'
  pop: true

in a with_prototype instead of the context that is pushed?

i.e something like:

- match: '^(\s*){{ident}}'
  push:
    - match: '::'
      push:
        - match: 'something'
  with_prototype:
    - match: '^(?:(?=\1\S)|(?!\1))' # pop on same indentation or less
      pop: true

note that these syntax grammars are not great for indentation based languages because they can’t account for non-whitespace affecting indentation, for example in Markdown:

9. test
10. test

   is this a loose list or a new paragraph?

if we were to match (\d+)(\.), there is no way to “convert” the two digits from 10. into two spaces etc.

[FichteFoll]

Didn’t backrefs not work in with_propotype. Ah no, I seem to remember it was variables.

Anyway, trying to get indentation exactly right is a tough task and you’ll most likely yield better results with a best effort approach, where you will be working with lookaheads a lot. I did this in the YAML Syntax, for example, where I only use indentation to determine the termination of block scalars.

[brandonchinn178]

Yeah, I’m trying with with_prototype, but there’s not a lot of documentation for it, so I’m not exactly sure what it’s doing. At the very least, it’s not working at the moment.

If I do

match: ...
with_prototype:
  - match: '^(?:(?=\1\S)|(?!\1))'
    pop: true
push: ...

Will that add this match to all contexts nested in push? And if the pattern matches, will it pop just one push, or will it keep popping?

[FichteFoll]

With_propotype patterns are added to all contexts in the pushed context, recursively. And they are also prepended, so they will have higher precedence than the normal patterns.

[brandonchinn178]

I see. I’m still running into issues. I think I’ve created a minimal example:

class Foo a where
  fooInt :: a -> Int
  fooInt = return 0 

where both fooInt identifiers should be entity.name.function.haskell. I have

main:
- include: type-class

type-class:
- match: '^class\b'
  push:
  - match: '\bwhere\b'
    set: functions

functions:
- match: '^(\s*)({{ident}})'
  captures:
    2: entity.name.function.haskell
  with_prototype:
  - match: '^(?=\1\S)'
    pop: true
  push:
  - match: '::'
    set: type

type:
- match: '{{ident}}'
  scope: variable.type.haskell

And now the first fooInt is entity.name.function.haskell, but the second is variable.type.haskell. If with_prototype has the highest precedence, shouldn’t the type context be popped before matching {{ident}} for variable.type.haskell?

[FichteFoll]

When you set the type context, the pushed and prototyped context is replaced, meaning the prototype patterns aren’t active anymore.

[brandonchinn178]

It still doesn’t work when I change

push:
- match: '::'
  set: type

to

push:
- match: '::'
  push: type

[brandonchinn178]

@FichteFoll Thanks for replying consistently so far. Do you have any thoughts on that last question?

[FichteFoll]

I’d have to test and debug this to give more insight. So far I’ve only commented while on mobile.

I’ll see when I get around to it, but no promises.

[brandonchinn178]

Ok, thank you so much! No rush