Sublime Forum

Is it possible write a syntax file for a self-describing grammar?

#1

I have a custom syntax for a simple preprocessing language where each line may contain a prefix. Prefixed lines are escapes into an embedded language that I can handle with an “embed” directive.

It works great, as long as I hardcode what the prefixes are in my syntax file. I just write some contexts that look for the pre-defined prefixes at the beginning of lines, and all is good.

But the language has a directive to change the prefixes. What I really want is instead of having to say:

- match '^#:'
  # handle embedded context

I’d like to say:

- match '^{{my_prefix}}'
  # handle embedded context

where my_prefix could change dynamically as I parsed directives that changed the prefix. Is such a thing possible? I see that I can define variables statically, or that I can use YAML Macros to be able to reuse a syntax file with different hardcoded prefixes, but I don’t see anything other than capture variables that would allow me to parse something from the file and use it in subsequent matches. I don’t think capture variables help me here, though, because I would imagine that they could be overwritten when I embed another syntax (which I don’t control).

Thanks.

0 Likes
#2

I’m fairly sure that the answer is no. Sublime’s syntax engine is context-free and that feature is not. You would need a fundamentally more powerful parser to do what you want here. If you had a small, fixed set of possible prefixes, you could probably handle those (with a lot of code duplication).

I’ve been kicking around an idea for a system that would allow this. A context could be parameterized by a map of variables. When you push a context, you could also push new values for those variables. The context patterns would be derived dynamically from those variables. This would be, in essence, a sane generalization of the old regexp backreference behavior that’s used to implement heredocs in some syntaxes. I think that such a system could reasonably be implemented, but it would be a lot of work for a fairly niche use case.

0 Likes
#3

Thanks! If such a thing is being considered, I think it would help more than just my home-grown case. I don’t know how you could highlight m4, for instance, without such a feature because of its ‘changequote’ directive (I looked at the Autotools syntax for m4, and although I haven’t really used it, I don’t see any handling for changequote).

0 Likes