Syntax highlighting in Sublime Text 2

[rhaye]

Hi,

ST2 is a great tool - partially thanks to availability of all the packages. But I have a real concern and there’s one thing that stops me from using (and buying) it: syntax highlighting.

I think that there are two models for syntax highlighting that work:

1. token based (used mostly in simple text editors) - the source file is tokenized and then every token is assigned to a group which has a specific colour assigned,
2. semantical/contextual (used by large IDEs) - the source is parsed and the tokens are coloured depending on semantic/context of the token.

If I understand correctly, what ST2 is doing is a strange mixture of these two: you can specify a set of regexes that match the input and single matches of that regex are mapped to different tokens.
Sorry to say this, but its… whacky. See the simplest example:

You could say that the package has bugs, but I would say that this approach clearly doesn’t work. I understand that it is inherited from TextMate, but in my opinion this should be changed. Regexes can match a very limited set of languages and what TM/ST2 is doing is abusing the concept.

Changing the language definition file(s) is not a solution for me.

Can we go to a simple token based solution where I can define the tokens and group them?
Please comment.

[ubuffon]

Regexes can match a very limited set of languages

Not quite true.

I think you’re really missing a point here. You can always tokenize everything you want through the regexps which is present in ST. In fact most parts of the current syntaxes have common groups for built-in functions for example.

P.S.
About the screenshot you have there - I think it’s a problem not in the syntax definition but in the syntax highlighting, based on definition. Though js syntax defs should be tweaked, true, I’m working on mine right now

[rhaye]

[quote=“ubuffon”]

Regexes can match a very limited set of languages

Not quite true.[/quote]

How’s that? You cannot use regexps to match that param in the function body is the same thing as param in the function signature (in the example above). You need a parser for that. So why try doing what’s not doable? In my opinion these regexes should be used to match simple tokens, not full constructs.

[quote=“ubuffon”]I think you’re really missing a point here. You can always tokenize everything you want through the regexps which is present in ST. In fact most parts of the current syntaxes have common groups for built-in functions for example.

P.S.
About the screenshot you have there - I think it’s a problem not in the syntax definition but in the syntax highlighting, based on definition. Though js syntax defs should be tweaked, true, I’m working on mine right now [/quote]

What point, really? MY point is that, as it is now, JavaScript syntax highlighting is broken. Furthermore, I would need to know a lot of trickery to fix it myself. Why not simplify the whole thing?

[ubuffon]

How’s that? You cannot use regexps to match that param in the function body is the same thing as param in the function signature (in the example above). You need a parser for that. So why try doing what’s not doable? In my opinion these regexes should be used to match simple tokens, not full constructs.

Yep, that can’t be done with regexp, and the behaviour of acknowledging the argument as the local variable of the function is out of the editor functionality. Because it’s an editor, not IDE.

What point, really? MY point is that, as it is now, JavaScript syntax highlighting is broken. Furthermore, I would need to know a lot of trickery to fix it myself. Why not simplify the whole thing?

First of all - the editor REALLY depend on some trickery (json settings files for example), so if you’re willing fancy settings windows - I guess this is the wrong editor for you The regexp is truly a great way to define complex syntax structures for an editor (not ideal, of course)