Sublime Forum

Custom syntax highlighting - issues with non-word-boundary boundaries

#1

In SmileBASIC, this code:

X = 20
IF Y == 4 THEN X = 40

is equivalent to this:

X = 20IF Y == 4 THEN X = 40

No space is required after a numeric literal. As soon as a non-decimal character appears, it is assumed to be the start of a new statement.

I cannot figure out how to get the syntax highlighting to detect that IF correctly.

Here’s my stripped down .sublime-syntax file:

%YAML 1.2
---
name: SmileBASIC
file_extensions: [prg, sb]
scope: source.smilebasic
contexts:
  main:
    # Conditional
    - match: '(?i)\b(IF|THEN|ELSEIF|ELSE|ENDIF|ON)\b'
      scope: keyword.control.conditional.smilebasic

    # Numbers
    - match: '\b[0-9]+'
      scope: constant.numeric.smilebasic

And what I’m testing it against:

X = 20IF Y == 4 THEN X = 40
VARTHATENDSWITHIF = 7

X = 20
IF Y == 4 THEN X = 40
VARTHATENDSWITHIF = 7

If I include the preceding \b in the keywords, then the IF in the first line is not highlighted. If I remove it, then the IF is correctly highlighted, but the final two letters of VARTHATENDSWITHIF are also highlighted - bad!

On the other hand… As I am typing this, I realize that when I get around to scoping variable names, the latter issue will disappear. Should I just remove the preceding \b?

0 Likes
#2

Should I just remove the preceding \b?

Yes.

The core JavaScript syntax only checks the end of identifiers and keywords, but for a different reason. JavaScript identifiers can contain dollar signs, which are not word characters, so the pattern \bif\b would match the string if$, even though the latter is a valid variable, not an if keyword. The trailing \b was replaced by a negative lookahead, but because lookbehinds hurt performance in sublime syntaxes the leading \b was simply removed. This means that we need special handling to make sure that stray word characters after numbers are not marked as variables.

1 Like