Sublime Forum

Dev Build 3153

#1

Dev Build 3153 is out now at https://www.sublimetext.com/3dev

The major change in 3153 is a change to how syntax embedding works, which makes languages embedding lots of others more practical, and can also significantly reduce memory usage. This change is Will’s baby, so I’ll let him to go into more detail on how it works.

Another change of note is that we’ve now switched to compiling with clang for the Linux builds, as part of solving the oldish-linux-distribution compatibility issues that have arisen in the current series of dev builds. Things may be broken on Linux because of this - let me know if they are.

Also in 3153 is “Hashed Syntax Highlighting”, which is the same concept that has been around for a long time in KDevelop under the name “Semantic Highlighting”. The idea is that a given word is assigned a color from a table based on the hash of its contents. This way most variable names etc are different colors, but the same name always has the same color.

Hashed syntax highlighting is available via the .sublime-color-scheme format: if a rule supplies a list of colors for the foreground, instead of a single color, then the list of colors will be treated as a gradient (with interpolation in HSL space) which is used to fill the color table. An example:

{
    "rules":
    [
        {
            "scope": "source - punctuation - keyword",
            "foreground":
            [
                "hsl(100, 100%, 56%)",
                "hsl(200, 100%, 56%)",
            ]
        }
    ]
}

Saving this as Monokai.sublime-color-scheme in your user directory will give you rainbow colored variables.

This is very much an experimental change, so let me know how you go with it.

17 Likes
Can someone explain to me the "syntax" and "embed" actions in .sublime-syntax?
#2

When the core CSS package is disabled and the CSS3 package is enabled instead, I get this error:

Error loading syntax file “Packages/PHP/PHP.sublime-syntax”: no such target CSS.sublime-syntax#rule-list-body

Is this something for Sublime Text core or should I report the issue to the CSS3 package?

Cheers!

3 Likes
#3

One quick note about storing the loading plugins in __plugins__, re: https://github.com/SublimeTextIssues/Core/issues/1991

From what I understand Python handles “__double_leading_underscore: when naming a class attribute, invokes name mangling (inside class FooBar, __boo becomes _FooBar__boo; see below).” https://www.python.org/dev/peps/pep-0008/#descriptive-naming-styles

I know that one is about class attributes, but maybe the double underscore could cause other hidden issues. I should have mentioned this in the issue.

0 Likes
#4

After the update I find an error like this with HTML syntax.

0 Likes
#5

Woot, Works on ubuntu 14.04 again!

0 Likes
#6

I was having some issues with the signature on arch,

error: sublime-text: signature from "Sublime HQ Pty Ltd <support@sublimetext.com>" is invalid
error: database 'sublime-text' is not valid (invalid or corrupted database (PGP signature))

Retrying the pacman-key commands didn’t help.

Otherwise, it seems to be running fine.

0 Likes
#7

I tested the code after utilizing the double underscore and didn’t run into any issues. If you do have any issues, let me know.

1 Like
#8

Here is some more info about this build:

  • There are now official docs for Build Systems
  • Memory usage should be improved (to a fairly significant degree) - this is related to syntax definitions, and not open files
  • Syntax definitions should load faster, especially large syntaxes like LaTeX. Previously the LaTeX syntax would cause a brief loading bar the first time a LaTeX file was opened. This should be noticeably reduced.

My plan is to add details on the embed action to the .sublime-syntax docs, but the following will provide the details for the time being:

A new action, embed is now available in .sublime-syntax files. This is used instead of push, pop or set. It accepts a single context/syntax/scope name to push onto the stack. The difference between embed and push is that embed must also specify an escape regex pattern. The first occurrence of this pattern will cause the context stack to be popped all the way up to the embed action. The escape pattern may use back references to capture groups defined in the match regex. Scopes may be added to all embedded content by the embed_scope and also to what escape matches by the escape_captures key.

You can see examples of the usage at https://github.com/sublimehq/Packages/commit/72bd6bdae9e185d9ebedab9679a55cd8ae8c3e60.

This is designed as a replacement for a with_prototype using a lookahead pop pattern to exit from nested contexts. Common usage of this would be for <script> tags, heredocs and fenced code blocks. In addition to being conceptually simpler than a with_prototype and a lookahead pop, it helps reduce memory usage. It won’t replace usage of with_prototype where additional patterns are added to each context. Some syntaxes with heredocs allow variables to be interpolated, so this won’t be usable there.

11 Likes
Can someone explain to me the "syntax" and "embed" actions in .sublime-syntax?
#9

This looks amazing. I’ve been a week or so trying to figure out a coloring syntax for an inden-based language, and this might just be a godsend for this task.

1 Like
#10

I see that we can use escape_captures; could we also get escape_scope or similar to assign a scope to the whole escape sequence?

0 Likes
#11

Capture 0 will assign to the whole match.

0 Likes
#12

Nice trick. Question, can embed be use along with with_prototype too?

0 Likes
#13

No. There wouldn’t be much point since with_prototype copies all of the contexts and creates new ones with added patterns. embed avoids this copying, but doesn’t allow adding patterns. If you want to use with_prototype, go ahead! :smile:

0 Likes
#14

Another question/request: I understand that embed behaves like push in that it adds a new context to the stack. Could we also get something with the functionality of embed but that instead behaves like set, replacing the current context? Or is currently there a workaround to achieve the same functionality with the current tools?

0 Likes
#15

You could use a lookahead to do the set into a context that has two matches: 1 for the embed and another to pop once anything but the embed is found (via a lookahead).

  context1:
    - match: '(?=foobaz)'
      set: embed-foo

  embed-foo:
    - match: '\bfoobaz\b'
      embed: FooBaz.sublime-syntax
      escape: '^\s*foobaz\b'
    - match: '(?=\S)'
      pop: true
1 Like
#16

One further question:
at which point is the escape regex evaluated when evaluating the embedded contexts? I would expect that the escape would be evaluated before any match in the embedded context, but this doesn’t seem to be the case.

0 Likes
#17

The escape is scanned for before any embedded patterns. It effectively creates a hard limit on how far the embedded context can consume anything.

If you are seeing different behavior, and can provide an example, I can look into it.

0 Likes
#18

I was just able to update an Arch machine. Do you have any other sources that could be providing a package named sublime-text?

0 Likes
#19

https://pastebin.com/jnmzEwyD

This test syntax should only allow in each line either a single “a” or a single “a” indented by at most one more tab than the previous line, it should everything else be invalid. To control the level of indent I’m using the

However, currently it allow an a to be preceded by any amount of tabs without being marked as invalid.

(I hope this isn’t be being clumsy with the syntax definition).

0 Likes
#20

I don’t think so. It was refusing to install and giving that error with powerpill, when I switched to pacman it gave me the warning but installed anyway. I just tried again and it’s reinstalling with no problem. ¯\_(ツ)_/¯

0 Likes