Sublime Forum

Specializing C++ grammar for custom patterns

#1

I am doing unit testing in C++ using a some simple macros that I’ve written myself. I would like create a new grammar which specializes the default C++ grammar to add scopes for my test macros. The reason is that I want to provide different syntax highlighting for the test macros which emphasizes the content of the assertion rather than the “ASSERT_EQUALS” text (for example I might dull the color of the “ASSERT_EQUALS” text or mark it with a special color).

The test files follow a simple shape:

// Example.test.cpp
// Other C++ code
TEST(MyTestFunction) {
    // Other C++ code
    ASSERT_EQUALS(1,2); // will fail
}

Where the macro “TEST” expands to function definition (among other things), and the macro ASSERT_EQUALS is self explanatory.

The issue I’m having is that I can’t extend the C/C++ grammar definition for a function block to override the meaning of ASSERT_EQUALS in the context of TEST methods but still support all the normal C++ syntax highlighting for everything else in the test method. Is there a way to do this?

Here is what I have tried so far (which hopefully illustrates what I’m trying to do). It works for TEST and ASSERT_EQUALS and for normal C++ syntax outside of TEST functions, but does not work for normal C++ syntax inside TEST functions.

{ 
	"name": "SimpleTesting test file",
	"scopeName": "source.c++.tests",
	"fileTypes": "tests.cpp"],
	"uuid": "bbfa5657-536d-4c19-83cd-2f0213dfe703",
    "patterns": 
        { 
            "name": "meta.block.c++.test",
            "begin": "TEST\\s*\\(\\s*[A-Za-z_][A-Za-z0-9_]*\\s*\\)\\s*\\{",
            "beginCaptures": {
                "0": { "name": "meta.function.test.head.c++" }
            },
            "end": "\\}",
            "patterns": 
                {
                    "match": "\\bASSERT[A-Za-z0-9_]*\\s*\\(.*?\\)",
                    "name": "meta.function-call.assert.c++"
                },
                {
                    "comment": "I'm trying to import all the normal c stuff here, but it doesn't work",
                    "include": "meta.function.c"
                },
                {
                    "comment": "I'm trying to import all the normal c stuff here, but it doesn't work",
                    "include": "meta.block.c"
                }
            ]
        },
        {
            "include": "source.c++"
        }
    ] 
}
0 Likes

#2

For those who are interested, I solved the problem by copying the C and C++ grammars into the User folder as an override to the existing C and C++ grammars. I then made direct modifications to those grammars to support the special syntax I have in my testing framework. For example I wrote a pattern to capture ASSERT_EQUALS and inserted it directly above the existing C pattern for scope “meta.function-call.c” so it takes higher precedence (and called it “meta.function-call.assert.c”).

And of course I copied my favorite color scheme into the user folder and made the corresponding modifications to highlight the new syntax.

0 Likes

#3

This is apparently the way to do things like this. Except that precedence isn’t based on the items’ positions but on the positions of their matches. Hard to explain but if you for instance define a syntax with a comment “//” and a string ‘".*?"’ the syntax coloring won’t be affected by which item would be first in the list.

0 Likes

#4

I’m not satisfied with this method at all. It means I’m copy-and-pasting somebody else’s work into my files. Apart from the licensing and versioning issues this might cause (if the original file gets updated it won’t update the copy-and-pasted code), it is also just a pain and an impediment to modular development. It would be much better if there was a way to augment existing packages with small modifications, in the same way the default settings can be overridden with user settings.

0 Likes

#5

I can see a way of doing this by implementing some “new filetype” which allows deleting some entries and adding some others. However, implementing this would be fairly complex because of how language definitions work and you would either have to use a numerical index for the entry to remove or specify a match and a name parameter to somewhat accurately find the correct entry in the original file and delete it. (Of course, this is using/writing a plugin.)

And btw, including “random” scopes in a language definition does not work. You can only specify “#repository-items” which you define locally in the repository root key or the “global” scopeName of other files which is “source.c++.tests” in your case; “include”: “meta.function.c” does not work.

0 Likes

#6

@FichteFoll

I understand why including “random” scopes doesn’t work. I tried it because including “source.c++” worked to include the existing C++ grammar, identified by the scope. So I was hoping that since the whole grammar is associated with a scope that the individual grammar patterns would also be.

Can you elaborate on your idea of a “new filetype”? Would this be a pluggin that generates a new grammar file based on an old one (for example a built-in grammar file) and a “patch” file of insertions and deletions? Or do you know some way of generating this information on the fly (as Sublime reads the grammar) so that the “new” file need not exist in the file system?

I wouldn’t need to actually provide a mechanism for deletion, since sublime already has the pattern prioritization mechanism. I would just be looking for a way to add new patterns to the beginning of a given pattern list. Since patterns have names identifying their scope it would make sense to have a file that can extend existing scopes (which would implicitly “hide” old behavior with the same match). Or a mechanism to include grammar patterns from the old file by naming their scope.

0 Likes

#7

Yes, mostly. I like the patch approach, such as you define a diff file or something and the plugin automatically applies this patch to the default file (e.g. using a build system or hooking into “on_post_save”) while backing up the default file.

0 Likes

#8

TextMate 2 solved this problem using injection grammars, which allow you to define overlays of all sorts.

0 Likes