Syntax colour for ST API

[facelessuser]

Glad I could help. I just happened to have done this stuff about a week or two ago. Had I not, I would have been pretty clueless.

Off-topic: What’s a straight-forward way (with the API) to position the cursor? I know I could use run_command, but…

What do you mean position the cursor…move the cursor to postion x on line y?

This is assuming you know how to calculate your new position and create a region.
You usually need to account for multiselect so you would clear the current selections and then add each new selection region:

view.sel().clear() map(lambda x: view.sel().add(x), new_sel_array)

or for one:

view.sel().clear() view.sel().add(new_sel)

Basically you can parse the view.sel keeping the regions you want or just delete them all without checking and add the new one/or ones you want.

[agibsonsw]

Thank you @facelessuser

So something like:

sel = view.sel()[0]
view.sel().clear()
view.sel().add(sel)

would clear all selections and re-instate the cursor to the latest position

[agibsonsw]

Well this works, thank you @facelessuser. Although I’m not entirely happy with my choice of colour (a sort of lilac…) but at list I achieved my aim

I’m not sure I want to pursue this any further just at the moment. I can imagine spending hours (days…) on this . I would learn a little more about regex, ST themes, and have control over colours I suppose.

Regards, Andy.

Edited: I also modified the scope from ‘…global.python’ to ‘storage.type.function.api.python’ so that it should behave better, and chucked the word ‘global’ in with def|lambda.

[agibsonsw]

@facelessuser. Sorry to bore but I’ve got *control *now without too much effort, whe’hay! I copied my new category/dictionary entry ‘storage.type.function.api.python’ to create ‘storage.type.function.general.python’, and added a few standard, and commonly used library, functions to it - see the yellow text in the screenshot. These two categories are near the front of the tmLanguage file, so conflict seems unlikely.

I was under the impression that I would need to study the whole language file to be able to slip these in . So now I can add functions as I go along, and create more categories if I so chose, whe’hay! Job done.

I might try and tackle objects like window, view… This might prove a little trickier though, as I haven’t got an opening brace ‘(’ to latch on to

Regards, Andy.

[facelessuser]

Window and view should be targetable. In the method I am using, I catch all of the object members (even the ones without (); I just don’t scope them with anything; like I said I changed it a lot of things to help me out in the future).

But yeah, there are always more ways than one to do things.

[agibsonsw]

I see now what you were indicating with self, cls . In a line such as

self.window.focus_view(OrderedFilesCommand.file_views[index][1])

focus_view is my chosen colour, but self and index are no longer the colour they should be. My expression

<string>(?:[a-zA-Z_][a-zA-Z0-9_]*\.)*(append|lower|sort|split|itemgetter)\s*(?=\()</string>

is not sufficient to discount everything else on the line.

Can you elaborate on “I just don’t scope them with anything” please? This should help me…

**

[facelessuser]

[quote=“agibsonsw”]I see now what you were indicating with self, cls . In a line such as

self.window.focus_view(OrderedFilesCommand.file_views[index][1])

focus_view is my chosen colour, but self and index are no longer the colour they should be. My expression

<string>(?:[a-zA-Z_][a-zA-Z0-9_]*\.)*(append|lower|sort|split|itemgetter)\s*(?=\()</string>

is not sufficient to discount everything else on the line.

Can you elaborate on “I just don’t scope them with anything” please? This should help me…
[/quote]

This is what I was trying to say earlier. I already accounted for all of the stuff, so I was suggesting that you leverage the work I already did.
Basically in my version I don’t do this (?:[a-zA-Z_][a-zA-Z0-9_].). This will greedily include any parent object that happens to come before your target. That is the way it was doing things before I touched it. If an object had a function in it, all preceding object members got pulled into the scope which I though overzealous. So what I did is change the default function check to not greedily include those members, and then I catch them with this and give them no additional scope (I should probably check for lists here as well):

<key>generic_object_names</key> <dict> <key>match</key> <string>(\.\b([A-Za-z_][A-Za-z0-9_]*)\b(?!\()|\b([A-Za-z_][A-Za-z0-9_]*)\b\.)</string> </dict>

So you can see here I look for built in things first, and if I don’t find those, I throw out all the dot members that aren’t function calls etc, then I check for API and then functions, and then lists etc.:

<dict> <key>include</key> <string>#builtin_functions</string> </dict> <dict> <key>include</key> <string>#builtin_types</string> </dict> <dict> <key>include</key> <string>#magic_function_names</string> </dict> <dict> <key>include</key> <string>#language_variables</string> </dict> <dict> <key>include</key> <string>#generic_object_names</string> </dict> <dict> <key>include</key> <string>#st2_api</string> </dict> <dict> <key>begin</key> <string>(?:\.)?([a-zA-Z_][a-zA-Z_0-9]*)\s*(?=\()</string> <key>beginCaptures</key>

I think I also made some minor adjustments to ensure that scope and built in globals get highlighted correctly too, but I would have to diff my changes to remember what I did.

[quote=“agibsonsw”]**
Not so silly now eh? [/quote]

[agibsonsw]

@facelessuser. What seems to be relatively straight-forward is getting complicated . Part of my confusion was/is that even though I use a non-capturing group (?: ), it seems that the more general term ‘function’ later on collects this text.

It also seems odd that only the word ‘self’ gets caught out, although I think ‘key’ has also lost its colour, and (possibly) index.

So changing the key ‘captures’ to ‘match’ is not likely to help me? And you wouldn’t recommend that I ditch the name-part ‘function’ all-together? Edited: Ignore this remark about ‘function’ - I realise this wouldn’t help (it IS a function, regardless).

I apologise . I should be studying all the content you have supplied me with in great depth, but I’m still taking pigeon steps at the mo. I could just “borrow” your language file, but I’m a bit obstinate

[facelessuser]

I apologise . I should be studying all the content you have supplied me with in great depth, but I’m still taking pigeon steps at the mo. I could just “borrow” your language file, but I’m a bit obstinate

It is what makes us programmers. Early on with javascript, I wrote my own simple framework (not because I thought I could do better than Jquery; I think Jquery is great and way better) but I just wanted to learn as much as I could about javascript…it was both fun and painful, but I ended up with a pretty usable framework. In the end, I think I learned more about web browsers and javascript than I ever cared to learn.

[agibsonsw]

@facelessuser. Ditto. I wrote my own JavaScript library . Admittedly, some methods such as $() and getElementsByClassName() I pretty much copied letter for letter, but most methods I adapted (improved upon…), including creating my own sort procedure and date parser. I wanted to look at jQuery, but I didn’t fancy reading another book. So I went on to the jQuery forum and answered tons of questions (before I’d even typed a line of jQuery code ).

Anyway, I think I’ve short-circuited my labours. I realised that the function greediness is how the language file behaves. I wasn’t looking to correct this behaviour, merely to add a number of specific methods/terms to the language. And it’s mainly the term ‘self’ that was an issue (the word ‘key’ isn’t even in the file!).

For the moment I’ve added the terms self and cls in with global, very near the top of the file. The screenshot shows that self, and my api calls, are colour-coded . If the function scope gobbles up other terms then that’s SEP! [Actually, placing ‘self’ near the top of the file may prevent ‘function’ from gobbling up text anyway…]

All I need to do now (hopefully) is decide what words/methods/additional categories I want and flatten out the colours a bit.

Regards, and thanks again, Andy.

[agibsonsw]

… in fact, I might just make *everything * orange and be done with it - it will remind me of a simpler time. Andy.

[facelessuser]

… in fact, I might just make *everything * orange and be done with it - it will remind me of a simpler time. Andy.

I have to admit, I understand what you mean. Digging around in these tmLanguage files isn’t that fun…with all of the regex, and ensuring the proper order of definitions, and the recursion…

[agibsonsw]

I realise that I’m opening myself up to mockery, but I welcome criticism of the attached screenshot

My view (excuse the pun…) on this topic is:

Every character should be legible;
There shouldn’t be too harsh a contrast between any of the colours;
Important *marker *words, like ‘class’ and ‘def’, should be more pronounced;
Comments should seep into the background, but remain legible.

Andy.

[facelessuser]

Not my taste, but it looks like you are getting the hang of the tmLanguage files. The important thing is that it works for you .

[agibsonsw]

Not my taste, but it looks like you are getting the hang of the tmLanguage files. The important thing is that it works for you .[/quote]

Yes, it is significantly different to your earlier screenshot!

For my further education, could you/someone describe in brief the difference between ‘match’ and ‘captures’ in tmLanguage? And what is the purpose of ‘includes’ please?

Regards, Andy.

[facelessuser]

The whole file is parsed by the regex definitions. As soon as a chunk of code is processed with a definition, the syntax parser moves on and that chunk of code will not be touched again.

Matches: matches your regex and assigns it your scope to group0 only (if one is provided) and moves on.

Captures: allow you to capture the specific groups and assign scopes to each group and also the entire match if desired. This example from python assigns group 1 a scope, but the entire match (group 0) has no no scope assigned to it.

<dict> <key>captures</key> <dict> <key>1</key> <dict> <key>name</key> <string>storage.modifier.global.python</string> </dict> </dict> <key>match</key> <string>\b(global)\b</string> </dict>

Begin/End Captures: Allow you to give a start capture and and end capture (as I understand, the end capture and reference the begin capture with lookaheads etc. and I think vice versa), but then it also allows you to then provide regex for the content between the Begin and End. You can even do recursion as I understand.

[agibsonsw]

Thanks. ‘includes’ ?

Am I mistaken, or does ST not recognise regex-lookbehinds? I tried and it seemed to fail.

BTW I’m tempted to try and colour braces ( ), but I think I would be in for a world of pain

Andy.

[facelessuser]

Oh yeah, includes. You can usually have a repository section in your tmLanguage and you give the definition key names. Then you can include the definition by its key name. Also, languages included other sources. You can see in things like the HTML language they will include “source.js” to pull in javascript tmLanguage file. It might be more powerful than that, but that is what I understand at this point.

[agibsonsw]

Mmmm lot to learn

The more I look at ‘Python.tmLanguage’ the messier it appears. There is a large collection of standard functions not listed. Also, a word like ‘set’ is used in a number of contexts, so it’s difficult to distinguish the ST-API version. And datetime is both a module(object) and an object, etc.

I think I’ll try and move on to something else for a while. Regards, Andy

[quodlibet]

Am I mistaken, or does ST not recognise regex-lookbehinds? I tried and it seemed to fail.

If you still haven’t worked this out, you need to escape the “less than” sign. E.g., (?<=\W) should read (?<=\W)

Alex