Sublime Forum

**agibsonsw** · January 13, 2016, 7:14am

Hello.

I’m trying to tweak the Python.tmLanguage file so that I can at least show the API methods in a different colour. I tried modifying

<key>captures</key> <dict> <key>1</key> <dict> <key>name</key> <string>storage.modifier.global.python</string> </dict> </dict> <key>match</key> <string>\b(global|window|etc)\b</string> <<-- add API terms here.
It doesn’t colour-code following a dot - any idea how I can modify the expression to accept a preceding dot?

But, more importantly, it must pick up words before mine. That is, a word like ‘get’ will be found before ‘get_clipboard’, so my colour never appears.

Perhaps if I create a new scope/category entirely, then it will pick up my words. Can someone offer a few pointers in the right direction for me? I realise that I would also need to modify my chosen theme file as well.

I can’t really create a new syntax file, because it’s still a standard ‘.py’ file.

An alternative approach I would like advice on: I could use ‘on_modified’, check it’s a Python file, check it contains ‘import sublime’, find the API methods, use add-regions to re-insert these words with a specified scope. This would give me more control over the scope, and I wouldn’t have to modify the language and theme files.

I welcome suggestions and advice

**facelessuser** · January 13, 2016, 7:14am

It won’t accept a dot because the words are between \b. \b is a word boundary (a zero width representation of where a word [a-zA-Z0-9_]+ begins and ends. This is also why the CSS syntax language will not highlight -moz-border-radius and such. The leading dash cannot be picked up because the word boundary would fall after it. I ended up fixing it for my own personal use.

I am not sure what you are wanting exactly, so I can’t really give advise. Ultimately what exactly are you hoping to highlight. I have done things like this before. I actually modified my python to scope all function calls only after the dot like so.

All function calls that are not built in functions are pink

If you give me a idea of what exactly you are trying to do, maybe I can help some. Some times you have to inject rules before other rules to catch things. I think I made a number of changes until I finally got function calls to highlight proper (without highlighting all parent dot object members and keep built in function names to not highlight when they were dot call functions and not get overridden with my new scope. It was a little tricky.

**agibsonsw** · January 13, 2016, 7:14am

Hello again @facelessuser

Your screenshot is what I’m looking for (although a little bright for my taste ). All my variables, function calls, attributes are plain white/ a single colour, in most themes.

I don’t mind if ALL functions calls are at least a different colour, but I was hoping to specifically colour the API methods: window(), sel(), etc…

Are you able to recount the steps to your kaleidoscope? Andy.

**facelessuser** · January 13, 2016, 7:14am

Even better you can just diff mine with the original and see all of the changes. It is right here on github (I store them here so all of my computers can pull them down):

github.com/facelessuser/sublime … tmLanguage

You can ignore my changes to highlighting python dict objects if there are any (can’t remember if it was a color theme change only, or a syntax language change too). The rest of the changes should all be related to function call highlighting. I think to do the same thing in the Javascript it was a three line change, Python I had to tinker a bit.

**agibsonsw** · January 13, 2016, 7:14am

I’m making some sort of progress with the following code. It manages to find the words and change their formatting, but with a weird background?! I tried a scope of “comment” but a similar effect occurs(?).

[code]import sublime, sublime_plugin

class ExampleCommand(sublime_plugin.EventListener):
# def on_query_completions(self, view, prefix, locations):
# if not view.match_selector(locations[0],‘text.html - source’):
# return ]

def on_modified(self, view):
	sel = view.sel()[0]
	pt = sel.begin()
	if not view.match_selector(pt, 'source.python'):
		print 'not Python'
		return
	if view.find("import\s+sublime",0) is None:
		print 'not using Sublime'
		return
	edit = view.begin_edit()
	print 'here' 
	api_regions = view.find_all("view|sel|begin")
	print api_regions
	view.add_regions('my_key', api_regions, "storage.type.class.python")
	view.end_edit(edit)[/code]

**agibsonsw** · January 13, 2016, 7:14am

@facelessuser Ta. I’ll have a look.

Does ST have a Diff feature? Although, I have WinMerge which is a nice app

**facelessuser** · January 13, 2016, 7:14am

[quote=“agibsonsw”]@facelessuser Ta. I’ll have a look.

Does ST have a Diff feature? Although, I have WinMerge which is a nice app [/quote]

Not really. There are some diff plugins, but they don’t really let you copy things from one to another. More of a simple diff view.

When I am on Windows, I usually use WinMerge as well. I use this other free diff thing when I am on Mac, but I am not very happy with it; I need to find a better one.

Oh keep in mind, my changes to highlight function calls was adding a new scope, but you could use an existing one and then you would not need a color theme change.

**facelessuser** · January 13, 2016, 7:14am

[quote=“agibsonsw”]I’m making some sort of progress with the following code. It manages to find the words and change their formatting, but with a weird background?! I tried a scope of “comment” but a similar effect occurs(?).
[/quote]

A plugin to highlight the API stuff isn’t going to be nearly as nice as it is being built into the tmLanguage file. Highlighted regions are not quite like scoping with a tmLanguage, they are literally a colored region; you can’t change the text color, font-style, or anything else. All you can do is create colored blocks, colored outlines, and colored underlines.

**agibsonsw** · January 13, 2016, 7:14am

Ahh, that’s a shame, thought I was almost there! But an interesting exercise non-the less.

Testing your regex skills, how can I make this:

api_regions = view.find_all(“view|sel|begin”)

only capture if preceded by a dot or other (non-word) character and followed by another non-word character, but NOT capture these additional bits?

**rchl** · January 13, 2016, 7:14am

Sublime has “Show Unsaved Changes…” option in the context menu which can be used for “diffing”. Open original file, paste changed file and use this option (without saving file).

**agibsonsw** · January 13, 2016, 7:14am

Amazing what ST can achieve

**facelessuser** · January 13, 2016, 7:14am

Something like this? API calls in green.

**agibsonsw** · January 13, 2016, 7:14am

[quote=“facelessuser”]Something like this? API calls in green.

[attachment=0]Screen Shot 2012-03-10 at 4.16.05 PM.png[/attachment][/quote]

So how did you achieve this - just for the API calls??

**facelessuser** · January 13, 2016, 7:14am

[quote=“agibsonsw”]

[quote=“facelessuser”]Something like this? API calls in green.

[attachment=0]Screen Shot 2012-03-10 at 4.16.05 PM.png[/attachment][/quote]

So how did you achieve this - just for the API calls??[/quote]

I will update with the info later when I get some time. Hang tight. Gotta spend time with the kids .

**agibsonsw** · January 13, 2016, 7:14am

@facelessuser. Okay. Go play with the kids and their ‘gamestations’

When you get back… I’m sooo… close to what I want with this expression:

It works if I follow the dot with a space ‘. begin_edit()’ but not (currently) without the space?!

**agibsonsw** · January 13, 2016, 7:14am

No worries. I needed a more complex expression - following your lead… That is, I looked through some of your additional regex in .tmLanguage.

This is in place of the keyword ‘global’ which is very near the top of the tmLanguage file. I should then be able to modify the theme colour (that refers to the scope for ‘global’), and if I want to format the word ‘global’ I can* bung it in* with something appropriate.

I might go for something yellow-ish or orange

**facelessuser** · January 13, 2016, 7:14am

Using the my Python modifications that I pointed you to earlier (it needs the other changes I made in the github version to work), I inserted a ST2 API check before checking for normal functions.

And I defined the API section by copying the function section and placing it under the Repository part in the XML with the same name you see above. Don’t replace the original function section, but make sure to add it before. You still want to make sure non API calls get scoped as functions.

<key>st2_api</key> <dict> <key>begin</key> <string>(?:\.)?(size|substr|begin_edit|end_edit|insert|erase|replace|sel|line|full_line|lines|split_by_newlines|word|find|find_all|rowcol|text_point)\s*(?=$)</string> <key>beginCaptures</key> <dict> <key>1</key> <dict> <key>name</key> <string>meta.function-call.st2-api.python</string> </dict> </dict> <key>end</key> <string>($)</string> <key>endCaptures</key> <dict> <key>1</key> <dict> <key>name</key> <string>punctuation.definition.arguments.end.python</string> </dict> </dict> <key>name</key> <string>meta.function-call.python</string> <key>patterns</key> <array> <dict> <key>begin</key> <string>(?=[A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*)*\s*$)</string> <key>end</key> <string>(?=\s*\()</string> <key>patterns</key> <array> <dict> <key>include</key> <string>#dotted_name</string> </dict> </array> </dict> <dict> <key>begin</key> <string>(\()</string> <key>beginCaptures</key> <dict> <key>1</key> <dict> <key>name</key> <string>punctuation.definition.arguments.begin.python</string> </dict> </dict> <key>contentName</key> <string>meta.function-call.arguments.python</string> <key>end</key> <string>(?=$)</string> <key>patterns</key> <array> <dict> <key>include</key> <string>#keyword_arguments</string> </dict> <dict> <key>include</key> <string>$self</string> </dict> </array> </dict> </array> </dict>

I attached the modified python language. Diff it against the version I pointed to on github if you need to.
Python.tmLanguage.zip (6.07 KB)

**agibsonsw** · January 13, 2016, 7:14am

@facelessuser. Thank you, I shall study at length

I’m guessing that your (?:.)? worked because it was registered within a beginCapture.

I thought my task was complete, but not of the theme files contain ‘storage.modifier.global.python’, or even the word ‘global’ at all. Am I able to just create a new dict entry, following an existing structure:

<dict> <key>name</key> <string>Keyword</string> <key>scope</key> <string>keyword</string> <key>settings</key> <dict> <key>fontStyle</key> <string></string> <key>foreground</key> <string>#FF3854</string> </dict> </dict>
and making up a name (Global?)? I’m optimistic that I can.

I’m not so concerned about other function calls at the moment - at least everything will not be the same colour

**facelessuser** · January 13, 2016, 7:14am

There are a couple of advantages to what my version does over yours. There were a number of changes to make sure self and cls still get scoped proper in lines like this:

self.object.api_call()

self and cls don’t get colored in the theme, but they are getting scoped, and my requirement was not to break any other scoping. The one thing I did change was that if you had this:

self.object.another_object.function()

object.another_object.function() would all get scoped as meta.function-call. I found this to be overzealous, so I reduced it to only scope function() as meta.function-call. After that, all I did was add generic to the function call so that all was pretty much the same meta.function-call.generic, but you add an extra specifier you could target it in your theme, but you don’t have to and it would then be unhighlighted and benign.

By inserting the api check right before the normal function check, you get two kinds of function calls to target (keep in mind, built in functions are not touched by these, so they still get scoped like normal; I think I might have made some adjustments to ensure this).

meta.function-call.generic.python and meta.function-call.st2-api.python

I highlight like this if all I want to highlight is the api:

<dict> <key>name</key> <string>Function Call</string> <key>scope</key> <string>meta.function-call.st2-api</string> <key>settings</key> <dict> <key>fontStyle</key> <string></string> <key>foreground</key> <string>#A6E22E</string> </dict> </dict>

**agibsonsw** · January 13, 2016, 7:14am

Cool, so I can copy a dictionary entry, use my scope and make up a name - and colour.

I’m not concerned about any other functions, variables, etc., at the moment - I just want(ed) the code a little less monochrome. I’m still learning the API at the moment, so I was particularly interested in these methods.

The ST API documentation is quite limited, and I’m sure it’s only mentioning a fraction of what’s available. It doesn’t mention on_query_completions, extract_completions, match_selector, or describe arguments in enough detail. So it’s been mainly a question of trawling through, and deciphering, any code samples I can find.

I’m quite keen to pursue ‘on_query_completions’ and might build on this, together with a completions (or other source) file. Python and/or, more specifically, the API would be a sensible choice for this (as I can learn the language at the same time ). Besides, CodeIntel seems to have fallen by the wayside. *

I’ll modify the theme file tomorrow… and possibly come back if I can’t get a nice orange effect . Andy.

Off-topic: What’s a straight-forward way (with the API) to position the cursor? I know I could use run_command, but…*

Syntax colour for ST API