TL;DR: I found a neat way to use Python’s generators to simplify using Sublime Text’s callback-based API. Templates and examples can be found here
Hey there! I’m the co-creator of the NimLime plugin for Sublime Text, and I’d like to show a useful technique that can be used to simplify code in Sublime Text plugins.
As you may or may not know, Python allows resumable functions (called ‘generators’) to be created when the ‘yield’ keyword is used in the body of a function definition:
def count_to_three(): yield 1 yield 2 yield 3 c = count_to_three() print(c.next()) print(c.next()) print(c.next())
I won’t go over the details of generator functions here as there are quite a few tutorials on the subject already, however there is a little-used feature of generator functions that isn’t mentioned very often - values don’t always have to come out of a generator function instance - they can also be sent into one via the send method, like so:
def echo(): a = None while true: a = yield a print(a) e = echo() e.send(None) # We have to send 'None' when initially starting a generator function instance e.send(1) e.send(2) e.send(3)
Since we can send any value into a generator function instance, what’s to prevent sending the generator function a reference of itself? Nothing!
def caller(): this = yield output = yield callback_accepting_function(this.send) print(output) c = caller() c.send(None) c.send(c)
Using this technique, I’ve created a decorator that sends a generator function instance a weak-referenced proxy of itself*****. This allows the generator function instance to send itself as a callback argument to any of Sublime Text’s API functions that take a callback, thereby allowing plugin writers to write their code in a linear fashion, instead of split-up into various separate functions (I’ve included a useful example as a second file in the gist).
Note that there are some caveats when using this technique:
Be aware of memory allocation during a function’s lifetime. Any variables which allocate large amounts of memory will only be freed either when the generator function instance itself is garbage collected, or when the variable holding the allocated object is reassigned and the value’s reference count hits 0. If a long running function allocates large amounts of memory to a variable at a single point, then be sure the reassign or delete the variable when done with it!
An empty yield must be put in every place where the function would normally return - even at the very end - otherwise the generator function instance will raise a StopIteration
exception******. This is primarily due to the design and original use of generator functions in for loops, which use a StopIteration exception as a signal that the loop should stop. Usually nothing particularly nasty will happen if a ‘yield’ is forgotten - the exception will just be printed to the developer console - however it’s sloppy, and not a good idea to rely on this behavior.
Be careful about which thread the generator function instance is running on. Whatever thread the generator function instance’s send() method was last called from is the thread the next code segment will run on. I generally avoid this by making all threaded functions run themselves via sublime.set_timeout . Also, a single generator function instance cannot directly call it’s own send() method, nor can two threads call a single generator function instance’s send() or next() methods at the same time - Doing so will result in an exception.
Although generator objects have ‘send’ methods in both Python 2 and Python 3, the ‘next’ method in Python 2 was renamed to next() in Python 3. I’ve included a helper function in the gist to get the correct ‘next’ method from a generator instance.
If you have any improvements to the code, feel free to send pull requests! I’ve tried to make the code both flexible and lightweight, and have tested it for memory leaks (which, although rare in a GC’ed language, aren’t impossible). Also feel free to point out any mistakes in this post. I’ve done my best to check it for errors both factual and grammatical, but it’s always possible I missed a few.
[size=85]*To prevent circular references. Yes, Python has cyclic garbage collection, but it only activates after memory reaches a certain point[/size].
[size=85]**I could probably write a wrapper to prevent this, but my aim was for the code to be lightweight and fast. Having yet another layer of indirection wasn’t part of the planned design[/size]