Sublime Forum

Standard way to wait for a command to complete (async / await / yield from)

#1

Hello,
when writing unit tests for my plugins, I am able to write fully reliable tests by using yield / from statements. yield / from let me asynchronously “wait” until a certain condition occurs before proceeding with code execution. This is particularly useful when running external commands (window.run_command(...)). Because the caller doesn’t know when a command completes, I can use yield from to implement logic which infer whether a command completed:

import unittesting  # https://github.com/SublimeText/UnitTesting


def read_view(view):
    region = sublime.Region(0, view.size())
    return view.substr(region)


class MyTestCase(unittesting.DeferrableTestCase)

    def await_content_in_view(self, view, content):
        yield lambda: read_view(view) == content

    def await_open_file(self, path):
        window = sublime.active_window()
        init_view = window.active_view()
        new_view = window.open_file(path)
        yield lambda: window.active_view() != new_view
        return new_view

    def test_write_in_view(self):
        view = yield from self.await_open_file("/foo/bar.py")
        view.run_command("write_view", {"content", "hello world"})
        yield from self.await_content_in_view(view, "hello world")

Unfortunately I cannot use the same strategy when writing ST plugins. E.g. I have this piece of code which sometimes does not work before the sub-command did not complete fast enough:

class MyCommand(sublime_plugin.WindowCommand):
    def run(self):
        self.window.run_command(
            "carry_file_to_pane",
            {"direction": direction, "create_new_if_necessary": True},
        )
        new_view = self.window.active_view()
        new_view.run_command("show_at_center")

To make it work I have to add some delay, like this:

class MyCommand(sublime_plugin.WindowCommand):
    def run(self):
        self.window.run_command(
            "carry_file_to_pane",
            {"direction": direction, "create_new_if_necessary": True},
        )
        sublime_plugin.set_timeout(self.on_open, 100)

    def on_open(self):
        new_view = self.window.active_view()
        new_view.run_command("show_at_center")

…but this is sub-optimal though and unorthodox. E.g., on a slower computer I may need a bigger timeout, and on a faster computer I unnecessarily wait 100 ms. Therefore I ask:

  1. What is the “standard” way, for a sublime text plugin to wait until a (sub) command completes? ST doc provides no direction about this.
  2. Are there frameworks similar to unittesting.DeferrableTestCase that lets you use yield / from from within a ST plugin?
  3. unittesting does a nice job in providing the yield / from abstraction, and it could probably do even better by adding async / await support (tulip / asyncio followed a similar evolution). I wonder if this sort of research/discussion ever occurred within the ST ecosystem.
0 Likes
#2

AFAIK, it’s not a problem of a delay being required to “wait” for a sub-command to be completed, but rather ST not updating UI until the main synchronous command is completed.

Means, carry_file_to_pane requests several UI modifications via API calls, but those are performed not before the main synchronous called command is finished. What set_timeout() is doing is to append on_open to the same message queue. This means the provided delay shouldn’t be of any interest as the callback is not executed before already queued UI modifications are processed.

I haven’t tried it, but you probably won’t suffer this issue, when passing the whole function to the async worker thread via set_timeout_async().

class MyCommand(sublime_plugin.WindowCommand):
    def run(self):
        def run_async():
            self.window.run_command(
                "carry_file_to_pane",
                {"direction": direction, "create_new_if_necessary": True},
            )
            new_view = self.window.active_view()
            new_view.run_command("show_at_center")
        sublime.set_timeout_async(run_async)

This way the main thread keeps idle to handle UI updates.

Support for async/await aka asyncio has been discussed as soon as python 3.8 plugin_host has been published.

The main issue is the totally different programming paradigm. To support asyncio all API end points would need coroutine compatible counter parts. The whole plugin host’s architecture would need major refactorings/rewrites as well, IIRC.

The benefit compared with downsides of introducing an incompatible completely different approach of writing plugins was judged rather low - assuming most plugins just performing small and quick operations.

0 Likes
#3

Thank you @deathaxe. This clarified things a bit. From a developer perspective what I care about is the guarantee that each line of code (including run_command()) blocks and completes execution before continuing to the next line of code. This way I can execute code serially (and reliably).

I assume this is what set_timeout_async is designed to do because your example using it apparently solved my specific issue. I’m not 100% sure I fully understand though.

E.g. if I have to execute two commands in a row, should I do this:

def run(self):
    self.window.run_command("cmd_1")
    self.window.run_command("cmd_2")

…this:

def run(self):
    def run_async():
        self.window.run_command("cmd_1")
        self.window.run_command("cmd_2")

    sublime.set_timeout_async(run_async)

…or this?

def run(self):
    sublime.set_timeout_async(lambda: self.window.run_command("cmd_1"))
    sublime.set_timeout_async(lambda: self.window.run_command("cmd_2"))

Also, are there cases where I should use set_timeout() instead set_timeout_async()? I took a peek at LSP source code, and they make massive use of both APIs interchangeably, but it’s not clear to me when to select one vs. the other.

0 Likes
#5

Your expectations are completely met with regards to calling synchronous functions.

It’s just the UI changes, which are scheduled. I don’t have any insights into ST’s internals, but my understanding is ST being a win32 app with a traditional message loop running in the main thread.

The API calls behind self.window.run_command("carry_file_to_pane", ... call something like SendMessage(...) to manipulate UI, while your MyCommand is being executed synchronously within the same main message loop of ST’s main thread. Therefore any sent message can be processed not before your command has finished. This is just how single threaded win32 apps have been working since Windows 3.11.

What sublime.set_timeout_async() does, is scheduling execution of passed callback within ST’s background worker thread, which runs independently from main UI thread. If a window.run_command() is invoked from background thread, the main thread is most likely idle and can handle any requested UI changes immediately.

Basically we can interpret

  1. sublime.set_timeout(callback) as scheduling a task to be executed in ST’s main thread
  2. sublime.set_timeout_async(callback) as scheduling a task to be executed in ST’s worker thread

That’s what LSP is doing. It tries to immitate the async/await mechanism, using Promisees, which are executed in background worker thread whenever possible, while using set_timeout() to synchronize execution of code pieces with main thread for whatever reason (avoid flickering, etc.)

That’s needed as python 3.3, which it needs to run on due to using dependencies, doesn’t support asyncio. I guess as soon as PC4.0 is out enabling dependencies on python 3.8, LSP will be the first plugin creating its own asyncio message loop to simplify code.

2 Likes