Sublime Forum

ST3: async callbacks are not async

#1

The new async callbacks use dummy threads internally so there are not really async (at least on my Windows 64bit Version, Build 3012).

Example plugin: gist.github.com/schlamar/4986014 (have a look at the timestamps!)

0 Likes

#2

Depends of the meanings of async,

Try to use set_timeout() in place of set_timeout_async() and you’ll see the difference:
ST3 freeze completely during the sleep() execution.

0 Likes

#3

It might be acceptable if one plugin only blocks itself but if one plugin could block ALL others plugins this violates the term “async”.

I don’t understand why he is using dummy threading instead of threading, because with the latter everything should be fine and working as expected.

0 Likes

#4

Yes you’re right, it doesn’t look optimal.
Only Jon could answer…

0 Likes

#5

I guess the current way just leaves you to manage your own thread life cycles, probably for the better.

Even using the *_async apis you generally don’t wanna do too much work without spawning your own thread.

0 Likes

#6

Your conclusion is incorrect. Dummy threads in Python simply refer to threads not created by Python. The async callbacks are indeed run from a separate thread, but the thread is not created by Python code.

That being said, all async calls are currently run in a single worker thread. There’s nothing to stop plugins from creating their own threads, however.

0 Likes

#7

Sorry, I don’t want to be rude, but IMO the new async API is just a half-baked concept which doesn’t solve anything.

Maybe you should developers encourage to use the new “futures” concept which is exactly intended for such async operations and much more powerful:
pypi.python.org/pypi/futures.

Here is a small example as proof of concept: gist.github.com/schlamar/4994029

0 Likes

#8

@schlamar

Try and understand before losing your shit :smile:

Btw, how would you imagine that add_future_callback queues a callback on the main thread? It won’t magically happen.

The callback will be run in the executor thread. It’s senseless to submit highlight_view to the executor. You should be doing what you are instead for ST2.

For ST2 you have to use sublime.set_timeout to call back into the main thread

0 Likes

#9

schlamar: There are plenty of places on the internet to be uncivil, but this forum is not one of them. If you feel the need to prefix something with “I don’t mean to be rude” or similar, then just don’t say it at all.

If a plugin wants to use futures, a plugin can use futures. S3 is fundamentally the same as S2 here, it relies on plugin authors putting in the (large) effort to ensure their plugin doesn’t do any significant work on the main thread. The async events provide one way to do this, and the threadsafe API is another way that S3 makes things simpler for plugin authors. Neither has to be used, they’re simply options that people have. Both of these could be done in S2, via threads and set_timeout, respectively, but the S3 approach makes things a little easier for plugin authors. Saying ‘use futures’, in and of itself, would not.

0 Likes

#10

@castles_made_of_sand
Thanks for the update. The other async library I’m currently working with is queuing automatically callbacks on the main thread so I missed this. But as you already have pointed out, a fix is really straightforward.

0 Likes

#11

@jps
I’m no native speaker so I don’t know if the term “half-baked” is meant offensive (because it shouldn’t). That’s the reason for the prefix :wink:

I get your point and would agree with the current design if every plugin get its own worker thread and the implications are clearly stated in the docs :smile:

0 Likes

#12

[quote=“schlamar”]@castles_made_of_sand
Thanks for the update. The other async library I’m currently working with is queuing automatically callbacks on the main thread so I missed this. But as you already have pointed out, a fix is really straightforward.[/quote]

If you are going to criticise something you should at least show you properly understand it.

Your example revealed the depth of your consideration on the topic.

You are really suggesting that for each plugin, for each *_async callback a new thread is spawned automatically ?

0 Likes

#13

No, one thread per plugin, so that one plugin cannot freeze another one.

0 Likes

#14

Just cause you were surprised by the *_async behaviour doesn’t mean they don’t currently have uses, as is.

Just use your futures and threading and get on with it and even while using the *_async, don’t do too much work.

Build the abstraction you want.

Just throw the callback on a spare thread in a pool type thing?

Gah, I’ve got too much of a headache to talk more about this now! Sick with a damn fever.

0 Likes

#15

This is still an issue - when I call async it makes execution take 300 to 600 seconds on a plugin. Removing it takes 0.01 to 0.05 seconds for loading cached data, and up to a few seconds of lockup for larger files. 10 seconds was recorded once with a file that was around 15,000 lines.

My current work-around plan is to use local networking to speak between my app and sublime text plugin so there wouldn’t be a lockup - and launch the plugin as a new python program.

I don’t like this solution as it does have a few issues - however, it would allow porting to other ides easier and quicker since the script would be stand-alone and the only things passed in would be file-names from a project, and the current file each time the tab is changed - this one would take priority… then all the project files can be mapped in the background when a current file isn’t being mapped…

I’d still rather be able to just call it normally, though so I don’t have to add in a bunch of netcode.

0 Likes