How to run part of my plugin in the second thread properly?

[r44083]

Hi, some times ago I wrote plugin for linting C/C++ code. But when I try to lint big projects my plugin blocks GUI of Sublime Text for few second, so I decided to run it in the background thread.
I try change code for threading like this:

class MainClanglinterCommand(sublime_plugin.TextCommand):
	def run(self, edit):
		clanglinter_thread = ClanglinterThread(self, edit)
		clanglinter_thread.start()

class ClanglinterThread(threading.Thread):
	def __init__(self, edit):
		self.edit = edit
		threading.Thread.__init__(self)
	
	def run(self):
		settings.update()
		if settings.get('enable'):
			context.update()
			analyzed_file_content = utils.get_view_content(context.get('view'))
			temp_file_object = utils.create_temp_file()
			utils.write_temp_file(temp_file_object, analyzed_file_content)
			
			cmd = utils.get_full_cmd(settings.get('clanglinter_cmd'),
									temp_file_object.name,
									settings.get('project_settings'))
			
			clang_output = utils.clang_launch(cmd)
			
			parser.update(clang_output)
			parser_output = parser.get_format_output()
			
			ui.regions_clear()
			if parser_output:
				ui.regions_create()
				ui.output_panel_clear(self.edit)
				ui.output_panel_insert_lines(self.edit, parser_output)
				ui.output_panel_show()
			else:
				ui.output_panel_clear(self.edit)
				ui.output_panel_hide()

But when I try to run this, I get error in the Sublime Text console:

Exception in thread Thread-3:
Traceback (most recent call last):
  File "./threading.py", line 901, in _bootstrap_inner
  File "C:\Program Files (x86)\sublimetext\sublimetext\Data\Packages\ClangLinter\ClangLinter.py", line 109, in run
    ui.output_panel_clear(self.edit)
  File "C:\Program Files (x86)\sublimetext\sublimetext\Data\Packages\ClangLinter\ClangLinter.py", line 414, in output_panel_clear
    view.erase(edit, sublime.Region(0, view.size()))
  File "C:\Program Files (x86)\sublimetext\sublimetext\sublime.py", line 701, in erase
    raise ValueError("Edit objects may not be used after the TextCommand's run method has returned")
ValueError: Edit objects may not be used after the TextCommand's run method has returned

Question: how to implement this idea correctly? What I am doing wrong?

[rchl]

ValueError: Edit objects may not be used after the TextCommand’s run method has returned

You would typically post message to a main thread (through sublime.set_timeout for example) and then run a command on a view. Along the command you can pass the data that you got from your thread and process it in the command handler.

[FichteFoll]

Edit objects get automatically “destroyed” once the command it was built for terminates. Since your thread is asynchronous, the object has most likely been closed before the code is reached in the second thread.

The proper way is to invoke a synchronous editing command from within your second thread (i.e. using view.run_command()) to make the changes. I cannot tell from your code what kind of action you perform and which built-in commands you could use, but if in doubt you can just create your own TextCommand to do the approproate actions for you (or use https://github.com/FichteFoll/CSScheme/blob/master/my_sublime_lib/edit.py).

[fico]

I ran into some similar issues recently - check out this thread if you need more info on the edit.py solution that @FichteFoll mentioned.

[r44083]

I can not understand how is the view.run_command() can help me to run part of plugin in background. Can you explain how to use your solution?

Also, I tried to use construction below:

class MainClanglinterCommand(sublime_plugin.TextCommand):
	def run(self, edit):
		sublime.set_timeout_async(AsyncClanglinter.clang_thread(self, edit), 0)

class AsyncClanglinter():
	def clang_thread(self, edit):
		...
		...

But it also doesn’t work. My plugin still freeze UI during working. Why sublime.set_timeout_async(AsyncClanglinter.clang_thread(self, edit), 0) doesn’t work?

[FichteFoll]

You run view.run_vommand() in your second thread (or the sublime.set_timeout_async callback) so that you can edit the view, because text commands automatically create an edit object, which is required for editing. You can absolutely not pass Edit objects to different threads!

[r44083]

So, I change the plugin code like you say. Now it is:

# This command automatically called from EventListener
class MainClanglinterCommand(sublime_plugin.TextCommand):
	def run(self, edit):
		view = sublime.active_window().active_view()
		sublime.set_timeout_async(view.run_command('AsyncClanglinter'), 0)

# This is a main plugin logic class
class AsyncClanglinterCommand(sublime_plugin.TextCommand):
	def run(self, edit):
		...
		...

After this changing, plugin doesn’t work at all. What I am doing wrong?

[kingkeith]

the parameter to the run command has to be a lower case version of the TextCommand class name, with underscores before the upper case chars - i.e. view.run_command('async_clanglinter')

[r44083]

I change
sublime.set_timeout_async(view.run_command('AsyncClanglinter'), 0)
to
sublime.set_timeout_async(view.run_command('async_clanglinter'), 0)
Now plugin works but still freeze UI like before

[bizoo]

Your set_timeout_async call is wrong, the first parameter is a callback.
In your example, the run_command is executed directly and the set_timeout_async does nothing.
Try to replace the 0 with 10000 and you will see that it doesn’t change anything.
You must use a lambda (or something similar):

sublime.set_timeout_async(lambda: view.run_command('async_clanglinter'), 0)

However, I’m not sure it really works as you expect.

IMHO, your example in your first post is correct, except that you cannot use the edit token.
Replace the call that use edit:

ui.output_panel_clear(self.edit)
ui.output_panel_insert_lines(self.edit, parser_output)

With your own TextCommand commands, something like::

self.windows.run_command('output_panel_clear')
self.windows.run_command('output_panel_insert_lines', {'chars' : parser_output})

[r44083]

Could you please explain how to use this part of code: , {'chars' : parser_output}) ?
How can I pass and receive this argument?

class OutputPanelInsertLinesCommand(sublime_plugin.TextCommand):
	def run(self, edit, chars):

Or what?

[kingkeith]

class OutputPanelInsertLinesCommand(sublime_plugin.TextCommand):
def run(self, edit, **kwargs):
        chars = kwargs['chars']

[r44083]

Now I have the error:

..\sublimetext\Data\Packages\ClangLinter\ClangLinter.py", line 22, in run
    clanglinter_thread = ClanglinterThread(self, edit)
TypeError: __init__() takes 2 positional arguments but 3 were given

This is how I start the tread:

# This command automatically called from EventListener
class MainClanglinterCommand(sublime_plugin.TextCommand):
	def run(self, edit):
		clanglinter_thread = ClanglinterThread(self, edit)
		clanglinter_thread.start()

This is top part of thread:

class ClanglinterThread(threading.Thread):
	def __init__(self, edit):
		self.edit = edit
		threading.Thread.__init__(self)
	
	def run(self):
		...
		...

How can I solve this problem?

[FichteFoll]

Please post your entire source code; I will correct it for you.

[r44083]

This is the plugin code
This is archive with all plugin sources

I think that you are interested in lines 21 and 56.

For this plugin you need to add path to the clang.exe into your PATH variable. When you open any file with .c or .h (you can change in settings) plugin will analyze this file.

[FichteFoll]

Try this: http://pastebin.com/AEEE6ANv

I did not test it, but I did eliminate all edit objects being passed around and instead used “synchronous view.run_command calls”, which ST will create edit objects for. I also changed commands to window commands where applicable (no edit object pitfall).

The way you utilize implcit global state with your ClanglinterContext instance is very error prone and may easily bite you in form of a race condition when multiple threads are running at the same time while relying on that global state.
The proper way would be to always pass context information along to whatever processes it, but I did not want to refactor that much. (Note, again, that Edit objects can not be passed because they are destructed at the end of the run method.

[r44083]

Now it works asynchronously! Thank you.
But I have noticed kind of strange behavior. When ST starting, in console I get this error:

sublimetext\Data\Packages\ClangLinter\ClangLinter.py", line 15, in plugin_loaded
    start_thread()
TypeError: start_thread() missing 1 required positional argument: 'view'

When I try to change code like this:

def plugin_loaded():
    settings.update()
    start_thread(view)

I get different error:

..sublimetext\Data\Packages\ClangLinter\ClangLinter.py", line 15, in plugin_loaded
    start_thread(view)
NameError: global name 'view' is not defined

P.S.: at this moment I haven’t enough time to test your modification of plugin, so I will write you after 4 hour (I am at work now).

[FichteFoll]

Ah yeah … just remove line 15, it doesn’t serve any real purpose anyway.

[r44083]

I spent some time to understand how it work. Interesting.
I have also notice strange and unplanned plugin behaviour. I think that the main trouble now is a EventListener triggering during plugin thread operation
For example: when on_activated_async() triggered by user (common case), the plugin thread strats run and cause triggering on_selection_modified_async(). So in analized tab cursor always jump to the position which ST get from analyzer output (It behave like user click on first like in output panel, but plugin does it automatically)
My guess is that happens because the command below

sublime.active_window().run_command('output_panel_insert_lines', {'chars': parser_output})

has diferent behaviour from

view.insert(edit, view.size(), parser_output)

Because of this ST think that user ckick on first line of output panel.
Maybe do you know how can I avoid it?

[r44083]

I solve it (add dirty hack):

	view.set_read_only(False)
	view.run_command('append', {'characters': parser_output})
	
	# Set cursor on the last line, so on_selection_modified_async() will
	# not be called when we don't want
	view.run_command('goto_line', {"line": 0})
	
	# Scroll output panel to first line without staying cursor on it
	view.show(0)
	
	view.set_read_only(True)

Can you also explain what did you mean by this [quote=“FichteFoll, post:18, topic:18962, full:true”]
The proper way would be to always pass context information along to whatever processes it
[/quote]

Do you mean that I need to compare context before thread and after?
P.S.: You were right. class ClanglinterContext() causes BIG problems when working with different files!!