Sublime Forum

**Acecool** · October 9, 2018, 9:50am

So I’m working on a code mapping system which looks over your code, provides error reports, shows you notes / tasks you’ve set up for yourself, and categorizes functions, objects, etc… and outputs it all in a categorized alphabetized layout which is easy to use and a click ( or double click if configured ) jumps you to the line of code related to the output line in the output panel…

I have been having some issues having it run async…

With threading mode off:
When the file needs to be re-processed, the UI Thread locks up for a small amount of time of 0.1 to 0.5, sometimes longer - I want to get rid of this.
When the file is cached, it loads up nearly instantly but there is about a 0.01 to 0.1 lockup time…

There is a lot going on in the background and the addon is highly adaptable and customizable…

I’ve been working on a Threaded system - the way it works is simple… on plugin_loaded I start the thread… In the thread I use a while true loop to wait until the plugin has loaded before proceeding ( note: If threaded mode is disabled, it doesn’t even get this far and if you switch to enable threaded mode later, via the use of a callback in the settings object it starts it if it hasn’t been started - if it was started and you stopped it just idles, etc… )…

Anyway, it doesn’t matter if the file has a cached output map somewhere or not, it locks the UI up about the same amount of time which seems odd…

The threading system is quite simply designed. In all of the places in my plugin event manager using async calls, if threading is enabled then add the job ( append to a list a simple set of instructions: the view_id, whether we force_remap the file, and whether we update the output panel with the result [ this is so I can enable preprocessing of project files, and / or open-files later on ] )… That takes no time at all.

The Thread determines that there are pending jobs, and processes them. This is where the trouble happens…

It happens in the thread, and I’ve tried using _view.run_command, sublime.set_timeout lambda: view run command, set_timeout_async lambda view run command, running the command by itself, etc… and I’ve had no luck…

The thread is working because I can spam text and the UI thread isn’t affected by it - so it could be how I made some of the commands, etc… I may need to call everything from a deeper level ( instead of some areas having calls, calls and more calls ), but I’m not sure…

Any help is appreciated. When I get this threaded system up I’ll name it version 0.9.X and put it in the public package control - for now it’s private ( although the threaded variant isn’t available yet ) and I’d prefer not to update the branch just yet until I’ve had some time for quality control…

I also haven’t added any type of sophistication to the threading system yet as it is a proof of concept - planned adds will be a timestamp and checking the currently focused view to ensure an incorrect view won’t be displayed ( which can happen if you click through too quickly because of how I pop the data from the list ), sometimes a blank piece of data shows up although that has only happened once and I have yet to re-encounter it after a few updates, and there may be some more issues and more will go into it to ensure no duplicate entries will be processed or queued at the same time, and more…

The ultimate goal, as I’ve said, is to have all files preprocessed in the background ranging from the opened files to the entire project, depending on user configuration, because the cached loader actually works quite quickly even on my 10 year old computer… but it could be quicker… I already have a core file system in place so when one is updated which has any bearing on the outcome of the output data ( config files, back-end code, etc… ) then the cache is not used and the file is reprocessed…

This is basically what is used to determine how to add the job vs simply load the file… I’ve tried set_timeout_async, run command within, and more… no real difference… I’ve tried also using it in things such as on_post_save, on_post_save_async, etc… no real difference either…

		## If the Threading system is used, add the job... Otherwise, use the standard call ( can use acms_preprocess_view, accms_map_file, etc... or the function call itself.. )
		if ( ACMS( ).UseMappingThreads( ) ):
			AddThreadJobs( { 'type': 'Update', 'view_id': _view_id, 'force_nocache_remap': _force_nocache_remap, 'preprocessor_only': _preprocessor_only } );
		else:
			sublime.set_timeout( lambda: GetPluginObject( ).ProcessSourceCodeMapping( _path, _force_nocache_remap, _preprocessor_only ), 0 );

on_post_save_async uses this instead of set_timeout GetPluginObject ProcessSourceCodeMapping:

sublime.set_timeout_async( lambda: _view.run_command( 'acms_preprocess_view', { 'view_id': _view.id( ), 'force_nocache_remap': True, 'preprocessor_only': False } ), 100 )

The thread jobs are added through:

def AddThreadJobs( _job ):
	ACMS( ).GetThread( ).AddJobs( _job );
	print( 'AddThreadJobs has appended the job! -> ' + str( _job ) );
	print( 'Total Jobs! -> ' + str( ACMS( ).GetThread( ).Jobs( ) ) );

Here’s the Thread:

##
## Thread...
##
class ACMSThreadBase( threading.Thread ):
	pass
class ACMSThread( ACMSThreadBase ):
	## Jobs( _override_default = None, _none_if_default = False ), JobsLen( < Getter Args ), JobsLenToString( < Getter Args ), PopJobs( _index = len - 1, _count = 1 ), AddJobs( *_jobs ) and many more....
	## Task: Update Len helper to ensure it is properly returning the length of the object - right now it isn't returning the proper values which is why I have it in setup override as lambda
	__jobs			= AcecoolLib.AccessorFuncBase( parent = ACMSThreadBase,		key = 'jobs',				name = 'Jobs',					default = [ ],																				group = '',			getter_prefix = '',			documentation = 'Thread Job Helpers',			allowed_types = ( AcecoolLib.TYPE_LIST ),							allowed_values = AcecoolLib.VALUE_ANY,			setup_override = { 'use_threading_lock': False, 'add': True, 'pop': True, 'get_len': lambda this: len( this.Jobs( ) ) } )


	## 
	## Helper: Process the Job Entry
	## 
	def ProcessJobEntry( self ):
		print( 'We have unprocessed jobs... #: ' + str( len( self.Jobs( ) ) ) + ' -- ' + str( self.Jobs( ) ) );

		## print( 'Jobs count: ' + str( len( self.Jobs( ) ) ) );
		print( 'Jobs count: ' + str( self.JobsLen( ) ) );

		## Grab the Job / Entry - Note: By default the len - 1'th entry is grabbed meaning the latest one inserted is the first one we look at, which is good... If we switch between files, then the most up-to-date will be the first to process.. Also, when I add a time-check to it, if another view is being processed and is asking to re-process the file and or output the data, we can easily check the timestamp to see which one is newer and also look at which view has focus to prevent any mishaps..
		_job	= self.PopJobs( );

		## Grab the view we're targeting... Could also be from file-name instead if we do project-wide preprocessing instead of opening all files in views, we target files...
		_view_id	= _job.get( 'view_id', None );
		_view		= None;

		## If the _view_id is set, then we should be able to grab the View - but having the view isn't everything - we can easily grab the full file path, view, or a few other things to process the update..
		if ( _view_id != None ):
			_view		= sublime.View( _view_id );

		## Grab the job type ( Update all open files in this window or all windows, all project files, or a single view / file )
		_type		= _job.get( 'type', None );

		## Does this job require us to remap the file?
		_path		= _job.get( 'path', AcecoolLib.Sublime.GetViewFileName( _view, True ) );

		## Does this job require us to remap the file?
		_remap		= True if ( _job.get( 'map', True ) or _job.get( 'force_nocache_remap', True ) ) else False;

		## Should we update the output panel?
		_update		= True if ( _job.get( 'output', False ) or _job.get( 'preprocessor_only', False ) ) else False;


		## 
		## Several ways to update the output panel...
		## Note: Depending on the job, ie whether or not to update all open files ( in all windows or just local ), or all project files, or just a single file / view - handle the job call here... There'll be another helper here, most likely, for each different job type..
		## 
		## ACMS( ).ProcessSourceCodeMapping( _path, _remap, _update );
		## sublime.set_timeout_async( lambda: ACMS( ).ProcessSourceCodeMapping( _path, _remap, _update ), 0 );
		sublime.set_timeout_async( lambda: _view.run_command( 'acms_map_file', { 'view_id': _view_id, 'file': _path, 'force_nocache_remap': _remap, 'preprocessor_only': _update } ), 0 );


	##
	##
	##
	def run( self ):
		## Wait until ACMS Is Valid
		while True:

			##
			if ( IsValidACMS( ) ):
				## If Threads are disabled, we shutdown... We can only do this when we have a reference to the plugin object since we're starting outside of the plugin class as the plugin class can be unloaded and reloaded whenever...
				if ( not ACMS( ).UseMappingThreads( ) ):
					print( '[ ACMS.WorkerThread ] Threading has been disabled in the configuration options - this may cause lockups in the UI thread, it is highly recommended you enable threading support to prevent coding delays.' );

					## Make sure the plugin knows the thread state isn't running
					ACMS( ).SetThreadState( False );

					## Abort the thread startup sequence...
					return False;

				##
				print( 'ACMS Plugin Object IsValid - Starting Thread...' );

				## Make sure the plugin knows the thread state is running...
				ACMS( ).SetThreadState( True );

				##
				break;

		## Assigning Referencce to thread...
		ACMS( ).SetThread( self );

		##
		print( '[ ACMS.WorkerThread ] Threading has been enabled and the Plugin Object has received the worker-thread reference: ' + str( self ) );

		##
		## while True:
		while True:
			## print( 'Thread is running...   ' + str( ACMS( ).GetThread( ) ) + ' / ' + str( self ) );
			## print( 'Thread Notes...   Jobs#: ' + str( self.JobsLen( ) ) + ' / ' + str( self.Jobs( ) ) );

			##
			time.sleep( 1.0 );


			## If Threads are disabled, we idle, check back 4 times per minute to see if we re-enabled ( We do this until we shutdown the thread properly with the ability to relaunch if the user changes their config )...
			if ( not ACMS( ).UseMappingThreads( ) ):
				print( '[ ACMS.WorkerThread.Shutdown ] Threading has been disabled in the configuration options - and is now entering IDLE mode until re-opened... If you start Sublime Text with threading off, it will not enter IDLE and you will have to restart Sublime Text for the thread to start until a system is in place to handle changes and referencing..' );

				while ( not ACMS( ).UseMappingThreads( ) ):
					time.sleep( 15.0 );


			## Preventing some repeats...
			## _lock.acquire( );
			_jobs	= self.Jobs( );
			_len	= self.JobsLen( );
			## _lock.release( );

			##
			if ( _len > 0 ):
				## print( AcecoolLib.Text.FormatPipedHeader( 'We have unprocessed jobs... #: ' + str( len( self.Jobs( ) ) ) + ' -- ' + str( self.Jobs( ) ) + '' ) );
				print( AcecoolLib.Text.FormatPipedHeader( 'We have unprocessed jobs... #: ' + str( _len ) + ' -- ' + str( _jobs ) + '' ) );

			## While we have jobs to process, we process them...
			while ( self.JobsLen( ) > 0 ):
				self.ProcessJobEntry( );
				time.sleep( 0.15 );

				## When we are out of jobs, exit the Job Loop...
				if ( self.JobsLen( ) < 1 ):
					print( 'Jobs Queue has been processed... Waiting for new jobs..' );
					break;



##
## Callback - This is called as soon as the plugin has been loaded by Sublime Text... This is where we initialize everything...
##
def plugin_loaded( ):
	## Create a new instance of the Plugin Object...
	PluginObject = AcecoolCodeMappingSystemPlugin( )

	## Plugin is already loaded - don't re-load it... Call plugin_refreshed( );
	if ( not IsPluginObjectValid( ) ):
		## ## Print out the object data...
		PluginObject.Notify( 'PluginObject Loaded for the first time!' );
		PluginObject.Notify( PluginObject );

		## This is the first load...
		SetPluginObject( PluginObject );

		##
		print( '[ ACMS.plugin_loaded -> AcecoolCodeMappingSystemPlugin.py' );

		##
		plugin_thread_start( );

	else:
		## Run the plugin_refreshed callback... and let it decide whether or not to update the reference or not...
		return plugin_refreshed( GetPluginObject( ), PluginObject );



##
##
##
def plugin_thread_start( ):
	ACMS_Thread_Worker = ACMSThread( );
	ACMS_Thread_Worker.start( );

Hopefully someone has an idea why it takes 0.1 seconds to load cache from using no async or async calls through the plugin event listener ( still a lockup ) or a little bit of time, but with the thread it takes so long…

The Thread isn’t set up with any blocking calls, and I have tried with and without sleep - I just like to keep the sleep to prevent any huge overloads…

If anyone is interested in taking a look, the threaded variant isn’t uploaded at the moment but I may release it later today to get some feedback to see if it locks up for everyone else, or just me ( 10 seconds seems to be how long it takes, and the cache isn’t reloading despite force remap set to False and the Cache being newer than core file updates, etc… — it works flawlessly without Threading, in terms of caching, loading cache, etc… but it’s slow, especially for large files… Small files are near instant but I work on 10,000+ line files and it shouldn’t take thing long… )

So this is the Developer report when not using threads - mine is AcecoolCodeMappingSystem

This list shows how much time each plugin has taken to respond to each event:

on_activated:
    QuickSearchEnhanced.quick_search: 0.000s total, mean: 0.000s, max: 0.000s
    AcecoolCodeMappingSystem.AcecoolCodeMappingSystemPluginReloader: 0.063s total, mean: 0.003s, max: 0.011s
    BracketHighlighter.bh_core: 0.006s total, mean: 0.000s, max: 0.001s
    AcecoolCodeMappingSystem.commands.edit_settings_plus: 0.053s total, mean: 0.002s, max: 0.014s
    SideBarEnhancements.SideBar: 0.015s total, mean: 0.000s, max: 0.004s
    WordHighlight.word_highlight: 0.001s total, mean: 0.000s, max: 0.001s
    Default.pane: 0.000s total, mean: 0.000s, max: 0.000s
    Modific.Modific: 0.000s total, mean: 0.000s, max: 0.000s

on_deactivated:
    Default.history_list: 0.007s total, mean: 0.000s, max: 0.001s
    AcecoolCodeMappingSystem.AcecoolCodeMappingSystemPluginReloader: 0.033s total, mean: 0.001s, max: 0.004s

on_hover:
    Default.symbol: 0.160s total, mean: 0.002s, max: 0.050s
    BracketHighlighter.bh_core: 0.004s total, mean: 0.000s, max: 0.001s

on_modified:
    PackageResourceViewer.package_resource_viewer: 0.095s total, mean: 0.000s, max: 0.002s
    AcecoolCodeMappingSystem.commands.edit_settings_plus: 0.091s total, mean: 0.000s, max: 0.001s
    Default.settings: 0.309s total, mean: 0.000s, max: 0.003s
    SideBarEnhancements.SideBar: 0.048s total, mean: 0.000s, max: 0.001s
    BracketHighlighter.bh_core: 0.081s total, mean: 0.000s, max: 0.002s
    Marking Changed Rows.MarkingChangedRows: 0.669s total, mean: 0.000s, max: 0.009s

on_post_save:
    SideBarEnhancements.SideBar: 0.001s total, mean: 0.001s, max: 0.001s
    Marking Changed Rows.MarkingChangedRows: 0.000s total
    Modific.Modific: 0.000s total

on_pre_save:
    PackageResourceViewer.package_resource_viewer: 0.000s total
    Project Planner.ProjectPlannerSave: 0.000s total
    Default.trim_trailing_white_space: 0.020s total, mean: 0.010s, max: 0.020s
    AcecoolCodeMappingSystem.commands.edit_settings_plus: 0.000s total

on_selection_modified:
    WordHighlight.word_highlight: 0.058s total, mean: 0.000s, max: 0.001s
    BracketHighlighter.bh_core: 0.180s total, mean: 0.000s, max: 0.002s

And with threads active ( restarted sublime text using G9 )

Note: I’m not sure why it isn’t showing the right times…

This list shows how much time each plugin has taken to respond to each event:

on_activated:
    QuickSearchEnhanced.quick_search: 0.000s total, mean: 0.000s, max: 0.000s
    Default.pane: 0.000s total, mean: 0.000s, max: 0.000s
    SideBarEnhancements.SideBar: 0.005s total, mean: 0.000s, max: 0.001s
    AcecoolCodeMappingSystem.commands.edit_settings_plus: 0.037s total, mean: 0.002s, max: 0.007s
    AcecoolCodeMappingSystem.AcecoolCodeMappingSystemPluginReloader: 0.051s total, mean: 0.003s, max: 0.006s
    BracketHighlighter.bh_core: 0.000s total, mean: 0.000s, max: 0.000s
    WordHighlight.word_highlight: 0.003s total, mean: 0.000s, max: 0.001s
    Modific.Modific: 0.000s total, mean: 0.000s, max: 0.000s

on_close:
    AcecoolCodeMappingSystem.AcecoolCodeMappingSystemPluginReloader: 0.000s total, mean: 0.000s, max: 0.000s
    Keep Open On Last Tab Close.keepOpen: 0.000s total, mean: 0.000s, max: 0.000s
    Default.settings: 0.000s total, mean: 0.000s, max: 0.000s
    Default.pane: 0.000s total, mean: 0.000s, max: 0.000s
    SideBarEnhancements.SideBar: 0.001s total, mean: 0.001s, max: 0.001s

on_deactivated:
    AcecoolCodeMappingSystem.AcecoolCodeMappingSystemPluginReloader: 0.016s total, mean: 0.001s, max: 0.005s
    Default.history_list: 0.005s total, mean: 0.000s, max: 0.001s

on_hover:
    Default.symbol: 0.002s total, mean: 0.000s, max: 0.001s
    BracketHighlighter.bh_core: 0.001s total, mean: 0.000s, max: 0.001s

on_modified:
    Default.settings: 0.001s total, mean: 0.000s, max: 0.001s
    SideBarEnhancements.SideBar: 0.000s total, mean: 0.000s, max: 0.000s
    PackageResourceViewer.package_resource_viewer: 0.000s total, mean: 0.000s, max: 0.000s
    BracketHighlighter.bh_core: 0.001s total, mean: 0.000s, max: 0.001s
    AcecoolCodeMappingSystem.commands.edit_settings_plus: 0.000s total, mean: 0.000s, max: 0.000s
    Marking Changed Rows.MarkingChangedRows: 0.000s total, mean: 0.000s, max: 0.000s

on_pre_close:
    AcecoolCodeMappingSystem.AcecoolCodeMappingSystemPluginReloader: 0.001s total, mean: 0.001s, max: 0.001s
    Default.settings: 0.000s total, mean: 0.000s, max: 0.000s
    Compare Side-By-Side.sbs_compare: 0.004s total, mean: 0.002s, max: 0.002s
    Default.history_list: 0.000s total, mean: 0.000s, max: 0.000s

on_selection_modified:
    BracketHighlighter.bh_core: 0.001s total, mean: 0.000s, max: 0.001s
    WordHighlight.word_highlight: 0.000s total, mean: 0.000s, max: 0.000s

This is my console output - the elapsed is part of my timekeeper class which tracks delta ( change between events ) so 7 seconds, and more just to save cache… without the thread running it takes a lot less to save the same file…

Jobs Queue has been processed... Waiting for new jobs..
[ ACMS.PluginEventsManager.Notification ] Returning to Source-View from Output-View... No action necessary!
UpdateSourceViewData -> _file != self.GetPanelName( ) :: ACMS - Panel
AddThreadJobs has appended the job! -> {'type': 'Update', 'preprocessor_only': False, 'view_id': 103, 'force_nocache_remap': False}
Total Jobs! -> [{'type': 'Update', 'preprocessor_only': False, 'view_id': 103, 'force_nocache_remap': False}]
◙                                                                                                                                          ◙
◙    We have unprocessed jobs... #: 1 -- [{'type': 'Update', 'preprocessor_only': False, 'view_id': 103, 'force_nocache_remap': False}]    ◙
◙                                                                                                                                          ◙

We have unprocessed jobs... #: 1 -- [{'type': 'Update', 'preprocessor_only': False, 'view_id': 103, 'force_nocache_remap': False}]
Jobs count: 1
[ ACMS_Python_ACMS > Python > Init -> Panel.Config -> Caching is Enabled -> Run -> Window: 3   -> View: 103   -> File: AcecoolCodeMappingSystem.py                        -> Mapping File -> File Mapped: [ TimeKeeper -> N/A - Elapsed: 7.182515859 ];  -> Cache.Saved: [ TimeKeeper -> N/A - Delta: 0.0          Elapsed: 7.201528072 ];  ]

Here’s the same file without - doing cache save and cache load… for reference…

Loading cache:

UpdateSourceViewData -> _file != self.GetPanelName( ) :: ACMS - Panel
acms_preprocess_view -> Checking Data -> View: <sublime.View object at 0x000001ED08C14E10> -> ReMap: False -> PreProcess Only: False -> File: AcecoolCodeMappingSystem.py -> Path: C:\AcecoolGit\acecooldev_sublimetext3\AppData\Sublime Text 3\Packages\AcecoolCodeMappingSystem\AcecoolCodeMappingSystem.py
[ ACMS_Python_ACMS > Python > Init -> Panel.Config -> Caching is Enabled -> Run -> Window: 3   -> View: 103   -> File: AcecoolCodeMappingSystem.py                        -> Loading Cache -> [ TimeKeeper -> N/A - Elapsed: 0.105022192 ];  ]

Saving:

[ ACMS.PluginEventsManager.Notification ] ACMS -> EventListener -> on_post_save :: The file is a CORE file - C:\AcecoolGit\acecooldev_sublimetext3\AppData\Sublime Text 3\Packages\AcecoolCodeMappingSystem\AcecoolCodeMappingSystem.py
acms_preprocess_view -> Checking Data -> View: <sublime.View object at 0x000001AE7BD1BA90> -> ReMap: True -> PreProcess Only: False -> File: AcecoolCodeMappingSystem.py -> Path: C:\AcecoolGit\acecooldev_sublimetext3\AppData\Sublime Text 3\Packages\AcecoolCodeMappingSystem\AcecoolCodeMappingSystem.py
[ ACMS_Python_ACMS > Python > Init -> Panel.Config -> Caching is Enabled -> Run -> Window: 4   -> View: 132   -> File: AcecoolCodeMappingSystem.py                        -> Mapping File -> File Mapped: [ TimeKeeper -> N/A - Elapsed: 6.926255941 ];  -> Cache.Saved: [ TimeKeeper -> N/A - Delta: 0.0          Elapsed: 6.944257020 ];  ]

Meh, about the same, but the threading support is supposed to make it happen in the background…

The caching loading issue with threading may be something else… It seems to be forcing the update each and every time - so I’'ll look into that, but the fact that it isn’t happening in the background is a bit bothersome…

Oh, link to the addon: https://bitbucket.org/Acecool/AcecoolCodeMappingSystem

**Acecool** · October 12, 2018, 12:19pm

I believe I may have figured it out… Thread scopes…

Because I am creating the single reference to the plugin object outside of the thread, then calling it inside - it is executing outside of the thread instead of inside… I can somewhat verify this by moving my initialization inside the thread and then getting a thread error for my sql system ( which is declared in the ui thread [ or the main plugin space ] too ) meaning it is most likely it…

I’ll set up a few tests to confirm - a large number of print statements outside of the worker thread, timed using timekeeper. Then within and timed - and also seeing if there is a felt lockup. If not, then I’ll create an object which does the same print statements and repeat the test by initializing it outside of the worker thread, printing, using that initialization within the worker, and initializing within the worker then calling again. If the results are different - then I have my solution.

I have been thinking about recoding my plugin object for some time anyways - instead of making it an object, I could set it up to act more as a library… then have an object store data to prevent needing to reload it, for instance file data, etc… This should mean that since it isn’t an initialized object and just a library of functions, that the thread won’t matter… If I don’t, then I’ll look into introducing thread-locking into the object just like the SQL object so that the reference can only be used in the thread it was initialized in which would provide a good error to anyone using it in the future - as a large chunk of my code is a library I am working on in one or more other languages and porting - will be able to find the problem that much quicker…

Although that doesn’t really apply to the library itself, but I have been working on a plugin base - it would be applied there… that way it’d be a nice multi-threaded base anyone can use to create their plugin and have it work as desired the first time around, in terms of multi-threaded implementation, no lockups, etc… as for functionality for the other parts, I’ll likely keep the base plugin as light as possible and build up the framework / library to have more such as my AccessorFunc system…

**slifin** · October 20, 2018, 11:07am

if you do have a threading base please let me know, I’ve been struggling with threading here: Stopping a while loop from another thread

**Acecool** · October 30, 2018, 4:29am

I tried that solution before testing externally because externally it seems to work…

Basically, what I did was set up a helper function to create / retrieve the reference to the plugin object and the SQLite Database object ( as it’ll only work in the appropriate thread it was loaded into ) and those helpers are called in the Listeners which schedule the calls and they use the appropriate thread either by adding a reference to a table and the thread processing that reference and the calls it uses uses the appropriate reference to the threaded object, and SQLite database object… And if threading is disabled, it uses the non-threaded plugin object and SQLite database object…

However, the problem persists, although I have decreased how long it took to load the file in the threaded environment from 0.1 seconds to 0.07, still up from non-threaded taking 0.01 to load…

When I do figure it out, I do intend to release a base plugin which can be used to easily reference the plugin object, create references, etc… ie a PluginNode which houses the references, then GetPluginObject( ), etc… used to return the appropriate one… Threaded scheduler and a simple way to add tasks that can be processed and more.

I’m going to work on that task outside of Sublime Text to make sure the way I have it designed isn’t the cause…

Also, for stopping a while loop from a thread, I’ll look into your setup and see if I can help. You should be able to use notifications / events, or simply have an object that is controlled by the thread and is read-only, then the thread can manipulate the object and the loop should have no trouble getting a result., You could add locking to be sure something else, or more than one thread, isn’t manipulating it while the thread is…

**Acecool** · February 27, 2019, 6:02pm

I figured mine out…

I was calling set_timeout_async on everything but it was still locking up… So, I called set_timeout_async inside the class which is called by the async function in the plugin…

So it goes like this:

Event Listener discovers reason to refresh or change the code mapping from on_save or on_activated -> Call set_timeout_async on my CodeMapper.Run( … )

Now CodeMapper.Run runs set_timeout_async on RunEx

RunEx runs the code which was in Run and then at the end, instead of returning the data ( I believe this is where the issue arose… because it was waiting for this return instead of using a callback to update the view when the data was ready to be updated ) so that calls set_timeout acms_update_view_contents command with the data in question…

Now, I get no more lockups at all…

The one downside is something that took 0.01 seconds to 5 seconds lockup now takes 300 seconds per switch…

I’m still experimenting a little bit… and I want to see if there is a way to set up a priority system with async, or actually use threads ( I have the threading test partially added, so this could make it even faster )…

If it takes 5 seconds to map a file, then I just don’t want the UI to be locked up while it is doing that… But to increase the time to 300 to 500 seconds is ridiculous.

So, I ended up changing all of the async calls to non async… And I only have the call inside of Run as async… If I change to non async, it locks up for 5 seconds or 0.01 to 0.1 if loading a cached file… If I change to async, then it can take 100 seconds to 300+ seconds…

I’ll probably have to create my own thread, which I was playing around with before but ran into other problems since I am also re-designing my mapper to do line by line ( one way ) or chunking, or simply use symbols ( which I need to make a way to edit them via config )… I also need to write my own Settings system since Sublime Text doesn’t allow proper inheritance which is really annoying…

At least it doesn’t lock up, but people can’t wait minutes for the code to be mapped ( I have yet to test the caching system on the async line simply because I’ve been redesigning certain aspects and had that feature disabled for the time being )… I’ll test it soon to see what the time is…

Edit: Actually… It’s down to 100 seconds approx for all of them! weird. That lower value seems to have come from removing all other async calls in my code except for the major one which locked up everything…

**Acecool** · October 9, 2019, 4:39am

Coming back to the problem - I am wondering if someone has a viable solution.

Right now, even if it is only mapping a single file ( even though with the async solution, many can be mapped simultaneously ) it takes too long.

I am thinking of going back to an old idea I had - to run my mapping system by itself, and use sockets or some other form of local communication to network the data between Sublime Text and the mapping software.

I was anyway planning on ripping out a lot of the Sublime Text stuff I wrote into the mapping solution by returning a table, instead of a completed output. The table would basically contain the original line of code, the numerical line, the start / end characters for the line, and what the mapper provides… Such as category the data will be entered into, if any, the typed data - ie: function / class name, arguments list, and more.

The plugin would be more encapsulated and would take the table and turn it into the output which could either be output into sublime text ( its own window, or in a group on the left / right, etc… in a tab as it currently is ) or into the mapping program. If I output the data into my mapping program window, there would be more freedom, but it would take longer as I’d have to code everything for coloring, etc… so I’ll likely avoid that for now.

The plugin would also read additional information from the table for promised features such as on_hover ( which will be 0* per entry ), and more… So each argument can have its own popup for live-wiki style information. I’ve already done tests and all of this is viable and means less can be displayed if the user wants it, further compacting the output panel…

But, it also means it can be ported from one IDE to another, much easier. The mapper as a standalone ‘unit’ means the plugin would handle everything pertaining to sublime text…

This is how I wanted to create it in the first place but I already had something which worked coded by me, as a mod / mapper to an existing plugin - and very little needed to be coded to isolate it to a new plugin…

I just wanted to write to let everyone know that I am still working on my system - and that it does work, and can be used as is but it causes some lockups.

I have tried so many solutions for ‘async’ and it just ends up causing the system to take a very long time to map a file. I don’t get it. I have done a lot of work since the previous public build, but I have decided to revert for the time being.

Does anyone have a solution without me having to essentially rebuild from the ground up? I have other projects I’d like to start on, but I created this to help me with efficiency and the problem of locking up the UI for seconds is too big for me to leave alone as precious seconds being ‘cut off’ from a project while a file is mapped is too long, and minutes being without a map of the file is also a waste…

At this point, I’d be willing to offer a reward - contact me on here or Bitbucket to discuss.

Trying to get my plugin running multi-threaded to prevent UI Lockups