Sublime Forum

**pixelami** · January 13, 2016, 7:19am

Thanks for the responses,

@Nick,
Yes I tried deleting the *.pyc files, and interestingly they did not seem to get recreated.
I then opened a terminal in the module folder and explicitly ran ‘python myPluginModule.py’ which did recreate the .pyc , but Sublime Text did not seem to be using this new version.

I suspect once the modules are loaded into ST2’s embedded python interpreter they are just sitting in RAM.

I did some more Googling and found this
docs.python.org/library/function … oad#reload
stackoverflow.com/questions/4375 … hon-module
I’m thinking that maybe during development, the main MyPlugin.py could use reload inside an on_activated event callback method.
I’ll give it a try and update this post.

**wbond** · January 13, 2016, 7:19am

If you look at SVN.py in the SVN package, or SFTP.py in the SFTP package, I use reloading so that I can actively develop in Sublime without having to constantly restart. Depending on the complexity of your dependencies, you may have to do a little work to make sure they are all reloaded in the right order.

**pixelami** · January 13, 2016, 7:20am

Thanks wbond !

That was just what I needed.
I had almost got there with something along those lines but there were 2 vital bits that I was missing.

if mod.startswith(“hxutil”) and sys.modules[mod] != None:

and I think reloading the modules in a specific order has helped too.

Many thanks.

**astrauka** · March 23, 2016, 7:12am

Implemented reloader https://github.com/astrauka/TestRSpec/blob/master/plugin_helpers/reloader.py .

import sys, imp, os, sublime, sublime_plugin
from rspec.rspec_print import rspec_print

# Dependecy reloader
# The original idea is borrowed from
# https://github.com/wbond/sublime_package_control/blob/master/package_control/reloader.py

rspec_print('Reloading rspec modules')
CODE_DIRS = [
  'plugin_helpers',
  'rspec',
]
PYTHON_FILE_EXT = '.py'

def _reload(dir, file):
  (name, extension) = os.path.splitext(file)
  if not extension == PYTHON_FILE_EXT: return

  dirs = '.'.join(filter(None, os.path.split(dir)))
  module = sys.modules.get('.'.join([dirs, name]))
  if not module: return

  if 'on_module_reload' in module.__dict__:
    module.on_module_reload()
  imp.reload(module)

for _ in range(2): # double reload required to update dependencies
  for directory in CODE_DIRS:
    for dir, _, files in os.walk(directory):
      for file in files:
        _reload(dir, file)

class ReloadPlugin(sublime_plugin.EventListener):
  PLUGIN_RELOAD_TIME_MS = 200

  def on_post_save(self, view):
    plugin_python_file = sys._current_frames().values()[0].f_back.f_globals['__file__']
    file_name = view.file_name()
    if not os.path.dirname(plugin_python_file) in file_name: return
    if file_name == plugin_python_file: return

    original_file_name = view.file_name()

    def _open_original_file():
      view.window().open_file(original_file_name)

    plugin_view = view.window().open_file(plugin_python_file)
    print("save", plugin_view.file_name())
    plugin_view.run_command("save")
    sublime.set_timeout_async(_open_original_file, self.PLUGIN_RELOAD_TIME_MS)

Imported from https://github.com/astrauka/TestRSpec/blob/master/sublime_rspec.py

import sys, os.path, imp, sublime_plugin

BASE_PATH = os.path.abspath(os.path.dirname(__file__))
sys.path += [BASE_PATH]

# reload plugin files on change
if 'plugin_helpers.reloader' in sys.modules:
  imp.reload(sys.modules['plugin_helpers.reloader'])
import plugin_helpers.reloader

It reloads nested plugin source files on any plugin file save.
In order for plugin reloading to work I save the main plugin source file and wait for Sublime to reload the plugin.
Current implementation opens plugin source file, saves it, waits 200ms and opens back the file where save was triggered. This creates a flickering experience.

Does anybody know a better way for reloading?
Is there a way to trigger Sublime plugin reload in background to avoid moving cursor from currently edited file?

**astrauka** · March 19, 2016, 6:55pm

@wbond have you solved the autoreloading problem? ^^

**fico** · March 24, 2016, 10:58am

@wbond

I tried implementing the method from SFTP, but couldn’t get it to work.
( I got the array to populate correctly, but not the reload )

I also tried simplifying it & performing the reloads manually:

from imp import reload
from sys import modules
reload( modules[ "_UTILS" ] )
reload( modules[ "_UTILS.__ALL_UTILS__" ] )
reload( modules[ "_UTILS.Comments" ] )

_UTILS.__ALL_UTILS__.load ( globals() )

System: Windows 10, ST 3103

I’m working on creating a dependency template for plugins that automatically loads all sub-modules from _UTILS.

It initializes a bunch of useful stuff into globals, such as:

@ each view change: syntax, filepath, comment prefix & suffix, etc.
the first level of variables @ sublime-settings
a bunch of functions to eliminate repetetive tasks like extracting region data

So far, it works great on the first load, but I would like to implement the reload method to speed up development.

Here is the __ALL_UTILS__ module:
#@ Gist

**wbond** · March 24, 2016, 11:13am

The key thing about reloading nested submodules in Python is that they have to be reloaded in reverse order. So first reload the files with no other project-level dependencies, then reload for all files that have had all dependencies reloaded until all files have been reloaded.

With that method you don’t have to reload twice, but you do have to know your dependency load order. There is an example at https://github.com/wbond/package_control/blob/master/1_reloader.py#L61-L214, but it is complicated by an issue with the Sublime Text .sublime-package loader. Hopefully I will get that fixed in one of the next few releases.

I’m not sure that Sublime Text is ever going to try and handle reloading packages completely on save, but I won’t rule it out.

**fico** · March 24, 2016, 2:16pm

@wbond

That code is kind of over my head .

It seems like it’s maybe more complex than what I need to implement.

Pretty much, the only scenario I’ll be working with is a single dependency @ Packages/_UTILS
The entire contents of the _UTILS directory is:

[ Comments.py, Edit.py, Indentation.py, Region.py, Sort.py, Verify.py, Variables.py ]

( Discarding the __ALL_UTILS__ module I mentioned before. I don’t think I would need it anymore using your method? )

I tried going back to the first method I attempted, like in your SFTP plugin.

Here is what I have:

from imp import reload
from sys import modules

reload_mods = []

for mod in modules:
	if mod[0:6] == "_UTILS" \
	and modules[mod] != None:
		print ( mod )
		reload_mods.append(mod)

mods_load_order = [
	"_UTILS",
	"_UTILS.Comments",
	"_UTILS.Edit",
	"_UTILS.Indentation",
	"_UTILS.Region",
	"_UTILS.Sort",
	"_UTILS.Verify",
	"_UTILS.Variables",
]

for mod in mods_load_order:
	if mod in reload_mods:
		print ( mod )
		reload(modules[mod])

Comments.test()

When I save, the print ( mod ) @ both for mod in modules & for mod in mods_load_order is returning:

_UTILS
_UTILS.Comments
_UTILS.Edit
_UTILS.Indentation
_UTILS.Region
_UTILS.Sort
_UTILS.Verify

So I’m guessing the values are valid as far as modules is concerned.
( although _UTILS.Variables isn’t being picked up for some reason… )

This is the traceback I’m getting @ Comments.test()
( I also tried _UTILS.Comments.test() )

Traceback (most recent call last):

  File "C:\Program Files\Sublime Text 3\sublime_plugin.py", line 76, in reload_plugin
    m = imp.reload(m)
  File "./imp.py", line 276, in reload
  File "<frozen importlib._bootstrap>", line 584, in _check_name_wrapper
  File "<frozen importlib._bootstrap>", line 1022, in load_module
  File "<frozen importlib._bootstrap>", line 1003, in load_module
  File "<frozen importlib._bootstrap>", line 560, in module_for_loader_wrapper
  File "<frozen importlib._bootstrap>", line 868, in _load_module
  File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
  File "C:\Users\Fico\AppData\Roaming\Sublime Text 3\Packages\Parkour_v1\Parkour_v1.py", line 48, in <module>
    Comments.test()
    
NameError: name 'Comments' is not defined

**wbond** · March 24, 2016, 1:16pm

Is your mods_load_order really in dependency order? Does _UTILS really have no dependencies on any of the submodules? To me it looks like you just put them in alphabetical order.

For instance, if _UTILS.Comments imports from . import Edit you’ll need _UTILS.Edit in the list before _UTILS.Comments.

**fico** · March 24, 2016, 2:19pm

The only submodule that uses other submodules from _UTILS is Variables, which for some reason is not showing up in the mod[0:6] == "_UTILS" filtered modules.

I just ran a full print of modules and it is listed, but just as Variables without the _UTILS prefix.

That issue is odd, but the test function I’m running from Comments should work since it doesn’t require a single module. Variables isn’t referenced at all during the test from my last post, it’s only called later once the TextCommand is executed.

###Note:

_UTILS is just a directory, there is no _UTILS.py

Does that affect how I would implement your code?

###Edit 1:

I just commented out import Comments, Edit @ Variables, and it now shows up in the mod[0:6] == "_UTILS" filter.

###Edit 2:

I changed

import Comments, Edit
to
from _UTILS import Comments, Edit

and Variables is now included with the filtered modules.

The primary issue with the traceback still persists.

**kingkeith** · March 24, 2016, 2:01pm

just an FYI, but there is a str.startswith function in Python, in case that would make your mod[0:6] == "_UTILS" check “cleaner”

**wbond** · March 24, 2016, 2:21pm

Can you post the contents of Parkour_v1.py somewhere?

**fico** · April 10, 2016, 2:00am

I just posted all of the code & a short explanation. I blocked off two sections @ Parkour_v1.py so you can easily toggle between them. One is your SFTP code, and the other is my original implementation.

#@ GitHub

**kingkeith** · March 24, 2016, 3:26pm

ah I wondered how you did your fancy comments in your Python code

**fico** · March 24, 2016, 3:29pm

Yep… about 5k lines of hot-mess, “I-have-no-idea-what-I’m-doing” code. But it works! And it’ll be a lot cleaner this time around.

**wbond** · March 24, 2016, 5:28pm

My guess is you are probably dealing with some arcane aspect of imports. At https://github.com/Enteleform/ST_Parkour/blob/master/Parkour_v1/Parkour_v1.py#L48 you haven’t explicitly imported that name. I presume your load() function is doing that. Most likely you have some sort of issue related to that.

This is all beyond my ability to dig in right now. Personally, I would recommend using simpler, explicit, relative imports.

**fico** · April 10, 2016, 2:01am

That’s what I was attempting with Parkour_v1.py#L32.
I also tried _UTILS.Comments.test() @ line 48.

I wouldn’t mind doing so to speed up development, and that is what I was attempting with your code. I guess I just don’t understand it enough to figure out what is missing in order to get it working. In my implementation of your code, I was completely discarding the use of __ALL_UTILS__

Thanks for your help
I’ll keep trying to understand what’s going on & see if I can get it working.

**fico** · March 25, 2016, 6:12am

In case anyone else is having issues implementing this & needs a temporary workaround, I wrote:

a plugin command that executes window.run_command("exit"). Ctrl + Shift + Alt + Win + F15
an AutoHotkey script that sends the above key-combo and then reloads SublimeText. Ctrl + Shift + Alt + Win + S

The windows reload instantly, but there’s about a 5-second delay for ST to be fully functional since ALL plugins & modules have to reload.
( maybe more or less depending on the amount of plugins you have installed )

@ Plugin.py

	def EXIT ( window ):
		window.run_command ( "exit" )

@ Plugin/Default.sublime-keymap

	{ "keys": ["ctrl+shift+alt+super+f15"], "command": "my_plugin", "args": { "MODE": "EXIT" }  },

@ restart_SublimeText.ahk

#Persistent
#SingleInstance Force

#IfWinActive ahk_class PX_WINDOW_CLASS

	^+!#s::

		Send, ^+!#{F15}

		Sleep, 200
		Run, "C:\Program Files\Sublime Text 3\sublime_text.exe"

	Return

**fico** · March 25, 2016, 11:55am

@wbond

I managed to put a solution together after looking further into imp, os, & sys.

The script uses Globals() as a parameter and dynamically loads modules to the plugin’s Globals via pluginGlobals[ moduleName ] = imp.load_module (...)

Implementation is concise, requiring only 3 lines in the plugin.

Also, I believe that I solved the issue you mentioned of having to reload modules in a particular order by looping over the load process twice.
( This hasn’t been extensively tested, but I’m assuming it should work. I don’t really have anything to test it with - there’s only one module with dependencies in Parkour v1 and it’s working fine. )

Loading modules twice would not be ideal for something like Package Control, where it would have a significant impact on performance due to the amount of modules, but for most plugins it shouldn’t be an issue.

Please let me know if you see any potential issues or circumstances where the script might not work.

See this thread for more info:

[MODULE] module_loader : Reload Modules & Dependencies WITHOUT Restarting Sublime Text

**astrauka** · May 2, 2016, 7:31am

@fico have you looked at [MODULE] module_loader : Reload Modules & Dependencies WITHOUT Restarting Sublime Text ?

Auto reloading of python module files used by plugin

[MODULE] module_loader : Reload Modules & Dependencies WITHOUT Restarting Sublime Text