Sublime Forum

[SOLVED] Recommended folder structure for plugins with .py files

#1

I have a plugin with several pieces: snippets, syntax files and tests, and soon a couple of build systems. This gets messy when the files are all in the root of the plugin folder, along with Package Control stuff, so I have put them in subfolders:

.
├── Build Systems
│   └── Check Justfile.sublime-build
├── Snippets
│   └── bash.sublime-snippet
├── Syntax
│   ├── Just.sublime-syntax
│   └── tests
│       ├── etc.

This is fine for most things, but now my Build System has become more complex than can be managed with a simple .sublime-build file. I need to define my own command, subclassing sublime_plugin.WindowCommand. This, in turn, needs to go inside a .py file. I’ve named mine check_justfile_build.py. But when I place that file inside the Build Systems folder, it is never loaded because (as I understand it) Sublime only looks in the plugin root to load .py files. But I’d really like to keep each category of resource files together.

As a workaround for this, I’ve had success with renaming the folder and putting this in a .py file in the package root:

from .BuildSystems.check_justfile_build import CheckJustfileBuildCommand

Although that doesn’t work in this particular case because “Build Systems” has a space in it. I tried the following, but it fails with No module named "Build Systems":

import importlib
importlib.import_module("Build Systems.check_justfile_build.CheckJustfileBuildCommand")

I’d be grateful for a pointer to some best practice examples of well-structured packages with multiple different categories of Sublime resources including .py files. I can’t be the first person to run into this?

0 Likes
#2

I’d create a plugins directory with all the source code and a slingle plugin.py file in root which imports API classes from `plugins directory.

Some package authors choose main.py or boot.py, but I find plugin.py best solution as it is displayed as reloading MyPlugin.plugin in ST’s console.

2 Likes
#3

Makes sense, I’ll do it that way. Thank you!

0 Likes
#4

So I tried this, and it does work on the first load of the plugin. However, when I make a change to the nested .py file and then save, the old version is still used by Sublime until I restart the app. So it’s not a very practical setup for development.

I copied this code into plugin.py (copied from MarkdownEditing), but it doesn’t seem to have helped:

import sublime

if int(sublime.version()) < 3176:
    print(__package__ + " requires ST3 3176+")
else:
    import sys

    # clear modules cache if package is reloaded (after update?)
    prefix = __package__ + "."  # don't clear the base package
    for module_name in [
        module_name
        for module_name in sys.modules
        if module_name.startswith(prefix) and module_name != __name__
    ]:
        del sys.modules[module_name]
    prefix = None

    # import all published Commands and EventListeners
    from .plugins.check_justfile_build import CheckJustfileBuildCommand

Am I doing something wrong, or is this just a limitation of the way Sublime loads packages?

0 Likes
#5

Saving / touching the plugin.py in root will re-import all sub directory modules, with the code you copied from MarkdownEditing.

An alternative would be https://packagecontrol.io/packages/AutomaticPackageReloader, which forces reloading all sub modules in correct order, but I find it overkill and dropped using it, as former mensioned step works perfect in most use cases.

Maybe one could write a plugin, which just opens and saves the main plugin.py in case a python module in sub directory is saved.

TL;TR

ST’ can’t reload modules from sub directories automatically as they are not treated as normal plugins. The only way to update them is via python’s default import mechanism.

I tend to go the lazy way by just dropping loaded modules from sys.modules to ensure the following imports import all sub modules in correct order - even if new ones were added or have been removed.

Some plugins like Package Control explicitly maintain “reload” lists and call implib.reload() on each in predefined order. That fails however if package structue changes (new/removed modules).

AutomaticPackageReloader intercepts the module importer to force reloading all sub modules recursively on demand in correct module loading/import order. It can be set-up to do so automatically, if a file is saved.

0 Likes
#7

Thanks for your insight! For posterity, I’ve written down my solution below.

I ended up using the Hooks package. I used Project > Save Project As… to create a Just.sublime-project in the repo and gave it these contents:

{
	"folders":
	[
		{
			"path": "."
		}
	],
	"settings": {
		"on_post_save_project": [
			{
				"command": "touch_plugin_py",
				"args": {},
				"scope": "window"
			}
		]
	}
}

Then I added this command to the plugin.py file in the root of the repo:

import sublime_plugin
from pathlib import Path

class TouchPluginPyCommand(sublime_plugin.WindowCommand):
    def run(self):
        variables = self.window.extract_variables()
        path = Path(variables["file"])
        plugin = Path(variables["project_path"]) / "plugin.py"

        if plugin != path and path.suffix == ".py":
            plugin.touch()

Now, whenever a .py file is saved anywhere in the folder structure of the project, the plugin.py file is touched, which causes Sublime to reload it and notice the modified dependency.

Two things to note:

  1. The command needs to be run using Python 3.8 because pathlib is new in 3.5. So I also had to add a .python-version file to the package.
  2. In order for the project settings to be reliably configured, you must ensure that you use the Project > Open Project… menu item, not the File > Open… item. Only the former will always set up your project settings correctly. Learn more in this Forum post.
0 Likes