Sublime Forum

Modules + Dependencies : Best Practices?

#1

@wbond   @FichteFoll

 
I have a plugin framework that I’ve been developing to speed up development.

For example, there’s an Input module with a QuickPanel class that easily implements things like nested hierarchy, groups of commands, multiple selections, etc.

I’m trying to figure out the best way to implement the framework with my plugins.
 


 
The way I see it; one option would be to include a full set of the modules with each plugin ( currently around 700kb ) so that each plugin is always accurate to the framework, and upgrades could be handled manually.

The other way would be to release it as a dependency, but the thing I’m worried about there is that I’m still learning as I do this so if I need to refactor things, it could potentially break functionality in any plugins that implement it.  Is it possible to release a versioned dependency?

1 Like

[MODULE] module_loader : Reload Modules & Dependencies WITHOUT Restarting Sublime Text
How to add Python Modules to Sublime Text 3
#2

There is currently no way to have versioned dependencies at this time.

If I have something that multiple of my plugins use, I usually opt to make it a dependency. I’ve done the “include the same module in multiple plugins”. It’s annoying every time you add something you want all the plugins to pick up. Expose the version of your dependency so the plugin layer can be aware of it and make decisions based on it. That way you can update your plugins to have alternate logic for the soon to be released version, and then you can release the new dependency. Then when you release your new dependency, all your plugins handle it gracefully.

There is no guarantee that anyone other than you will use the dependency, but if it is likely, you just have to be a bit more careful when reworking things, like have a window where you deprecate features with a warning to give other developers time to switch over to the new changes. They can use the old way through the deprecation period, and have time to switch over to the new way, but when that period ends, you remove the old feature.

5 Likes

#4

I wrote a bit about this here: https://github.com/wbond/package_control/issues/1070

I suppose your only truly version-specific usage of a dependency would be to check the dependency’s version (module.__version__, assuming it’s defined) in your plugin and then check from a plugin standpoint whether the version is still supported. This is very similar to checking sublime.version(). Following http://semver.org for this version would probably help.

Other than that, stick to one of the options I outlined in the referenced issue.

4 Likes

#5

The real issue with dependencies is that Python is a global namespace. The only way to make versioned dependencies would be to have distinct module/dependency names whenever an API breaks.

Dependencies were designed with distributing binaries and shared libraries that rarely change. This would allow packages to be installed as a .sublime-package. Additionally, if you fixed a bug in your 2,000 lines of Python, you wouldn’t need to re-ship a 5MB shared library to every user. Another nice side effect would be that once someone figured out how to compile the sqilte3 module for every platform and version of ST, not every package developer would need to include it in their package.

Once a feature exists, people figure out ways to use it to their advantage. Personally, I would recommend against distributing some centralized framework as a dependency. It just seems like a recipe for pain. Instead, I would include the files as relative imports in each package you use. That said, 700kb of Python sounds like a lot isn’t going to be used in each package.

4 Likes

#6

 
I might have a potential workaround that’s somewhere between 2 & 3.

I posted my module loader script a while back as a workaround to reloading modules during devlopment.

Since then, I’ve implemented a few updates, including “self” detection for all modules within the framework, along with separate loaders for individual modules & entire directories.
 


 
Here’s the import code for each implementation:

#▒▒▒  Load Specific Modules ▒▒▒#
from enteleform_utils import module_file_loader ; moduleFiles = []
moduleFiles.append( "__packages_relative_directory__/__module_name__.py" )
module_file_loader.load( moduleFiles, globals() )

#▒▒▒  Load All Modules Within A Directory ▒▒▒#
from enteleform_utils import module_directory_loader ; moduleDirectories = []
moduleDirectories.append( "__packages_relative_directory__" )
module_directory_loader.load( moduleDirectories, globals() )

 

The workaround I have in mind is to implement a version loader:

import module_file_loader ; moduleFiles = []
moduleFiles.append( "module_1.py" )
moduleFiles.append( "ModuleGroup/module_2.py" )
module_file_loader.load_version( "MyDependency", "1-04", moduleFiles, globals() )

so that the loaders would load the appropriate files using the dependency & version arguments:

/Packages/MyDependency/1-04/module_1.py
/Packages/MyDependency/1-04/ModuleGroup/module_2.py

 


 
That kind of conflicts with @wbond recommendations in his post above ( maybe? ), but it somewhat meets the requirements detailed in your post & also addresses the global update sentiment that @facelessuser detailed.

1 Like

#7

Yeah, I guess I just don’t worry about these kinds of things too much when it comes to my open source Sublime plugins. As I don’t make money off any of the plugins I provide, it just isn’t worth the worry.

I think because you are developing a dependency to use with your plugins, you have complete control of all the variables. I would think updating your plugins to be prepared for a new version of your dependency wouldn’t be that big a deal.

If you are unsure of the plugin’ s final state and think it may change drastically, I would consider it still in beta, and would probably wait to have all your plugins adopt it until it is out of beta.

If other people use it, I usually just do my best. Don’t formally release it to everyone until I am satisfied with it. Deprecate old features and spam the console to encourage adopting of the new change. Sometimes some features cannot avoid breakage. I save those features for major releases and prepare with F.A.Q to help people move past the problem.

I would just hate having to keep shipping an ever growing dependency of multiple versions. Because if I ship it, I have to support it all. When I ship version YYYY, I don’t want to have to support version XXXX. Since I’m not getting paid to do this, my answer is going to be “adopt version YYYY” and your problem is solved.

But that’s just me :).

6 Likes

#8

I am probably wrong, but isn’t the package reloaded when it is disabled and enabled again?

2 Likes