Sublime Forum

Dependencies in Package Control 3

#1

Hi,

can someone explain dependencies in Package Control 3? I know about this page packagecontrol.io/docs/dependencies but I still don’t really get it.

One of the main hurdles when I work on a package is to import third party python libraries or libraries that are not bundled with ST3. Some just work but it’s a mess in other cases because I have to change a lot of the import commands to something like ‘import mypackage.crypto’. Can dependencies be used to make third party python libraries available for other package developers? Let’s take crypto, sqlite3 or paramiko as an example. Is it possible to have these libraries in a separate package other packages can depend on? How would I go about doing that? Is there a list of shared libraries others already implemented? I tried to search package control but wasn’t able to find such a list.

I do have some programming experience but never received a proper education in computer science. So anything related to compiling, dependencies etc still troubles me.

Thanks!

1 Like
How to add Python Modules to Sublime Text 3
Importing paramiko (& other libraries)
Need some help with async
#2

Yes. See github.com/packagecontrol/requests for an example of an “external” pure-Python dependency that I hooked over from pypi.

Yes, dependencies are installed in their own “Packages” (which is imo a terrible design choice, but what Will did since he had the mechanics from normal Packages already in place (see github.com/wbond/package_control/issues/800).
It basically works as follows:

  1. Each dependency gets an implicit loader.py which will look for a few pre-defined folders that are added to sys.path for importability, depending on OS and ST version.
  2. Package Control takes care of providing a package named “0_package_control_loader” which will be loaded directly after the Default package so that it is available to all other Packages.
  3. Since the dependencies have been added to sys.path, every Package can use import dependency in their source code.

For binary dependencies you mainly want to use ST, OS and arch-specific binaries that you pre-built (e.g. the [ssl dependency](ssl dependency)) and for pure-python you mostly want to use either one code base (all) or one for ST3 and one for ST2 due to the different Python versions (3 and 2). The github orginization where the requests package resides in includes a few examples and the docs for dependencies on the packagecontrol.io site also lists some binary dependency examples.

Edit: Forgot the last part.

No, dependencies are not searchable using package control or using the website. All currently available dependencies are here though, even though anyone may define some of their own in a custom repository.json without us noticing.

3 Likes
#3

Hi!

Is there any chance the info you shared here could be included on the https://packagecontrol.io/docs/dependencies page?

It would have saved me ~30 minutes.

Cheers

0 Likes
#4

I’m not sure I understand. The “info” you referenced is included in the page under the “Publishing” section.

0 Likes
#5

So this is basically the unmodified PyPI module following the directory structure described in the docs?

For a ST package, I would like to include a PyPI module “dependency A”, which requires another module “dependency B”. I guess I would create two separate dependency packages, but does this require any changes on the original modules or should I simply import both modules in my plugin. (I’m fairly unexperienced with Python and I tried something similar before, that’s why I’m asking.) Also, does the load order have an effect on this? Its purpose seems unmentioned in the docs.

0 Likes
#6

Yes.

Yes, you should create two separate dependency packages for each.

Yes, you can then import both modules freely in your plugin.

No, load_order does not have any effect on this case. It is only relevant if you provide custom loader code in a loader.py. The default loader only adds paths to sys.path.

0 Likes
#7

Is there a way to test dependencies before adding them to the repository? I see two Package Control commands for dependencies, but they are both undocumented.

0 Likes
#8

There is. “Install Local Dependency” installs a dependency from … your local pc instead of downloading it.

I never used it and am not sure how it works though. @wbond should probably add some documentation about the two commands to the site.

You can also always add a custom and local channel/repository with the test data for a near production level test.

0 Likes
#10

When a dependency is updated, how can users update it locally? Are dependency updates handled the same ways as packages?

It might be a coincidence, but I tried running Package Control: Upgrade package 12 hours after I released a new version of a dependency. This didn’t do anything. Then I tried Package Control: Satisfy Dependencies and it did the job. Not sure if the standard user ever runs that command, that’s why I’m asking if that’s the only way to update dependencies.

0 Likes
#11

Satisfy dependencies is the only way i know to force a dependency update. I never see PC update dependencies on its own unless forced… maybe I’m impatient.

0 Likes
#12

New versions of dependencies are installed when a package requiring it is updated or installed. The current implementation isn’t really intended for them to be things that end users think about, hence no “upgrade dependency” or “install dependency” commands. This may mean you need to make a new package release if your package needs an updated version of a dependency. This all kind of ties back into the fact that dependencies were mostly intended for things that were awkward to include in a package, e.g. binaries and shared libraries. Most package developers have a different idea based on the name. Perhaps in the future I will rename them to something like “assets” or some other term that doesn’t confuse people so much.

In hind sight, I probably would never have wasted all of the time on them seeing how most users have a different use case than myself.

2 Likes
#13

That’s good enough for me, thanks for clarifying

0 Likes
#14

Hmm. I swear I’ve had people update plugins but still not have the dependencies update. I usually have had to tell them to run satisfy dependencies.

0 Likes
#15

I have just tested this on the outdated computer at work, and you’re right!

0 Likes
#16

I would consider that a bug then. Feel free to open an issue so I can get that fixed.

1 Like
#17

I’d like to come back to this. What kind of dependencies can be submitted? Can I create a new dependency package and submit it whenever I need to use it? Does a dependency need to be of interest of a wider audience or are all dependencies accepted.

I have submitted dependencies before, but in this case I wanted to ask, because this new one is a bit of an oddball that I doubt anybody else would ever need.

0 Likes
#18

I don’t think there is a restriction on what can be a dependency. Often it is simply 3rd party python modules, but it could easily be something you use in more than one plugin. For instance: I created mdpopups for all my plugins; I don’t know or care if anyone else uses it, but I don’t want to support the same dependency in X different packages, so I released it as dependency. I did the same with another module called backrefs. If it is your own dependency that only one package will use, and you can’t see anyone else using it either, then in makes more sense to ship it with the plugin instead of offering it as a dependency.

4 Likes
#19

Yeah, this is an existing 3rd party module.

Thanks for clearing that up!

0 Likes