How to change "Data\Lib\python3.3" to become a different folder?

[BPL]

Maybe you guys got some fresh ideas about this very interesting question… Little bit of background first, right now each time I want to use python dependencies my workflow goes like this:

• I try to see if the dependencies I want to use are available to be installed via dependencies.json… if they are I just stop on this step and I celebrate \o/

• If the code is not ready to be consumed via dependencies.json I’ve got a python 3.3.5 virtualenv living in Data\Lib\venv… In that virtualenv I’ll install packages such as:

   a) local packages by using `pip install -e .`
   b) pypi packages by using `pip install`
   c) git packages by using `pip install git+...`
  

• Once I’ve got these packages installed, finally I’ll just copy/paste them manually to Data\Lib\python3.3 folder so they will be available to be used with sublime text builtin python interpreter

• If I need to modify code of these dependencies I’ll modify it first directly on the Data\Lib\python3.3 code and once happy I’ll update the upstreams (should be the other way around).

So… as you can see, this workflow is really slow and error prone so today I was wondering if with the current SublimeText build 3176 I could improve it somehow. Few months ago I’d already checked than creating softlinks (windows softlinks) pointing out to the virtualenv site-packages wasn’t possible. Mainly because sublime text would ignore these shortcuts and it’d create the real Data\Lib\python3.3 folder if it didn’t exist… guess this is hardcoded in ST source code… —> It’d be great if we the users could specify our own “python3.3” additional path somehow <—

Anyway, let’s assume there is this little limitation and live with it… even so, can you think of any “hack/trick” to make ST read packages from the site-packages virtualenv directly instead the rigid hardcoded addtional Data\Lib\python3.3 path? If this was possible the workflow would be improved x100!!!

Said otherwise, if this could be tweaked somehow basically you’d just be able to install stuff into your virtualenv and have the whole python3.3 python ecosystem cooked for SublimeText to be used… wouldn’t that be great?

Only way i can think of right now is by trying to tweak PYTHONPATH SublimeText python builtin interpreter somehow before any package is loaded…

Another way I can think of is by using an hex editor and looking for “python3.3” references and change those to point out to my Data\Lib\python3.3\Lib\site-packages… but I’ll hold back my horses to see if someone has a better idea here :'D

So… thoughts? ideas?

Thanks in advance

[BPL]

One hacky way to go around this would be creating a hard link, which on windows is also known as junctions. More info about it here.

Here’s a little guide for those who’re interested on opening ST to the outside world:

• Install python 3.3.x (ideally the version & arch matching sublime’s)
• cd c:\your\path\to\sublimetext3\Data\Lib
• virtualenv --python=c:\python3.3.5\python.exe py335
• mklink /J python3.3 py335\Lib\site-packages

And that’s it, thanks to this little trick you’ve just opened ST3 to the outside world (pypi, local, vcs packages…). On the other hand, you need to be aware most of the cooler pypi packages outside there won’t provide python3.3 support , but even so… this is great.

/mode suggestions on/

That said, it’d be great if ST coders would allow a way to customize the builtin python pythonpath through an env.var, settings, .pth files… Providing a simple hardcoded Data\Lib\python3.3 directory as a way to use additional packages is very limited and you end up adding these type of hardlinks or hacking around… we don’t want to have our SublimeText plugin devs living in a little bubble, do we?

Also… if you guys just provided access to moder python (ie: >=3.6) ST would reach the next level when it comes to better plugins.

/mode suggestions off/

PS: Credits for the hard link suggestions go to @OdatNurd , which recommended me to do so on ##sublimetext freenode channel… \o

[wbond]

Hooking the Sublime Text python env to an outside set of packages is probably just a little too far away from what we want to support, from a support perspective. Many, many things can go wrong, or affect performance. If you are enough of a Python user to figure out how to hook it up, then you are probably sufficiently capable of working on debugging possible downsides.

The original intention of that folder is for Package Control to install dependencies into. I want to get it in place before we started using it. So far PC hasn’t started using it, but be forewarned.

[BPL]

Will, I totally understand your point from a support perspective and makes total sense, it really does… But I want you to think about this as a simple way to make easier the life of “Plugin/Package developers” and of course, I understand that with great powers comes great responsabilities so I wouldn’t ask for support if something goes wrong here

But yeah, your point is totally valid… on the other hand, the cardinal rule should be Packages uploaded to ST package repository should be self-contained (ie: using dependencies.json properly) and if they work or not on end-users side should be the responsability of package developers not core developers. But again, think than being able to use the whole python ecosystem (subset of pypi repo as we’re attached to the old good 3.3) is just good news when it comes to make better SublimeText plugins, right now we “limited” to use existing dependencies or upload our own ones, which is not bad but it’s not ideal… . For instance, look how much fun I’m having right now

Nah, I just kidding, I think this is great… at least to me, no more copy/paste after installing packages on my virtualenvs. Let me try to play with opengl on sublime for a little bit now, be right back…

About python>=3.6 support, guys… pleeeeease!

[BPL]

@wbond After little bit of toying around over this I must to say the experience has been great so far:

• When installing pypi packages it works great (only limitation here is many pypi packages won’t support 3.3.x)
• When installing packages locally, pip install my_local_package works great
• When installing packages by doing pip install -e . it won’t work that great. Why? it seems .egg-links won’t be supported by sublime’s builtin python, even if they work ok when tested directly on the virtualenv. A possible workaround over this is just replacing the .egg-links with hard links… after that, it works great
• About performance… it’s too soon to report anything about it, when I’ve got installed few dozens of packages and loaded on Sublime I’ll profile some of my packages and comment about it

So, what can I say? Overall I feel full of power with this new workflow atm.

The original intention of that folder is for Package Control to install dependencies into. I want to get it in place before we started using it. So far PC hasn’t started using it, but be forewarned.

I see… well, everything that improves the our dearest Sublime and makes our life as package developers & users easier will be well received of course. But please, answer me some little questions here, what do you think it’s wrong with:

• Bringing the whole python package ecosystem in a nice way to Sublime (not “claiming” the way i’m doing it right now is the proper one)
• Packages not having to rely on dirty tricks like bundling external dependencies like this (look the dependencies) or tweaking sys.path like this
• Being able to use local code on sublime packages before creating a nice distributable package and upload it to ST package repo.
• Being able to use modern python (at least >3.6)

Of course, let’s be clear here. Your project, your rules, your pace, your roadmap and the most important… you’ll know exactly what you want or don’t to support.

But I’d like to know at least whether the above bulletpoints only make sense to me… IMHO I believe they’d take ST to the next level and believe me, I already consider this editor has reached the highest bar (personally)… in my book is actually the best tool a coder could dream of. So, just to be clear, not complaining or anything but just trying to understand a little bit more of your future plans regarding to this

[FichteFoll]

Pip accepts an install dir parameter for the install command. You can use that to make it install packages into st’s lib folder from within your 3.3 venv.

[BPL]

@FichteFoll Wow! That’s serious cool stuff, thanks a bunch to point it out. I wasn’t aware about –target, no more hacky stuff. Even if @wbond will forbid us kids eventually to toy around with PC Data\Lib\python3.3 directory I’ll still be able to use pip targetting Data\Packages, thanks a bunch, you’ve saved the day!

I’d give 2 hearts to your awesome hint unfortunately I can’t