Sublime Forum

How does os.environ["path"] work on ST3/windows?

#1

I’d like to understand how `os.environ[‘path’] works on ST3/windows…

Why? Few hours ago I was experiencing a mismatch between the os.environ[“path”] result provided by Sublime’s python process and the info from my terminal (ie: echo $PATH$) and I haven’t been able to figure out why.

At first I assume such a mismatch could be produced by my .sublime-workspace, .session, .caches, index… But after deleting all of them and spawning a new ST process there was still a mismatch.

Finally I decided to restart the PC and when I spawned again St os.environ[“path”] would provide the updated information!

So… could you please explain how does os.environ[“path”] work in ST3/windows? Is this information coming from the registry or somewhere else? What happens if I use setx as an admin to modify PATH, would it be persisted as well in St or I’d still need to restart my pc?

Hopefully you can bring some light to this one… reason I’m asking this is because I’ve wasted quite a lot of time trying to solve some related issue to this and I would like to be faster next time I run into similar problems… :confused:

0 Likes

#2

Across all operating systems, the environment is something that’s set up on a process by process basis; when a process spawns, it gets a copy of the environment of it’s parent, and that environment then becomes distinct (due to it being a copy). This is an important distinction because otherwise a process modifying its environment could conceivably break other software by changing something that it relies on.

If you notice that the environment (in your case the PATH) is different between two different processes, the reason for that is related to one of two things:

  1. The environment in one process was modified after the second one was started; since they’re not connected at all, only the one with the modification sees the change.

  2. The way a parent process sets the environment as it spawns the child can change depending on things like the OS and circumstance. For example:

    • Your shell (e.g. bash) sources different rc files to set itself up depending on how you invoke it; this is shell specific. For example if the shell is spawned as a normal shell instead of a login shell, or if it’s not interactive (or if it is).

    • On MacOS, programs that are started from the dock explicitly get a different environment set than you see in the terminal because the launch daemon doesn’t read any shell configuration files; that makes sure that no matter how you set your shell rc files, applications always work the same for everyone.

    • On Linux, the window manager that you’re using is the parent of any application that you start via it’s menus, dock items, etc. So whatever the window manager thinks the environment is, that’s what your child process gets. The environment is set by the parent and remains, so changing the environment without restarting the window manager results in the window manager maintaining the same environment until you restart it.

Based on your description, my guess would be that last one there, except that you mentioned Windows, in which case restarting Sublime should have updated the path for you.

Certainly if you install software that modifies the PATH, anything that’s already running is not going to see the PATH modification (or any other environment modification) until you restart it.

If you’re on Linux and you install something and then modify the PATH, running applications won’t see the path, but when you open a Terminal, it sources your rc files and gets the PATH you expect. However the Window Manager is still running with the original environment, and continues to do so.

Thus restarting Sublime has no effect because it’s still inheriting the same environment the window manager had the first time, but restarting the computer restarts the window manager, which picks up the brand new environment, and everything works as expected.

On Windows, just restarting a process is enough (so far as I have ever seen) for it to pick up the new environment.

On MacOS, Sublime starts a shell in the background to collect the environment to get around the issue of MacOS always using a constrained environment, so that build systems and the like will work as expected on that platform.

For full completeness here (as if this wasn’t long enough already), os.environ is part of Python and Sublime just accesses it as any Python program would.

As documented in the link above, it represents the environment at the time the module was first imported unless modifications are made to it directly, which tracks with the notion mentioned above that all processes have a separate environment.

1 Like

ST4 updating os.environ in a plugin is not working
#3

@OdatNurd First of all, thanks for such an elaborated answer, I think you’ve provided really good content/foundations to make a good thread out of my simple/diffuse question.

I’d like to clarify all you’ve explained so far was clear to me in the first place when asked my question… which is the main reason why I’ve got confused hehe… because under those assumption something wasn’t making any sense.

Specially by considering these 2 facts:

While I was testing I was keeping in mind both and i couldn’t explain what was going on.

On the other hand you also say:

Thus restarting Sublime has no effect because it’s still inheriting the same environment the window manager had the first time, but restarting the computer restarts the window manager, which picks up the brand new environment, and everything works as expected.

And this is actually quite an interesting fact I didn’t pay much attention to… I should have paid more attention to it… For instance, there is this tool called process-explorer where you can inspect easily all processes’s environment’s… I should have checked the ST’s parent process in the first place before assuming the environment had been modified by sublime’s process itself (.caches, .workspace, .session or any other internal data), that way maybe my doubt would have been clarified at that time.

I think I’ll experimenting a bit more about this interesting topic… one of the tests I’ll be doing will be checking how setx affects to both explorer.exe process & windows registry. In addition I think I’ll be read the os.environ’s source code to fully understand what’s going on.

as if this wasn’t long enough already

Welll… I must to say I love these type of elaborated answers, I think they’re really good content with relevant info so tyvm :slight_smile:

0 Likes

#4

@OdatNurd I think I’ve understood what was the reason of the mismatch I was facing few hours ago.

Here’s the thing, there may exist several explorer.exe processes (some with no parent and others with) running… When you use setx you’re affecting the explorer.exe process that has no parent directly (desktop window) but you won’t affect other explorer.exe processes that have parents (ie: the ones spawned by scvhost).

Thing is, at that point I was using setx to tweak PATH but I was spawning SublimeText from a different explorer.exe process… So all my confusion was because I was assuming ST was being spawned by the process I’d modified using setx… that was a wrong assumption I should have checked properly :slight_smile:

Anyway, to make things short & simple… the moral of the story here is next time something like this happens the best way to proceed is to analize carefully the process tree with a tool such as process explorer and compare the inherited environment between processes… that way you’ll realize very fast what happened.

Leaving this related code [https://github.com/python/cpython/blob/master/Lib/os.py#L735-L766] as a reference, although i still don’t understand very well some bits of it… specially this line, mmm

0 Likes

#5

Ahh that’s good to know; I’ve never used setx before (in fact until you mentioned it and I googled it, I didn’t even know it existed). Generally if I need to set an environment variable I just go into the system dialog for it and restart whatever needs it.

Good to have in the back pocket as extra information though, particularly for cases like this.

0 Likes

#6

There is a lot of really good sys-admin tools out there for windows (ie: sysinternals et at)… I knew about setx but never used it too much till today… Reason wny started using it was because managing my USER/SYSTEM PATH with the good old system dialog wasn’t “convenient” anymore to properly manage all my dozens of paths (just written a python script to keep this information nicely sorted :P).

Anyway, while setx is quite handy I’m still not sure is the best choice here… take a look to this one http://www.p-nand-q.com/download/gtools/pathed.html … Come up with that link after facing this issue https://superuser.com/questions/387619/overcoming-the-1024-character-limit-with-setx . Anyway, I’m going a bit offtopic already and I think this thread has been already solved pretty nicely. As usual, tyvm :wink:

Ns. Or even better… https://www.rapidee.com/en/download

0 Likes