Sublime Forum

Subprocess issue on b_3146 / windows

#1

Hi,

Something seems to have happened between b_3126 and b_3143 with regards to uses of the Subprocess module.

Calls to subprocess.check_output or subprocess.Pipe encounter issues with handles still in use when shell=True is passed-- I think that these may be used by Sublime itself.

Example calls that break:
p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=global_folder, shell=True)

from the Perforce module
The backtrace is:
File “./python3.3/subprocess.py”, line 784, in init
File “./python3.3/subprocess.py”, line 980, in _get_handles
File “./python3.3/subprocess.py”, line 1030, in _make_inheritable
OSError: [WinError 6] The handle is invalid

however, sometimes it is possible to work around this as follows:

try:
from subprocess import DEVNULL
except ImportError:
DEVNULL = os.open(os.devnull, os.O_RDWR)
ignore=subprocess.check_output([alt_python, full_path, host, username],stdin=DEVNULL, stderr=DEVNULL,shell=True)

This is new between b_3143 and b_3126, and I have no ideas why.

This is running on windows 7 using a 64 bit python 3.3 intepreter.

I’ve tried cheating by deleting the subprocess module and reloading it from the previous zip file, but so far, no luck.

0 Likes
#2

The pipes created for a subprocess.Popen aren’t attached to anything within Sublime Text.

We also didn’t make any upgrades to the Python environment between 3126 and 3143 other than upgrading OpenSSL and SQLite.

You will likely need to look into the source of the subprocess module to see what might be going on. Unfortunately I’ve run into weird issues with subprocess and invalid handles issues before on Windows, but was never able to come up with a solution that seemed to bulletproof.

0 Likes
#3

I don’t have a decompiler, so I wasn’t able to diff the zip files other than to prove that they were different, but I found that copying python3.3.zip and python3.3.dll from the prior installation to this new one fixed those issues.

Do you have any idea what features this reversion this would break?

0 Likes
#4

python 3.3 has an issue which can trigger this issue, if no stdin pipe was applied on windows. You might try to add stdin=subprocess.PIPE. It was reported several times to fix such kinds of issue.

But I never ran into such issues on my own.

0 Likes
#5

Tried that. It does work for some calls (check_output), but not subprocess.Pipe with a stdin specified.

I was able to resolve this so far by copying python33.dll and python3.3.zip from a prior version over, but I’d prefer to understand the root cause of the issue.

0 Likes
#6

It is weird that an old version works though supposedly no changes have been made to the Python 3.3 install. Have you done a binary compare of the new and old? If they are binary equal, then swapping them shouldn’t matter, but if they are not binary equal, then something did change even if it wasn’t code. Maybe the building method changed?

0 Likes
#7

They are binary different. Not sure how they were built – seems like the compiler was the same, but that’s all I can easily check.

0 Likes
#8

Then something is in fact different, even if the team didn’t physically change the code, it is not the same. I imagine something in their build environment changed.

0 Likes
#9

I have identical problem… not skilled enough to know how to fix. Help?

0 Likes
#10

The fix is to explicitely assign subprocess.PIPE to the stdin argument of Popen.

if WIN32:
    startupinfo = subprocess.STARTUPINFO()
    startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
else:
    startupinfo = None

return subprocess.Popen(
    args=args,
    startupinfo=startupinfo,
    stdin=subprocess.PIPE,   # python 3.3 bug on Win7
    stderr=subprocess.PIPE,
    stdout=subprocess.PIPE
)
0 Likes
Git p4 / custom menu entries