Sublime Forum

Unable to find user installed python libraries

#1

My Env

$ uname -ro  # returns: 4.17.9-200.fc28.x86_64 GNU/Linux
$ subl --version  # returns: Sublime Text Build 3176
$ python --version # returns: Python 3.6.6
$ which python  # returns:
# alias python='/usr/bin/python3'
# 	/usr/bin/python3
$ pip --version  # returns: pip 10.0.1 from /home/<username>/.local/lib/python3.6/site-packages/pip (python 3.6)
$ which pip  # returns: 
# alias pip='/usr/bin/pip3'
# 	/usr/bin/pip3
$ echo $PATH  # returns:
# /usr/share/Modules/bin:/usr/lib64/ccache:/home/<username>/.cargo/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/<username>/.local/bin:/home/<username>/bin:/usr/local/cuda-9.0/bin:/home/<username>/workspaces/go/bin:/home/<username>/.cargo/:/home/<username>/.local/bin:/home/<username>/bin:/usr/local/cuda-9.0/bin:/home/<username>/workspaces/go/bin:/home/<username>/.cargo/:/home/<username>/.local/bin:/home/<username>/bin:/usr/local/cuda-9.0/bin:/home/<username>/.local/lib:/home/<username>/workspaces/go/bin:/home/<username>/.cargo/

# Running the following from inside sublime:
import sys
print(sys.executable)

returns:
/usr/bin/python3
[Finished in 0.0s]

My Problem

When I import user installed python libraries ST3 fails to import some, but not all libraries. For example:

A file containing the following:

import numpy

Returns:

[Finished in 0.2s]

A file containing:

import textblob

Returns:

Traceback (most recent call last):
  File "/home/<username>/test.py", line 1, in <module>
    import textblob
  File "/home/<username>/.local/lib/python3.6/site-packages/textblob/__init__.py", line 2, in <module>
    from .blob import TextBlob, Word, Sentence, Blobber, WordList
  File "/home/<username>/.local/lib/python3.6/site-packages/textblob/blob.py", line 28, in <module>
    import nltk
  File "/home/<username>/.local/lib/python3.6/site-packages/nltk/__init__.py", line 114, in <module>
    from nltk.collocations import *
  File "/home/<username>/.local/lib/python3.6/site-packages/nltk/collocations.py", line 39, in <module>
    from nltk.metrics import ContingencyMeasures, BigramAssocMeasures, TrigramAssocMeasures
  File "/home/<username>/.local/lib/python3.6/site-packages/nltk/metrics/__init__.py", line 16, in <module>
    from nltk.metrics.scores import          (accuracy, precision, recall, f_measure,
  File "/home/<username>/.local/lib/python3.6/site-packages/nltk/metrics/scores.py", line 18, in <module>
    from scipy.stats.stats import betai
  File "/home/<username>/.local/lib/python3.6/site-packages/scipy/stats/__init__.py", line 345, in <module>
    from .stats import *
  File "/home/<username>/.local/lib/python3.6/site-packages/scipy/stats/stats.py", line 171, in <module>
    from . import distributions
  File "/home/<username>/.local/lib/python3.6/site-packages/scipy/stats/distributions.py", line 13, in <module>
    from . import _continuous_distns
  File "/home/<username>/.local/lib/python3.6/site-packages/scipy/stats/_continuous_distns.py", line 113, in <module>
    class norm_gen(rv_continuous):
  File "/home/<username>/.local/lib/python3.6/site-packages/scipy/stats/_continuous_distns.py", line 175, in norm_gen
    `optimizer` argument is ignored.\n\n""")
  File "/home/<username>/.local/lib/python3.6/site-packages/scipy/misc/doccer.py", line 159, in _doc
    start_of_notes = cls_docstring.find(notes_header)
AttributeError: 'NoneType' object has no attribute 'find'
[Finished in 0.4s with exit code 1]
[shell_cmd: python3 -OO -u "/home/<username>/test.py"]
[dir: /home/<username>/]
[path: /usr/share/Modules/bin:/usr/lib64/ccache:/home/<username>/.cargo/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/<username>/.local/bin:/home/<username>/bin:/usr/local/cuda-9.0/bin:/home/<username>/workspaces/go/bin:/home/<username>/.cargo/]

If I launch python from a terminal and import textblob the following returns:

>>> import textblob
>>>

What I think the problem may be:

Sublime is only seeing packages installed at the global level, not the user level.

In my first example I imported Numpy, which I installed with my distros packages manager.

$ sudo find / -name "numpy"

returns:
/usr/lib64/python3.6/site-packages/numpy
/usr/lib64/python3.6/site-packages/numpy/core/include/numpy
/usr/lib64/python2.7/site-packages/numpy
/usr/lib64/python2.7/site-packages/numpy/core/include/numpy
/usr/include/boost/python/numpy
/usr/include/numpy

In the second example I imported textblob. Textblob was installed with pip. The version of pip I have prompts users to use the --user flag when installing libraries.

$ sudo find / -name "textblob"

returns:
/home/<username>/.local/lib/python3.6/site-packages/textblob

Conclusion

The $PATH listed in the error message from sublime is not the same as $PATH in bash. I am not sure how to alter the $PATH in sublime or have it point to my $PATH defined in .bash_profile.

Additionally ST3 seems to be calling the -OO flag. When passing this flag in a bash environment I am able to recreate this error outside of ST3. Running without the flag operates as expected.

This problem persists across multiple work stations running the identical environment.

Any help would be greatly appreciated.

0 Likes
#2

Like every other language that Sublime supports (both natively and through external packages), external programs are executed using the contents of a sublime-build file that’s set up to tell Sublime how it should execute those programs and in what way.

The sublime-build file that ships with Sublime for executing Python programs is Python/Python.sublime-build, which looks like this:

{
    "shell_cmd": "python -u \"$file\"",
    "file_regex": "^[ ]*File \"(...*?)\", line ([0-9]*)",
    "selector": "source.python",

    "env": {"PYTHONIOENCODING": "utf-8"},

    "variants":
    [
        {
            "name": "Syntax Check",
            "shell_cmd": "python -m py_compile \"${file}\"",
        }
    ]
}

Note that in particular the default just executes python and leaves it up to you to ensure that this works as expected; on some systems this will execute Python 2 instead of Python 3, for example.

Based on your problem, you’re not using that built in build system and are instead using either one you created yourself or one provided by an add on package, and by the description of your problem and confusion over the use of the -OO flag, it’s probably the latter.

So at some level your issue is likely being caused by the build system that you’re using; I would check to see if you installed a third party package such as Anaconda or something that might be doing you favours.

As defined above, using shell_cmd will cause Sublime to use bash -c to execute the command line, which should pick up your path the same as it would if you ran it in a console. The sublime-build can alter the path that gets used by adding it to the env key in the build system, which may be what your issue is.

Note that since Sublime is not using the --login and instead using -c, bash won’t read the .bash_profile or the .bashrc file and will instead check the $BASH_ENV environment variable for the file to execute (at least in the version of bash installed on my Linux machine, anyway), which may also be the source of some environment weirdness as well.

Generally I would say that if the sublime-build doesn’t contain anything that sets the PATH to something else, then the invocation of bash used to execute the interpreter is going to use the environment as Sublime sees it, which would be the environment that was in effect when it was launched, either from your terminal or from some window manager dock application.

I’ve had issues in the past where changes to my .bashrc did not seem to take effect and the ultimate reason is that the window manager was started using the original .bashrc and giving that base environment to everything that didn’t manually take actions to fix the environment, for example.

0 Likes
Environment Variables
#3

Thank you for being so quick and detailed in your reply.

You are correct, I had downloaded a Python3 build system from package control (specifically github.com/petervaro/python). I had forgotten about this, thank you for reminding me.

I copied the build system you provided and changed instances of “python” to “python3”, and it worked perfectly.

I have been struggling with this problem for a little while now, and your diagnosis was insightful. This solved my problem.

1 Like