Sublime Forum

Python Improved - a better Python language definition


OK, so technically this is a language definition and not a plugin, but I figured I’d post it here anyways. [size=150]Python Improved[/size] has (finally) been released on Package Control for all the world to use. Basically, I started out with the default Python.tmLanguage files from ST2 and TextMate, and ironed out a few minor (if I recall) differences between them, and started making some improvements, like having any *Exception or *Error highlighted, instead of just the predefined ones, and clarifying the difference between a “builtin function” (like open() or isinstance()) and a “builtin type” (like “ascii” or “bytearray”), even though many of them are also functions. I’m also a huge fan of @wuub’s SublimeREPL and its IPython integration, so I added scopes to highlight the “In #]/Out #]” line markers.

At some point I discovered @facelessuser’s Better Python and adapted many ideas from there. I then started playing with Django and installed Djaneiro, but wasn’t completely happy with it, so I cannabalized the parts I liked most and added them to what I was now calling “PythonImproved”. The project kind of languished for a while, but the success I’ve had with my Neon Color Scheme, which contains syntax highlighting for all of the new scopes in PythonImproved, I decided to clean it up, put together a README, and submit it to Package Control, and here we are.

None that I’m really aware of, having been using it and tweaking it for most of the past year, but there is one thing I’d like to ask for help on: I haven’t put an extensive amount of work into it, but I’d love to support Python 3’s function annotations, even just the straightforward “:” and “->” operators, perhaps with a specialized scope for the word that comes after:

def greet(name: str, age: int) -> str:
    print('Hello {0}, you are {1} years old'.format(name, age))
# instead of the standard
def greet(name, age):

So, if you’re an expert regexer and want to contribute, head over to GitHub and open a pull request. Otherwise, if you have any other questions, concerns, implementation problems, whatever, feel free to open an issue and I’ll get back to you as soon as I can.

This will go into the README soon, but if you love Python Improved so much you want to make it your default Python language definition, do the following (in ST2):

  1. Copy “Packages/Python Improved/PythonImproved.tmLanguage” to some other location in your filesystem, not under the Sublime Text hierarchy.
  2. Open PythonImproved.tmLanguage in ST2.
  3. Change the contents of the key on line 44 to “Python”
  4. Save the file as “Python.tmLanguage”
  5. Quit ST2.
  6. From your OS/command line, rename “Packages/Python/Python.tmLanguage” to “Python.tmLanguage.backup” or something similar, just in case.
  7. Delete “Packages/Python/Python.tmLanguage.cache”
  8. Copy the newly edited and renamed Python.tmLanguage from its other location to “Packages/Python”
  9. Restart ST2.
  10. ???
  11. Profit!
    (sorry, couldn’t help myself)

So, I hope you enjoy it, and if you have any questions just let me know!




Will this support ST3?



@chid: considering that is only language file, it should work just fine.



I will have to check this out :smile:.



Don’t know if you have an ongoing list of added scopes, but if you do, it might be worth including those in the README. That way people can customize their own color scheme rather than relying only on your Neon theme.



It’s definitely on my list of things to do, I’ve just been very busy with work and family life the past several days.

What, one theme doesn’t work for every single person? I’m shocked! :astonished:



In the year(s) of using ST with Python I also experienced some things that the syntax file did not cover. Maybe some of it hasn’t been fixed yet by you (didn’t check myself):

  • next keyword (Py3)
  • __bool__ function (Py3)
  • 0b100101 numeric literal notation
  • When splitting a def function definition into multiple lines, comments are not recognized

You might want to consider monitoring … state=open :wink:




  • next is included in “support.function.builtin.python” - it’s possible your current color scheme doesn’t highlight that scope properly.

  • I just added __bool__, so next time you update it should be there.

  • PythonImproved does include support for binary literals, both regular (‘0b01100101’) and long (‘0b01100101L’)

  • I also just added support for comments within function definitions:

def myfunc(self,            # gotta have self
           param1="value",  # values are cool
           param2=True,     # or False, whatever
           **kwargs):       # you never know

I’ve put a list of new/modified scopes in the README. I think I got most of the major ones, but I did leave out some minor housekeeping, regex tuning, and whatnot.

As always, if something doesn’t look quite right, or you have any feedback at all, please let me know!




@MattDMo I decided to go ahead and give your language a try. I think I can probably help you with the Python 3’s function annotations. I like what you are doing with this.



I see why you were struggling with those annotations. They are flexible enough to make things a pain. I’ll take a crack at it. It may be one of those things that will require multiple iterations over it before everything is 100% covered (or near 100% covered), but if a couple initial passes can bring it up cleanly greater than the current 0%, it will be an improvement.



[pre=#2D2D2D]def greet(name: str, age: int) -> str:
print(‘Hello {0}, you are {1} years old’.format(name, age))

instead of the standard

def greet(name, age):

How does it look?



These theme files always make my head hurt, but here is a more complicated example; comments and all:

[pre=#2D2D2D]def foo(
(x1, y1: expression), # Is this a comment?
x2: 3 + 2, # why yes, that is a comment!
y2: “some random string” # Yet another
) = (None, None)
) -> “yes this is the result annotation as a string”:
print(‘Hello {0}, you are {1} years old’.format(name, age))[/pre]



That definitely looks like a start. Can you send me a pull request?

I just saw you fixed the <string> issue in the comments at the top - I meant to do that with my last release, but must have missed it. Thanks!



I always convert my themes to json, and the syntax broke my plugin’s conversion, so I went ahead and patched it.

I will issue a pull request within the hour. I will document the new scopes for the annotation separators as well.



Pull request made.



Just stepping into this (and promoting): There is also the possibility to convert syntax definitions to YAML instead of JSON which is much more flexible, doesn’t require {} all over the place, does not require “” all over the place and does not require you to escape everything all over the place. Oh, and it supports multi-line strings and stuff. Here is an example: … tmLanguage

If you encounter any bugs (especially with the syntax highlighting, because YAML is a beast to highlight with regular expressions), please report them on the repo.



One thing I would love to see is a simple plist to yaml and yaml to plist plugin (without all of the other AAAPackageDev stuff I don’t use). Maybe I will take a look and strip it out for personal use. I haven’t played with yaml much, but I think I would like to.



The only relevant files are file_conversion.yaml and fileconv/ for conversion. The other syntax definition-helping stuff is in and Support and Syntax Definitions subdirs. Only downside is that it’s currently only for ST2 (because I have problems). But well, that’s not in the scope of this thread.



(Off-topic vote for YAML support)

I have been following along to this thread, chiefly because I am working on an improved Markdown syntax. I have gotten used to writing in JSON, chiefly because of the economy of the syntax. But the lack of multiline strings is an issue, as I’d like to annotate my regular expressions before I forget what it is they do :smile:

I have failed (angrily) to use AAAPackageDev in the past and am a happy devotee of PlistJsonConverter – especially the on-save conversion. I would like to see YAML supported in the hope that it might be a more efficient format to work in.



@quodlibet I am considering ripping the yaml portion out and creating a plugin. I mainly use JSON because the plugin was easy to throw together, and it was “good enough”. But if getting yaml module up and running from AAAPackageDev isn’t too bad, this might be a very nice alternative. If I did this, I would construct it similar to the PlistJsonConverter. I just haven’t had the time to look into it yet.