Sublime Forum

Is there a tree-sitter like code navigation functionality in sublime

#1

is there a way to select the text contextually based on language syntax (for all supported languages):

lets say I have the following code:


# class block expression
@dataclass 
class Foo:
    text: str
    count: int

f1 = Foo(text="foo", count=10) # single expression

for i in range(f1.count): # loop block expression
    print(f"hello {f1.text}") # single expression

so when I put the cursor anywhere within the Foo class body I would like to select the whole class code (base on the context around the cursor).

I could use regex, but that’s a lot of work and error prone to do manually. another option would be to use a tool like tree-sitter but that is not native to sublime and requires special wirings to get it working.
I know that sublime has its own builtin syntax engine, however I’m not sure if it exposes any api for plugin developers to make use of.

(BTW, happy new year in advance)

1 Like
#2

I image it can be done via following APIs.

The above APIs, more or less, expose the scope around the cursor position. That can be expanded to the region you want, depending on how well the syntax definition is written.

Update:

image
image.png827Ă—125 19.7 KB

So it’s obviously that I am too naive. It’s probably not that easy or even not feasible in this way.

There is built-in “expand to scope”, but I never know how it works. It works more like it’s expanded randomly to me :upside_down_face:

image

0 Likes
#3

thank you @jfcherng for the reply :hugs:

as you already noticed the builtin expansion functionality is not that concise as it does not take the language syntax into account. it just understands simple text constructs and in languages like python where you have things like decorators it simply does not recognise those as part of a related block of code.

I suppose the feature I am looking for is not in sublime out of the box and probably possible through a plugin.

0 Likes
#4

For Python, LSP-pyright provides accurate “expand selection” but that’s language server specific.

This is how the selected region changes when sequent "expand selection"s are triggered.

Update: Lol… no. This is a Pylance-only feature…

1 Like
#5

Unfortunately this is not the case. LSP-pyright doesn’t have the selectionRangeProvider capability:

LSP just falls back to the built-in “Expand Selection” from Sublime Text here.

Perhaps the alternative Python language server from LSP-pylsp supports this; I haven’t checked it though.

2 Likes
#6

interesting… wasn’t expected this is a Pylance feature…

image

I guess it should be grey out if pyright doesn’t claim supporting it?

0 Likes
#7

Huh … so without wanting to be at all negative, and just to understand the state of affairs.

  1. This is a feature in the proprietary pylance that isn’t available in the open/free pyright? Not surprising and pylance is still very nice. I for one have been on the look out for these and we should all have as well while the whole LSP thing and some of the MS servers took over … embrace and extinguish and all that.
  2. Is this an instance where Sublime is somewhat behind the whole Tree-sitter thing? I’m not over all of the details of Tree-sitter, but I do hear nice things and I can imagine that a constructive conversation about what Sublime could learn from it would be helpful.
    • Or is this more an instance where the python syntax could be better written to take advantage of the newer Sublime syntax engine?

Generally though, and I am getting into Text Editor wars stuff here, sorry, really … I haven’t been attracted back to the likes of neovim/emacs as much as I have been recently simply because Tree-sitter seems nice (based on superficial impressions) and I feel like I’m increasingly reliant on LSPs anyway which means the text editor becomes less relevant as a choice (again, embrace and extinguish).

Interestingly, I recently ran Sublime without LSPs, and it was wonderful how snappy it was, but the awareness of the code-completion system (which I may not have set up appropriately?) just wasn’t enough for me. And with python, the type information is just wonderful, which takes me back to pylance. And whenever I see pylance being slow, I wonder if the python runtime or Sublime is the bottle neck (however unlikely that is) and wonder whether I should check out neovim or VSCode just to see.

1 Like
#8

So getting back to the initial question … is this possible with tree-sitter generally? Can the Sublime syntax engine recognise this? Can tree-sitter be used in Sublime?

0 Likes
#9

No. Sublime Text itself has nothing to do with tree-sitter. As for using Python to execute arbitrary external shell command so theretically you can do anything, I don’t think one would be satified by this solution.

Either, I don’t think there is a tidy way for ST’s line-based syntax engine to detect whether a def class(): block has ended or not. Especially, for indentation-based language like Python.

0 Likes
#10

Interesting … I find that mildly surprising.

I can imagine some sort of parsing that basically looks for the last line that has more leading whitespace than the initial def ... line, with the nested exception of parenthetical blocks which, if I recall correctly, can have whatever indentation they want.

0 Likes
#11

tree-sitter has an open specification and it is (almost) easy for the sublime core team to add it to the editor, so the editor can benefit from the syntax definitions available online.
I wish we could bring this topic up to the core team attention and get their feedback if this is even a possible option.
I do like sublime a lot and I bought the license and I want to use it everywhere, but as @maegul implicitly mentioned, I also dislike the LSP feature as it slows down the editor by an order of magnitude, I believe relying on LSP can hurt sublime in long term.

also regarding the python engine performance, I hope soon we could be able to switch to python 3.11 which has a nice bump in terms of performance.

0 Likes
#12

This is very unlikely to happen. It would break all ST syntax packages or mean to support a second highlighting engine.

I wouldn’t want to maintain syntaxes for the messy js driven tree-sitter thing.

1 Like
#13

I mean the specific capability asked for here seems like quite a minor quality-of-life improvement. If you’re already willing to tough it out with no real autocompletion (no LSP), then selection taking a few more keystrokes than it possibly could doesn’t seem like much of a burden.

ST already has some “force multipliers” for movement and selection that aren’t bound by default (e.g. calling the "move" command with the "by": "stops", "empty_line": true, "extend": true arguments).

0 Likes
#14

I use LSP extensively (with Python) and I personally don’t find the editor lagging at all (It may lag for large projects, but so does every other IDE). LSP is probably the best thing that has ever happened to ST.

I doubt they will do this anytime soon. It just means maintaining a 3rd python run time, different from 3.3 & 3.8

0 Likes
#15

If the python performance improvement continues with the same success they’ve had so far, it might be hard for Sublime to ignore a newer version of python3 by about v3.14 or so.

Also, how many backwards incompatible changes are there from 3.8 to 3.11?

0 Likes
#16

  1. Sublime Text claims compatibility with Windows 7 and python 3.8 is the most recent version supporting it as well.

  2. 95-99% of all available plugins are still running on python 3.3 and therefore don’t even benefit from 3.8’s performance gains. Most of them probably won’t ever do so, if we don’t find external ways to force them to (maybe via Package Control).

  3. Performance concerns are not of major interest for majority of plugins as almost all are rather simple and small. Beyond that the limiting factor for larger text operations is IPC and required thread safety messures between ST and plugin hosts - not python itself.

  4. Compatibility is a minor issue for un-compiled plugins, except more recent python versions dropping features, which may probably be used by older plugins.

  5. Compatibility is a major issue for pre-compiled plugins (e.g.: SFTP, …) as nearly each python version breaks byte code structure. Same for compiled libraries/dependencies. Hence each change in python version requires plugin/library authors to migrate their plugins while probably maintaining separate builds for older ST releases.

3 Likes
#17

I just noticed something!

sublime already has the folding functionality and it perfectly recognises the block boundaries for all supported languages. that means this functionality is somewhere in sublime where fold is based on, however I’m not sure if it is being exposed or not.

the point is the same way you can clearly expand and collapse code blocks using fold functionality, we should be able to select the text/code blocks

2 Likes
#18

For anyone that finds this thread, there’s now a TreeSitter package for Sublime Text: https://github.com/sublime-treesitter/TreeSitter.

It provides Sublime Text with a performant and flexible interface to Tree-sitter. It works out of the box with around 40 languages, and can be configured to work with any language that has a Tree-sitter grammar.

It ships with commands for “tree-based” selection and navigation. For example, you can select ancestor, descendant, or “cousin” nodes based on the current selection. This makes it easy to select the whole class or function if your cursor is currently “within” that class or function. You can also go to symbols returned by configurable tree queries, with symbol breadcrumbs for context.

And it exports APIs that let package developers build Tree-sitter based packages and custom commands.

0 Likes