Sublime Forum

Continuous Re-Indexing

#1

I have a very large project that I am currently working on.
I have carefully excluded everything that I can from being indexed.
Still indexing can take 30+ minutes and can really bog down my system.

I have checked that I do not have the MarkdownEditing package installed (as seen in other posts that can cause longer indexing)

My main frustration right now is that every time I switch projects (and I do this often) and every time I restart sublime I seem to be forced to re-index the entire project. Is there any way to have sublime cache these results and only index incrementally or when required?

I am using the LSP and while I find it is almost good enough, there times where I still really want the simplicity of the sublime index and goto-anything to function, so I would like to avoid turning off the indexing functionality.

Any help would be greatly appreciated!

0 Likes
#2

Do you have a license? If so, you could try a 4xxx build by joining the Discord server - Resources and Bug Tracking. There are some improvements in those builds that should make indexing faster, and can help with frequent indexing.

0 Likes
#3

Yes I have a license.
How do I get access to the ST4 builds?

0 Likes
#4

Join the Discord server.

0 Likes
#5

It is slightly faster. Still 25 minutes every time I restart sublime.

0 Likes
#6

ST always caches the index, though if the files have been changed they are re-indexed incrementally. How many symbols do you have in your project? (You can check using Help > Indexing Status…) Are you using any custom syntaxes? Have you tried reverting to a freshly installed state?

0 Likes
#7

is using 26352081 bytes for 331966 symbols across 2706510 locations

0 Likes
#8

index “gecko-dev” collated in 2.56s from 228895 files
index “gecko-dev” is using 75467904 bytes for 739791 symbols across 7181115 locations

A much larger index that I just loaded from cache in ~3 seconds. Caches for indexing should be in ~/.config/sublime-text-3/Index/.

0 Likes
#9

Does the index status in the status bar start at 0% each time you start Sublime Text?

Edit: also do you see an messages in the console about the index? If the index is detected as corrupt it will rebuild fresh.

0 Likes
#10

I don’t get a specific error about the index. I do get the following:

error parsing lexer: Packages/User/Log.tmLanguage: Unexpected capture value
error parsing lexer: Packages/zzz A File Icon zzz/aliases/CoffeeScript (Gulpfile).sublime-syntax: Unable to read Packages/zzz A File Icon zzz/aliases/CoffeeScript (Gulpfile).sublime-syntax

Could a failure to parse a syntax cause the indexer to decide to run?

0 Likes
#11

It does start at 0% every time.

0 Likes
#12

Could be. You could try removing those syntaxes to confirm.

0 Likes
#13

Removed them.
It is still re-indexing every time I restart sublime.
This has me wondering if I have some odd package that is triggering this, I really should have tested that before posting here.

Is there a setting to tell sublime to load without loading any packages? I can go through and disable all my packages, but that gets pretty tiresome when all you want to do is validate something like this.

I have to say 4’s indexing IS noticeably faster than 3 though! More excellent work by the sublime crew.

0 Likes
#14

Since you’re using ST4 you can try starting it up in Safe Mode; that’s like a temporary revert back to initial settings for everything, so only the core Sublime packages would load.

1 Like
#15

Oh sheesh.
Once I have removed all packages it is not choosing to index repeatedly.
That is rather annoying. So my problem all this time is that some package out there is causing the indexer to think, for some reason, that it needs to re-index.

It would be great if the indexing system could tell us why it thinks it needs to re-index in the log.
Time to go search through my packages looking for what is causing this.

Thank you for the help so far!

1 Like
#16

Had the same issue here. Sublime would take 4+cores for ~12m to re-index our ~200K file project from 0% each time I (re-)launched

…Turned out that the broken plugin that was causing it was an internal corporate devtool. :crying_cat_face:

After I removed the symlink to that from ~/Library/AppSupport/Sublime Text 3/Packages/ it came back to normal

(Sublime Text 3.2.2, Build 3211; Licensed ; macOS 10.15.7 (19H1030) ; MacBookPro intel 13" 2020)

1 Like