Indexing slow again

[rowen]

For awhile indexing performance was really good, but in the most recent builds (including 3148) it has become intolerable on my Mac. Every time I save a file (even of a type that is excluded from indexing) ST pauses for about a second. Performance is even worse when I’m running a build; even getting a window to give me focus can take a second.

I’m running on a a very fast machine: the current generation iMac with 1Tb SSD (half full), 24Gb of RAM and the fastest CPU.

I do use DropBox to synchronize two sub-folders of ~/Library/Application Support/Sublime Text:

• Installed Packages
• Packages
  so I can share my config with home. But all other sub-folders are directly on the SSD: Cache, Index, etc.

I exclude everything I can think of from using “index_exclude_patterns”: *.log, *.dox, *.html and *.xml.
I have tried setting index_workers = 1. No help.
I have tried disabling all extensions. No help.
The only thing that gives me good performance is to disable indexing entirely.

Any suggestions on other things I could try?

[jps]

The issues you are seeing are unrelated to indexing, it’s almost certainly triggered by a plugin

[rowen]

Yet when I disable all plugins I still see the performance problem, and when I turn off indexing, ST’s performance is excellent. So how could it be a plugin? I suppose it’s possible I messed up in my disabling of plugins (I wish that was easier to do).

[jps]

Sorry, I missed the part where you said disabling indexing does indeed help. Take a look at Help/Indexing Status menu, and verify that you haven’t added your root file system to the sidebar or similar

[rowen]

The indexing status looks normal. A small, reasonable # of files is being indexed.

Thanks for telling me about that menu; I had already used a clumsier way to see the indexing status (issuing some command in the console) after reading about diagnosing index issues. I was hoping my index was corrupted, but I saw no message to that effect.

I tried your suggestion of disabling packages again, just in case I messed up last time. I started a huge build of my code (to keep my machine busy and stress ST’s performance). Then I got the list from Package Control.sublime-settings and pasted it into my exclude_packages setting. This time ST’s performance was good – suggesting you were right!

However, after re-enabling my packages in batches (restarting ST each time) performance is still good.

I’m very confused. I’m glad to have ST behaving well again, but I wish I had a clue what fixed it.

Anyway, thank you very much for your help.

[rowen]

Performance slowed down again shortly after I posted my previous message. I have no idea why it was OK for a bit.

The slowdown does seem correlate with the indexer working, even if it is not very busy. When I save a source code file I get a hang of a second or two, with a pizza wheel cursor, but the indexer has just spawned a single job that is indexing just one file (presumably the file I saved).

I see this even with all packages disabled.

It may have something to do with the number of files in my project. The code base I work with is fairly large – about 5k each headers, C++ source files and Python source files, distributed among several dozen “packages”. I usually have a the whole code base open in one project window, so I can search for things in in the whole code base. I also often one or two single “packages” open in other project windows, so i can easily limit searches to just the “package(s)” I’m working on.

If I close the project window that contains the whole code base then things seem saner. So I wonder if the slowdown is due to the indexer searching all files to see what it needs to index? If so, could it do that in the background?

[bphunter1972]

I’m having a similar problem. Since 3148, although Help/Indexing says that it’s Idle, the “KITT light” is going on in all of the folders in the sidebar and the folders show no files.
Closing the window or switching projects has no effect. Only quitting out of st3 seems to resolve the problem, only for it to happen again.

[jps]

@rowen that number of files should be no issue, while I haven’t checked for any regressions (there haven’t been many changes in the area), when the indexing system was built I was testing with the chromium code base, which has an order of magnitude more files, and no perf issues.

In the past leveldb has gotten into an unhappy state; it’s worth deleting your ~/Library/Application\ Support/Sublime Text 3/Index/ directory (with Sublime Text closed) and seeing if that makes a difference

[rowen]

Deleting the index worked for several hours. Performance was great with indexing turned on. However, the performance is bad again.

I wonder if editing files in overlapping projects (one ST project window is open to one set of files, another project window is open to a larger set of files that includes those in the first window) could corrupt the index?

[bphunter1972]

This worked for me, thanks!

[rowen]

I have now tried deleting the index several times. Even while the index is being build performance is reasonable, but within a few hours of the index being built performance is unacceptable again – suggesting the index keeps getting corrupted. It seems as if I may have to give up on either the index, or on my preferred workflow of 2-3 windows open, one with the full project, one or two for individual packages in our project.

[parejkoj]

I was having this same problem, when working with a large project directory. When I had the whole project open in one SublimeText window and I opened and edited a subdirectory in a different SublimeText window, I would get several seconds of Sublime non-responsiveness (on Linux, so no beachball). Deleting the index has fixed it for now: I can edit in the single large project window) without noticeable slowdowns on save.

I’m running build 3143.