Sublime Forum

Indexer improvements

#1

Hi Sublime Community,

I like sublime text, but it’s not always available option to work with. The problem is: indexing. It takes more than 20 minutes to index all project files on slow file system. As a developer, I find it strange, we have indexers in our products and all of them work instantly. This is possible because all of the index results SAVED in storage and updated whenever it’s required.

I have seen similar topics on ST2 and ST3, there were a lot of performance improvements made to rebuild full index faster optimizing fs requests and internal work, but why? There is no need to optimize hard and heavy work if you don´t need to do it.

If I were ST4 developer, I would make some kind of .Sublime-index file (or database like sqlite), where list all of the indexed files, mtime, symbols. And when an editor starts I would load this index and started file scan (in a separate thread or even other process, to avoid locking on ui while fs responds to all requests). Simply check all folders and check if files were added, removed, modified and push these changes to indexing queue, whenever a user opens a file, push index update for this file on top of the indexer queue and sync with .sublime-index file time to time.

I’m almost sure this strategy is used by most of the popular IDEs. And even though indexing is slow, it won’t block the interface for 20 minutes.

0 Likes

#2

ST has been doing this for decades. The index is stored in the cache directory and is only updated when files change.

1 Like

#3

That’s nice! Thankyou @bschaaf.
But then what could freeze ST4 on project load for so long? I have tested directory lookup with find and it takes only about 30sec to list all of the project files. When ST window unfreezes I have seen indexing done so I thought it is related, but may be not.

0 Likes

#4

Indexing is done asynchronously; it doesn’t block anything. What version of ST are you using? What OS?

0 Likes