I’ll give that a shot, but now I’m expecting to see RAM climb slowly until there’s none left.
I see there’s some variables related to excluding, like index_exclude_patterns. On our project I think it’d be more useful to have an inclusion filter option, there’s too many filetypes I never want to scan, and comparatively few I do. Anyway, I’ll start trying that out and a few others. Cheers.
Well, 0 is generally good. On a 4 core or less machine we will run 2 indexers. On a 5+ core desktop, it will use n-1 indexer by default, where n is the number of CPU cores. If you are seeing too heavy cpu utilization, I would try 2 or 1.
At around 11GB memory usage the UI starts to show a loading icon and I can’t interact with Sublime Text.
[Edit] Actually, at the moment it does that from a fresh open within 10 seconds, when it picks up where it left off.
FWIW indexing does in general scale well to large folders: On my desktop, indexing Chromium (13GB, 250k files, 18M lines of code) takes a few minutes and doesn’t otherwise cause any issues. This is partially file type dependent though: any syntax definitions that trigger the oniguruma slow path (e.g., with backrefs) can take longer to index.
Co-worker just mentioned he suspects it’s our virus checker, rather than just Sublime Text’s fault? I’m not sure why that would be, but just throwing out his comment.
[Edit] For now we’re just shutting off file indexing since it seems to get stuck, maximizing our 64GB of RAM.
I could see the big change in behavior on Sublime’s part (reindexing everything all of a sudden) could trigger a virus checker heuristic. That would definitely make sense why everything would get laggy as the virus checker would be blocking requests check to see what is going on.
Wanted to add that by removing the full parent directory from my project, adding back only the folders containing the files I need to edit and adjusting the index_workers from 0 to 1, my cpu and memory issues are back in the normal range. I have always included the parent directory in my projects and never had an issue until recently though so I’m not sure what has changed causing the cpu/memory spikes but at least I can work around it. Thanks for the suggestions.
I have the same issue on Sublime 3 Build 3170 on macOS.
1GiB RAM and 100% CPU. My workspace ~20 windows with several directories per window open contain several GiB of git repos. Probably ~100GiB with binaries (linux-kernel, GCC, GLIBC, QEMU and dozens of other smaller projects).
Thanks to this thread I found Indexing Status… on the Help Menu.
What I would really appreciate as a user is some indication somewhere of background work/progress without needing to search the forum and find some hidden menu item. A little working throbber, spinner or something animated in the status bar, that when clicked brings up the indexer status would be absolutely awesome.
Hope to see a status bar spinner/throbber for background processes in an upcoming build.
Thanks for the great product. I updated to build 3170 about 30 minutes or so minutes ago and its about 70% through my files. Its a fast indexer.
I’m seeing the same thing. Everything was ok with 3.0 (although a bit sluggish given the number of files I have indexed), then upgraded to 3.1 3170 and now CPU and memory are very high for sublime_text.exe (82% on a 16-core machine, using up 12GB out of 32GB). Sometimes it gets to a point where the machine gets unresponsive and I need to close it. This wasn’t a problem before the update.
I also see a few instances of this message in the console: “number of files to index exceeds sanity limit, bailing out”. Again, I don’t remember seeing this message before the update. I see some sort of progress report stuck at 0% at the bottom of the window (status bar?). Help -> Indexing Status says “Indexing” but doesn’t show anything else.
Humm, interesting, I should have looked into this before posting. I found out my project file was pointing to a couple of folders that didn’t exist. Not sure that was the problem, but after fixing them, indexing progress up to 100% somewhat quickly (took maybe 2-3 minutes), then went back to 69%, then completed. After that it showed 0% a couple of times but it seems to be stable now. CPU utilization for the main Sublime process never went past 17% and memory never went above 2GB. It’s in idle now (0% CPU and 385MB). I tried to replicate the problem by editing the project and adding a couple of non-existent folders again but after some activity, CPU and memory utilization went back to normal.
[Update]
I’ve seen sublime_text.exe increasing CPU and memory utilization while sitting idle. It got up to ~50% and 6-7GB but eventually backed down. I wonder if it will go crazy again if I just let it sit there with the non-existent folders in the project settings. BTW, when it goes crazy there’s only a single sublime_text.exe process, contrary to the regular indexing behavior where indexing processes get spawned up.
Now my indexing status is “idle”, however Sublime Text process went up to ~390% CPU and about ~3.9GiB memory (quad core i7 macbook, build 3170).
I restarted Sublime Text and CPU goes up to 100% and memory is about ~2.5GiB after a couple of minutes.
After restart, Indexing Status says “idle”. There must be some other background work that is not reported.
Indexing Status shows “idle” but my system is getting close to unusable. I guess I have to close all projects (unfortunately I like to have about 20 windows open) and then open them up again one by one to find out which one is causing the CPU usage.
There is - in the status bar there is a progress percentage that increases over time. If you click it, the indexing status window opens.
Does this happen with a clean install? Perhaps one of your plugins is having an issue? Perhaps see what the Tools > Developer > Profile Plugins menu shows?
@mjc if you exit Sublime Text, then delete the “~/Library/Application Support/Sublime Text 3/Index” folder, and then open it up again, does the problem go away?
That will force all files to be indexed again, so there will be some work done at startup, but I wonder if things will be working well after that. The index format changed between 3.0 and 3.1, and perhaps Sublime Text is trying to parse the old index data when it shouldn’t be.
Thanks all for the advice. I solved the problem.
I have a habit of creating anonymous workspaces and relying on the fact that Sublime Text will remember them when I quit and reopen Sublime Text.
I went through every open workspace and did “Save Workspace As…”. The system was crawling while I did this, and there was no indication that Sublime Text was indexing. In any case, this took me a few minutes, as I got the macOS spinning ball after saving and closing each workspace. It turned out I had 14 windows/workspaces open so it took about 15 minutes. After this was done, I quit and re-open Sublime Text and opened just 3 of the workspaces and everything was fine.
I like to have many workspaces open, even ones that I’m not actively working on, because Alt-Tabbing through them reminds me of all things that need attention at some point. i.e. they hold my context for the various projects I’m working on…
I suspect there is some background task outside of indexing that is not reporting its status in the Indexing status window, possibly when upgrading from an earlier build of Sublime Text to build 3170. I noticed after opening a new workspace that if I added a directory to it, that the icon was animated suggesting indexing activity. These animations were not present on the workspaces that were restored from the earlier version of Sublime Text that I upgraded from. i.e. this issue was possibly related to upgrading workspaces and some interaction with the indexer.
All is good now.
@wbond I’ll look out for the indexing status in the status bar next time, however, the indexing status was not apparent to me. I did notice that when I added a new directory to a project, that its icon was animated, suggesting indexing. However, this didn’t show up on any of the old projects prior to the upgrade, even though I knew indexing was happening (via Help -> Indexing Status… ). Is this animation visible on all status bars? (I had 14 workspaces open) or is it just visible on the workspace that is being indexed?
I know the thread is quite old, but I had just had the same issue where Sublime was using 100% ! of the CPU of my new Alienware R2… luckily I tried adding ‘node_modules’ to the “index_exclude_patterns” list and the CPU usage immediately dropped to 0%! Hope that helps any JS devs out there 
