Obviously I don't know the ins and outs of how Jon's implemented the indexing system in Sublime, but it does happen on a background thread or process so it shouldn't affect Sublime to the extent you're suggesting. Once the indexer's done its thing, it should only need to index changed files; that's how it should work at any rate.
So now I'm curious. I wonder if the indexer obeys the "file exclude" and "directory exclude" rules in the project settings? Because I'm thinking that when you do a build or whatever, a lot of (non-source) files will change in your project tree, but many of them could well be text and, from Sublime's perspective, indistinguishable from your source files. Java builds for example generate a lot of text files in the "bin" and "gen" folders which are of no interest for source browsing and/or goto definition. Repeated generation of these files during the build process could cause the indexer to keep rescanning a potentially huge amount of files. If, in your project settings, you exclude these folders and/or other unwanted file patterns, they disappear from the sidebar and (I wonder) should be ignored by the indexer?
Failing the above, turn off indexing as jbjornson suggests. If you're working with massive projects with many thousands of files the indexing may be less desirable and/or impractical. If you want to try that I'd suggest making the setting in your project file.