I believe it is mostly to do with syntax colouring. Sublime along with textmate are both particularly sophisticated in this area, allowing things like context specific colouration / syntax selection. Both are slow with large files, due to their heavy dependence on regexs. In addition, Sublime has the minimap which requires everything to be coloured correctly in realtime, making lazy colouring more difficult. Large edits on large colourised files will be particularly slow, as will undos, and large amounts of multi-cursor edits will exaggerate things. Forcing sublime to turn off colourisers with "plain text" should speed things up alot.
Sublime 3 has improved in this area, though it would be nice to be able to disable colouring by default in certain situations (per session ie. View.. disable auto colourising) or to make this a setting available as either a global, per-project or per-file setting. Rather than doing it "after the fact".
I wonder if S3 has moved to doing this on a background thread (or will), so you can get working with your large files right away while the thread parsers through and applies the colouring. From what I've gleaned, the symbol indexing thing only happens when folders are added to the project, so this shouldn't be an issue if the project's empty.