Is this text file loaded as plain text, or with a different syntax? Also, what version of Sublime Text are you using?
The answer below is a general answer about how Sublime Text works:
The way Sublime Text editor pane works is that is lexes source code files into tokens. Tokens are use for syntax highlighting, spell checking, word wrapping and more.
Depending on what syntax is being used, the lexing process can take some time. The actual syntax definition can be a big part of this. Syntaxes use regular expressions and state machine to determine when certain constructs are valid and what characters make up a given token.
In general we’ve been working on making this process faster with various of the default supported languages over time. Certain regular expression constructs are ignored to improve the lexing performance. Jon also wrote a custom regular expression engine for lexing that drastically improves performance for many syntaxes.
In addition to lexing, word wrapping has an effect on general performance since it affects the calculation of vertical offsets of a given line of text.