A good explanation is Jim Roskind on C ambiguity.
A tree-sitter is basically a GLR parser, and GLR parsers are context-free. Therefore, it’s impossible for tree-sitter to handle this kind of ambiguity. From the link:
Other threads of response included the use of a GLR parser, which tries all alternatives. Alas, as will be shown, the ambiguity in C is too great, and such an approach will not provide a unique syntactically correct parse when it does exist.
…
Since one of the respondees included suggesting that a GLR parser could just try all interpretations, and that only one would be valid, the following is a rather interesting example:
typedef int T;
main() {
int T=100, a=(T)+1;
assert(101 == T);
}
> Notice that the text "(T)+1" could incorrectly be interpreted, if "T" were still a typedef-name, as a cast to type T of the expression "+1". Note that the GLR approach would yield two distinct and valid parses, and hence there would be no single winner (without additional heuristics).
Real C parsers handle the ambiguity in the lexical analysis stage. They build up a symbol table as they go along, so they can tell when they encounter a name whether it belongs to a type or a variable.
[Tree-sitter's documentation](http://tree-sitter.github.io/tree-sitter/creating-parsers#lexical-analysis) indicates that the algorithm does separately perform lexical analysis, but as far as I can tell there is no way to write a custom lexer that maintains an internal symbol table and uses that table to distinguish variables from types. In any event, a custom lexer of that sort probably couldn't be compiled in the same way that the grammar is, and even if it could it would mean an extra layer of abstraction between the code and the parser, which wouldn't be great for performance. Plus, I'm skeptical that arbitrary user-defined lexers would be compatible with tree-sitter's reparsing guarantees, but I could be wrong about that.
I suspect that Sublime's engine actually could be augmented to handle C types. Sublime's syntax definitions are essentially descriptions of a pushdown automaton. Executing a PDA is very simple and “dumb”, so you're not likely to run into thorny algorithmic dilemmas when extending its behavior. In the [nondeterministic parsing proposal](https://github.com/SublimeTextIssues/Core/issues/2241), I identify only one place where the proposed behavior might interfere with existing behavior, and it's a performance optimization that could be remedied. The tree-sitter algorithm is much more sophisticated, which makes it potentially brittle. Adding context-sensitive features would likely violate core invariants of the algorithm, and it might not be easy or even possible to remedy those invariants.
For instance, I can't find what regex flavor tree-sitter uses, but I suspect that it doesn't support non-regular features like backreferences. Sublime does — it has its own highly-optimized regex engine that only handles truly regular expressions, but it will fall back to Oniguruma to handle non-regular features. There is a performance penalty, but when used sparingly backreferences add real expressive power to the engine that I'm not sure tree-sitter can match.
