If files aren’t tracked by git directly, then the mini_diff functionality shows you changes made to the file since you last opened the file or first saved the file, whichever happened last.
For example, if you create a new tab and start typing, you don’t see any inline diff functionality because the file is not on disk yet. Once you save the file, when you make changes to it, the changes are tracked based on the file as it was the first time you saved it. As long as you keep editing the file, the diff still shows you changes based on that first save.
Similarly, if you close an untracked file and re-open it, no changes are indicated, but as you make changes, they will reflect the state of the file since you opened it; in order to reset the diff state you need to close and re-open the file.
For files that are tracked by git, the basis of the diff is either the head or the index, depending on how you set the git_diff_target setting. The default is "index", which as the name suggests tracks the index, in which case the diff only shows you unstaged changes; changing to "head" would show you all uncommited changes instead.
If desired, you can set mini_diff to "auto", which tells it to only show the mini diff on files that are tracked by git; in that case anonymous files will never show diff indicators in the gutter.