Dependencies should get removed when the package that uses them is removed. Possibly there are other packages that you’re using that depend on the same dependencies as GitGutter does (those dependencies in particular are used in a lot of popular packages).
Technically speaking you could remove them from the
Packages folder manually, but if other packages are using them that could cause issues (which Package Control should eventually resolve).
If you want to check first, the OverrideAudit package (disclaimer: I am the author) has a command named
OverrideAudit: Package Report that lists all installed packages and dependencies; hovering the mouse on a dependency in the report shows you a hover popup that tells you what packages are using that dependency: