All of the customization you’ve done to things like your preferences, key bindings, settings for packages you installed and so on are stored in your User package. You can use Sublime Text > Preferences > Browse Packages... from the menu or the similar command in the command palette to find where that location is if you’re not sure where it is.
In the general case, shipping that from one machine to another is enough to have that machine set up the same was as a previous computer. What this doesn’t cover is packages that you installed though, since they are stored outside of the User package.
If you used Package Control to install your packages, you’re good to go (if not, you did a lot of extra work 
) on this front. One of the things that Package Control does is install any packages that are set in its settings (in your User package) that are not currently installed. So generally speaking , installing Sublime, deploying your customized settings into the User package and installing Package Control are enough for the system to bootstrap itself up to the way you want, although you might need to restart Sublime in the process.
The location where Sublime stores all of the data on things like your settings, the packages you have installed and so forth is known as the Data directory, and this directory is left alone when you uninstall or upgrade Sublime. This means that you can reinstall Sublime all you want without disrupting anything. It also means that if you think uninstalling and reinstalling Sublime is going to fix a problem you might be having with it, it’s probably not.
Having another copy of it on your computer may be of limited use since if your computer failed, it would probably be gone regardless.
In any case, many users solve both of your questions here by syncing the contents of their User package with a cloud service or a git repository or such like (make sure you keep it hidden from the public in case it contains any personal information though).
There are some docs on syncing on the Package Control website that cover this in a less verbose manner than I’ve laid out here, including instructions on how to use Git or DropBox to keep your settings synced and safe.
