You could potentially create a plugin that does this. Here’s a first draft of something that works (mostly; see below):
import sublime, sublime_plugin
# The last view we were in, which is None if there isn't one yet
# or it was closed
lastSeenView = None
class ToggleToLastViewCommand(sublime_plugin.WindowCommand):
    def run(self):
        # Jump to last seen view, if any
        if lastSeenView != None:
            self.window.focus_view (lastSeenView)
class ToggleEventListener(sublime_plugin.EventListener):
    def on_close(self, view):
        global lastSeenView
        # If the view being closed is the view we're toggling to, don't try
        # to toggle there any more.
        if view == lastSeenView:
            lastSeenView = None;
    def on_deactivated(self, view):
        global lastSeenView
        # If the view being deactivated isn't a file, do nothing
        if view.file_name () == None:
            return
        # If the view being deactivated is the view that we already think is 
        # the last view, then do nothing.
        if lastSeenView == view:
            return
        # Save this view as the last seen view
        lastSeenView = view;
Saving in (for example) Packages/User/toggle_to_last_view.py, you can bind it with something similar to:
    { "keys": ["alt+q"], "command": "toggle_to_last_view"}
(Note: The command name is derived from the name of the method that implements the command, not the name of the file that you save the code in).
As I mentioned above, this only “mostly” works. In particular, it gets confused if you open up the command panel or if you give the focus a panel (e.g. the console). As such, this is more of a proof of concept.
Also, this (crudely) checks if a view represents a file by checking if view.file_name () returns a value, which means that this only works for files that have been saved to disk at some point.
Replacing that test with some code that can accurately detect if a view represents a file (saved or otherwise) and not a panel or an input widget would make this more robust.