You could turn “auto character pairing as you type” off completely, by setting auto_match_enabled to false in your user preferences.
I had the idea that an alternative would be to override the default keybinding for ’ so that it won’t work in a clojure scope, but unfortunately it just falls back to the default which doesn’t have the restriction:
{ "keys": ["'"], "command": "insert_snippet", "args": {"contents": "'$0'"}, "context":
[
{ "key": "setting.auto_match_enabled", "operator": "equal", "operand": true },
{ "key": "selection_empty", "operator": "equal", "operand": true, "match_all": true },
{ "key": "following_text", "operator": "regex_contains", "operand": "^(?:\t| |\\)|]|\\}|>|$)", "match_all": true },
{ "key": "preceding_text", "operator": "not_regex_contains", "operand": "['a-zA-Z0-9_]$", "match_all": true },
{ "key": "eol_selector", "operator": "not_equal", "operand": "string.quoted.single - punctuation.definition.string.end", "match_all": true },
{ "key": "selector", "operator": "not_equal", "operand": "source.clojure", "match_all": true },
]
},