Sublime Forum

Unicode Replacement Character � (U+FFFD)

#1

I am currently working on a c++ project where it sometimes happens that typing a space is interpreted as the unicode character �. This results in the compilation crashing which is rather annoying. I then return to the code I manually change all occurrences thereof and it is interpreted as wanted. At a later point, when I again change some code it randomly (I say randomly, because I have not yet found why and when it happens) interprets it as a space or �. I am using Ubuntu 18.04.4 LTS and Sublime Text Version 3.2.2. Do you have any idea why it happens and what I have to change such it does not?

0 Likes
#2

Packages like UnicodeCompletions sometimes tend to trigger if space is pressed, but this normally causes visible characters to be converted to unicode code points.

The best chance to find the issue, was to enable command logging by sublime.log_commands(True) and check the console output once such an unicode space was added. With some luck, you can find the plugin/command causing that, then.

0 Likes
#3

I was able to pinpoint the commands leading up to my situation. I was writing the following line of code by hand and it inserts the characters after the equal sign.

Shadow =��combineShadows(Shadow,ScopeLabel,Pos);

I actually began with

combineShadows(Shadow,ScopeLabel,Pos);

and prepended it with

Shadow =��

The commands are the following:

command: drag_select {“event”: {“button”: 1, “x”: 522.024902344, “y”: 235.879241943}}
command: toggle_comment {“block”: true}
command: drag_select {“event”: {“button”: 1, “x”: 409.985229492, “y”: 226.698822021}}
command: insert {“characters”: “\n”}
command: insert_snippet {“contents”: “($0)”}
command: insert_best_completion {“default”: “\t”, “exact”: false}
command: left_delete
command: move {“by”: “characters”, “forward”: true}
command: move {“by”: “words”, “forward”: false}
command: move {“by”: “words”, “forward”: false}
command: move {“by”: “words”, “forward”: false}
command: move {“by”: “words”, “forward”: false}
command: move {“by”: “words”, “forward”: false}
command: left_delete
command: save
command: drag_select {“event”: {“button”: 1, “x”: 592.408691406, “y”: 367.914794922}}
command: drag_select {“event”: {“button”: 1, “x”: 625.93762207, “y”: 408.361328125}}

0 Likes
#4

Ok, I found the reason. Using space while pressing shift creates that unicode character. Since I often write uppercase letters after a space I seem to press it prematurely which causes it to appear. I will google how to disable that but if you already know it, I would appreciate it if you could point it out for me :).

Edit: Here, some user had a similar problem. Might be related to mine. https://apple.stackexchange.com/questions/219686/disable-spaceshift-in-sublime-text

Edit*: I eventually found the root of the problem. On my system, shift+space is mapped to an input called “nobreakspace”. This is translated in sublime into that unicode character. So, I solved it by mapping that key combination to space (using xev and xmodmap on Ubuntu).

1 Like