Hex Viewer (a hex viewer and editor)

[facelessuser]

About
Hex Viewer is a plugin for Sublime Text 2 that allows the toggling of a file into a hex viewing mode. Hex Viewer also supports hex editing.

**Installation **

• Download is available in Package Control or you can download directly and drop into your Sublime Text 2 packages directory (plugin folder must be named HexViewer)

• You may need to restart Sublime Text 2 after installation

Features

• Toggling a file into hex view mode

• Search bytes by address

• Customizable highlight of byte and corresponding ASCII representation

• Customizable byte grouping and bytes per line

• Hex inspector to view current selected bytes as different unit types (endianness is configurable.)

• Display total selected number of bytes and addresses of first group of consecutive bytes in status bar

• Hex editing

• Checksumming of files

• Generate a hash from a string

• Auto open binary files in Hex Viewer (disabled by default)

Source Code and More Documentation
github.com/facelessuser/HexViewer

Version 1.6

• Report Tiger-192 as big endian as most hashing programs do

Version 1.5

• Allow the hashing of current selection(s). Multiselect regions’s content will be combined and evaluated together
• Update hash progress quicker
• Chunk hashes entered in the input panel in case very large amounts of data are pasted in

Version 1.4

• Fix issue where some selections will select an extra hex char that is not really selected
• Update missing item from changelog

Version 1.3

• Decrease debounce threads to one

Version 1.2

• Auto open specified binary files in Hex Viewer (disabled by default)

Version 1.1

• Add MD2 so Windows OS can find it (Mac and Linux do not have this one; Mac uses MDC2 for some reason)

Version 1.0

• Allow unicode text in “Generate Hash”

Version 0.9

• String to hash not always working now fixed
• Fixes for when certain hashes are not found

Version 0.8

• New hash algorithms: Whirlpool and Tiger-192
• Checksums are now threaded
• Exposed new command to tap into hashes called “Hex Viewer: Generate Hash”
• Rework logic to determine which hashes are available

Version 0.7

• New hash algorithms from openssl: sha, mdc2, md4, ripemd160 (dynamcially added if system supports them)
• Rename “Get Checksum” command to “Run Checksum”
• When invoking shortcut or command for checksumming, show quick panel with all valid hash algorithms to choose from

Version 0.6

• Commands accessible from menu Tools > Packages > Hex Viewer
• Fix checksum not responding to settings change
• Add CRC32 and ADLER32 hash algorithms for checksums

Version 0.5

• New progress animation in status bar when reading bin file
• Setting to enable or disable highlight throttling (default is disabled)
• Don’t display incorrect byte count when highlight is throttled
• Get hex char range only once
• Better highlighting of edits
• A little cleanup

Version 0.4

• Hide Inspector tmLanguage file
• Chunk checksum and export for faster, more linear performance with large files
• Threaded binary file reading and show progress of current bytes parsed in status bar
• If hex view command is issued while a file is being converted, abort the conversion
• Simplify logic with hex view; always close original tab and load in new one opposed to conditional logic (easier with threading)
• Add setting to throttle highlighting selected bytes
• Add divider in settings menu

Version 0.3

• On export, rename tab to new export name
• Fix checksum failing when passing certain data through Sublime Text run_command API
• Account for NaNs in floats and doubles in Hex Inspector[/list]

Version 0.2

• Fix issue where large blocks get highlighted when the end of line is selected
• Add checksum command (also runs when export is made)

Version 0.1

• Initial release

[facelessuser]

If you would like to have upper or lower nibble stand out, you can add this to your theme file and configure the nibbles’ color to compliment each other.

<dict> <key>name</key> <string>Upper Byte Nibble</string> <key>scope</key> <string>raw.nibble.upper</string> <key>settings</key> <dict> <key>foreground</key> <string>#66D9EF</string> </dict> </dict> <dict> <key>name</key> <string>Lower Byte Nibble</string> <key>scope</key> <string>raw.nibble.lower</string> <key>settings</key> <dict> <key>foreground</key> <string>#FFFFFF</string> </dict> </dict>

I am purposely not scoping this by default because certain themes might not have good contrast colors for whatever I scope them as. This way, you can add it if you want it and have complete control to make sure they stand out how you like them.

[facelessuser]

FYI, there is a small issue with checksumming from Hex Viewer. I already have a fix that I will commit tonight.

Basically I cannot pass hex data through the run_command api, at least not how I am doing it. If any of the hex data is not a printable ASCII, it will fail. This is easily remedied by postponing the conversion of the file data content to hex until right before checksum is ran.

I just had this nail me at work today .

[facelessuser]

New version posted:

#Version 0.3

• On export, rename tab to new export name
• Fix checksum failing when passing certain data through Sublime Text run_command API
• Account for NaNs in floats and doubles in Hex Inspector

[facelessuser]

I am in optimization mode now. If you have noticed the speed of checksumming to be abysmal on very large files, don’t worry, I have noticed as well.

I was able to improve the checksum speed by orders of magnitude on large files and probably exporting as well. I am cutting down anywhere we are applying code to extremely large buffers. This will allow python not to bog down trying to manage such large buffers. So in the next little bit, I will try to optimize all big processes (or at least attempt to further optimize). Loading is pretty good right now, I will see if I can push it a bit more. Checksumming and exporting should be much better. I will do some tests on restoring a tab speeds on large files, and also see if I can throttle highlighting on large selections.

Whether it happens before next commit or not, I will also be threading heavy hitting processes, and hopefully provide a cancel option in case you were silly enough to try and edit a 100MB binary file.

I will at least have the optimized checksumming/exporting in a commit tonight. I just need to get some more real world testing in.

[facelessuser]

#Version 0.4
-Hide Inspector tmLanguage file
-Chunk checksum and export for faster, more linear performance with large files
-Threaded binary file reading and show progress of current bytes parsed in status bar
-If hex view command is issued while a file is being converted, abort the conversion
-Simplify logic with hex view; always close original tab and load in new one opposed to conditional logic (easier with threading)
-Add setting to throttle highlighting selected bytes
-Add divider in settings menu

[facelessuser]

A couple of notes on the last commit:

I am only threading the reading of the binary file currently, so the editor still hangs on large files when loading the formatted hex data into the view buffer (I am using the replace API command which is the process that takes the longest). I will probably thread this portion as well when I am sure I am accounting properly for all things that could happen when this is handed off to the thread. I may even break up the loading of the data into the view buffer by chunking the data and using the insert API command instead. This might add a little overhead to the loading, but it would allow for a load to be canceled during the buffer load process as well, and also allow for some visual indication of how far along the buffer writing is.

I have throttled the highlighting of selected bytes, mainly because highlighting them requires me to parse each selection and then find it in the ASCII table (time consuming). You will notice that the byte count and displayed addresses in the status bar are tied to the bytes processed and highlighted, so the range of selected bytes and byte count will only represent currently selected bytes. In the next commit, I will probably just display the first address and not display the byte count (until I come up with a quicker way to count the bytes independent of byte highliting) if the byte highlight threshold has been reached.

I am currently not threading writing the binary file and check-summing the binary file (these are very quick compared to loading the binary file, but the time they take to finish is linearly proportional to the size of the file). I may thread these in the future when I finish addressing the loading of the file threading.

[facelessuser]

New version posted:

A couple of notes:
I tried to thread the loading of the content to the buffer, but that is not reasonably possible. Edits to the view buffer must be on the main thread. So basically, the parsing of the bin file is threaded allowing you to cancel during that time, but as soon as the content is being written to the buffer, you just have to wait.

I also have decided against threading any other processes. The bin file loading is by far the weightiest process, and I have done about all I can there. Checksumming and bin file exporting is much much faster than loading. If you can wait for the file to load, checksumming and export will be no problem.

A couple of frivolous things made it in: new progress animation in status bar when parsing binary file, and better highlighting of edited bytes.

I also added another setting to turn off highlight throttling; it is now off by default. I originally added it when I thought the highlighting was causing the editor to lock up if you selected very large blocks (1500+ bytes). Though, while it will pause to highlight the bytes, the locking up is actually caused by multi-select. I turned off my plugin and tried to do a large multi-select in a normal code file, and the whole editor froze up and never gave me back control, so it is not the plugin. So if you are doing large multi-select blocks, you can enable throttling if you choose, but by default I have it disabled (I don’t really do large multi-select blocks often, I mainly do large single selection blocks which are no problem at all; moderate multi-select blocks are also fine).

[facelessuser]

#Version 0.6

• Commands accessible from menu Tools > Packages > Hex Viewer (thanks to jsadeli)
• Fix checksum not responding to settings change
• Add CRC32 and ADLER32 hash algorithms for checksums

Somewhere along the lines I broke checksums responding to the settings file for the algorithm; this is now fixed. Also, I added some obvious checksumming algorithms that were not there before: CRC32 and ADLER32.

That should be all of the major things I wanted to get done. Let me know if any bugs turn up or you have any feature ideas.

[facelessuser]

Got some new checksum hashes in: sha, mdc2, md4, ripemd160.

I already had the ones from hashlib, and zlib, but I pulled in the ones that are included with openssl. These may differ on some systems, so I currently check if they are available and only include them if they are. If I find some other ones that are available that I am not including, I will add those.

I was so close to getting whirlpool algorithm in here. The algorithm is public domain, and I had it working, but I could not get it working yet when the data is chunked; the hash comes out wrong. So, to get the right hash currently, the entire file’s data is loaded into one big buffer and the hash is applied on that, and since it is 100% python, it takes a long time on big files…but it worked. Due to this, I have not included it. If I can deconstruct the algorithm enough to figure out how the get chunked data to evaluate the same, I will include it later.

Since the current supported hashes could vary on certain systems, when the checksum command or shortcut is invoked, I will now show a quickpanel populated with the valid hashes that can be used for selection.

[facelessuser]

Got whirlpool hash working with chunking. I got the python port of the hash from this one site, and it looked pretty exact with the C source. The guy who did the python port I guess assumed that the adding to the hash function worked the same as update in hashlib, so he wrapped similar, but this just wasn’t the case. With a little analyzation, I was able to adapt the algorithm to mimic hashlib’s update and allow for proper hashing of chunked data. Now the hash comes out correct whether chunked or not.

It works pretty good checksumming the average source code files, but if you try this on a 3MB file, it will take a bit because it is straight Python, and the algorithm is fairly complex. Because of this, I went ahead and added checksum threading. So if you are checksumming a really large file, you will see the progress in the status bar. If you want to cancel, just run the checksum command again, and the current thread will be canceled.

I am only adding these hashes because I am finding it fun figuring these out. I may allow in the future to tap in to these functions to generate hashes from selected strings etc, which is useful if you need to generate hashes in your source code. Maybe I will split out the hash code into its own plugin; we will see.

I will probably commit the whirlpool hash along with the checksum threading tomorrow.

[facelessuser]

I was going to port it myself, but someone already ported Tiger192 hash algorithm. Again, it wasn’t allowing chunking, but I added that in and wrapped it like hashlib. So, I have now postponed the commit till tomorrow. I will commit both whirlpool and tiger192 hash for checksumming, and I think I am done playing with hashes for now.

I may add in some kind of command that will allow you to enter a string and return your desired hash. It isn’t really directly related to hex editing, but since I spend this time adding in all of these hashes, might as well give access to them.

[facelessuser]

#Version 0.8

• New hash algorithms: Whirlpool and Tiger-192
• Checksums are now threaded
• Exposed new command to tap into hashes called “Hex Viewer: Generate Hash”
• Rework logic to determine which hashes are available

So the new hashes are added, and I added a new command to generate a hash from a string at any time you need. You do not have to be in hex view to do this; as a matter fact, it really has nothing to do with the hex viewer, I am just tapping into the hash framework I put in because I think it might be useful to some.

[facelessuser]

Missed some commas in windows and linux keymaps…I guess you can tell I have been testing in Mac recently .
Apparently, it was reported to me this small mistake, prevented ST2 from starting in Windows. It kind of surprises me that this kind of error could take down the entire editor in windows, but I shouldn’t have missed those commas either.

This should be resolved now.

[facelessuser]

Now that I am done goofing off with hashes, I added a useful feature.

I just added auto opening of specified binary files in Hex Viewer. If you directly open a file, select in the side bar, or select it via “Go To Anything”, it will open your will open files defined by patterns in the setting file automatically in Hex Viewer.

Direct opening should open without delay, but I have defined a 2 second delay for previews of binary files; this prevents opening a Hex View tab if you are just passing a binary file in “Go To Anything”. If the first “Go To Anything” selection is a binary file, if you linger for more than two seconds, a Hex View tab will be opened (you can increase this delay via the settings file). After you start moving around in the “Go To Anything” menu, you know longer have to worry about lingering.

Currently it is disabled by default. I am going to wait and do more testing before I enable it by default, but feel free to try it out.

[facelessuser]

I may have spoken prematurely. I think I have the logic worked out now. I removed the delay. You should be able work as normal without fear of accidentally triggering Hex View. Previews should now function as they always have, and when the file is actually opened, Hex View Auto Open will kick in. Try it out and let me know what you think. Just enable it in the settings file, and define your file patterns.

[facelessuser]

Version 1.5

• Allow the hashing of current selection(s). Multiselect regions’s content will be combined and evaluated together
• Update hash progress quicker
• Chunk hashes entered in the input panel in case very large amounts of data are pasted in

Decided to add the ability to select text, and hash it. It handles multi-regions too. It just adds the regions together in the order they come in the file and hashes them as one entry.

[facelessuser]

Version 1.6

• Report Tiger-192 as big endian as most hashing programs do

Was always a little curious why tiger hash seemed to give me valid results, but differed from every hash program I have compared it to. The endian convention I was using was different from what everyone else used…go figure. The hash was still completely valid, and matched the tiger hash web site results, but every hash tool in the world uses the other convention. Well, now I match them as well to avoid confusion.

[facelessuser]

Version 1.7

• More responsive highlighting
• Turn off gutter icons for highlighting by default (does not seen needed)
• Convert tabs to spaces in settings file

Pulling in the more responsive highlighting from BracketHighlighter.

[Callidus]

Would it be possible to have the hex editor open AND be positioned to where the cursor or selection currently is? E.g. Say I put the cursor at the end of a line, then open the Hex Viewer, it should open to that same spot at the end of the line (so that I can view 0d 0a).

Currently it always opens with the cursor at the start of the file.