Binary File Reading Plugin

[mah]

I’ve downloaded this and it looks pretty nice to start with. A few things that I think would be nice (if they’re possible):

1. Obvious – having edits able to be saved for proper binary editing. I can see that this might be rather difficult though, unless you can control how the user applies edits (in which case, things could get very slow!)
2. A list of file extensions; if a file with an extension in the list is opened, the real file is closed and the hex_viewer representation is automatically opened.
3. An ability to add extension-specific representation. For example, if the extension is BMP, rather than just displaying a simple set of words and their ascii, display the BMP structure header and display the body in standard hex format. Perhaps this is well beyond the intention of Hex Viewer.

Is there a way to bind a key to the “Hex Viewer: Toggle Hex View” command?

[facelessuser]

1) Obvious – having edits able to be saved for proper binary editing. I can see that this might be rather difficult though, unless you can control how the user applies edits (in which case, things could get very slow!)

This is not impossible, but will be a little tricky. I will probably need to restrict some things and give warnings to malformed data. I am saving this for the very very end. The viewer will be released without edit capability and then I will revisit and see how I may go about adding this.

2) A list of file extensions; if a file with an extension in the list is opened, the real file is closed and the hex_viewer representation is automatically opened.

I have experimented with this, but ST2 doesn’t always fire the event for opening a file correctly…or at least as expected. It has caused me some issues to do this. I may revisit this again when I get the essentials in.

3) An ability to add extension-specific representation. For example, if the extension is BMP, rather than just displaying a simple set of words and their ascii, display the BMP structure header and display the body in standard hex format. Perhaps this is well beyond the intention of Hex Viewer.

Can you provide more in depth explanation to BMP structure headers? Is there an hex editor that does this sort of thing that I could view as an example? I would like to take a look into this before I give a yes or a no.

Is there a way to bind a key to the “Hex Viewer: Toggle Hex View” command?

Yes, you just use the standard key binding methodology. I will post an example in a bit.

[facelessuser]

Its a work in progress, but following features have been added.

[quote]1. highlight current byte - highlight a byte when your cursor is inside it
4. highlight corresponding chars - highlight the currently selected byte’s printable char in the ascii column[/quote]

It might be a little rough, but it works. I need to make sure and smooth it out and add expose some knobs to be tweaked in settings file.

Overall, not too bad.

[facelessuser]

Looked at it again this morning and made a couple of tweaks to the highlight code; should be pretty reliable now.

@mah oh yeah…shorcut. As promised. Just change the shortcut to what you want.

{ "keys": "ctrl+shift+b", "ctrl+shift+h"], "command": "hex_viewer" },

[bizoo]

The performance of file loading isn’t great right now.
A 2.25MB file take more than 11 seconds to load.

I made some changes in the code and the same file open now in less than 3 seconds.
I suppose that it’s acceptable and it look difficult to optimize the code a lot more.
Note that half this time is spend in the view.replace() call.

I need a little more time to cleanup the code and test it. After that I will put it on github and place a pull request.

[facelessuser]

I made some changes in the code and the same file open now in less than 3 seconds.
I suppose that it’s acceptable and it look difficult to optimize the code a lot more.

Yeah, having to go through python slows things down a bit; doing this in native code would be tremendously faster. I haven’t spent much time optimizing it yet, so this effort is much appreciated.

Note that half this time is spend in the view.replace() call.

I am afraid there is not much we can do about that. Well there might be other ways, but none that I am willing to traverse at this time.

[facelessuser]

Should have “Find Address” in tonight.

[facelessuser]

This was a big update. Bigger than I planned on.

-Better highlighting of bytes
-Configurable highlighting
-Show consecutive address range in status (also indicate additional bytes not consecutive with first group)
-Show total bytes in status
-Don’t highlight bytes if selection is out of range of data
-Find address input
-Hide hex settings and find commands when not in hex view
-Fix lint errors and general cleanup

Let me know what you guys think. I will probably add some configurable limits to the highlighting in the future.

[bizoo]

Just put a pull request on github for my ‘need4speed’ branch.

Really nice plugin, I don’t think there is lot more to do on it without some core implementation in ST2.

[facelessuser]

Just put a pull request on github for my ‘need4speed’ branch.

Thanks, this is much needed. I will be running some comparisons of the output, and take a look at the code. From a quick look, it looks nice, I am very pleased with this effort.

Really nice plugin, I don’t think there is lot more to do on it without some core implementation in ST2.

As a viewer, I think you are right. There is are still a couple things on my agenda though.

• configurable Endianness
• maybe highlight byte when a value in the ascii column is selected
• general testing before release
• keybindings
• access to settings and keybindings from menu

After these couple of things, I will probably do an official release.

Beyond Viewing
I do have a couple of ideas on how I might implement editing. I will play around with these ideas on the side after an official release. I imagine some indirection during editing will be needed.
I think I might try to implement a shortcut to enable an input panel with the hex content from a line. That way on submission, validity checks can be applied to the edits on that line before writing back to the buffer, and then a new ascii column line can be generated on the fly. I also will need to develop an export command. I will probably have to manage undos or just have a discard edit command.

Large copy and paste blocks might be a bit trickier, but even if the plugin only allows simple edits per line, I think that would be pretty sufficient.

[facelessuser]

Thanks to bizoo, we now have faster binary loading on the master branch. Thanks again.

[facelessuser]

Goofing around and added a pop up inspector panel to show info about the word, byte, etc that is selected (only works on single byte selection). It is not on the master branch yet, I need to cleanup a number of things.

The panel can be turned on or off and the endian of the value scan be configured to show it the data how you like.

I copied the naming convention from XVI32 (if you have a better suggestion, let me know). If I figure out the proper way in Python to display IEEE singles and doubles, I will add those as well.

[facelessuser]

All planned features are in. I finished up the reverse selection of hex bytes when ascii characters are selected. The old hex view panel is removed. Fixed a small issue were the address displayed was 1 byte off.

Also polished up the inspector panel and themed it a little.

I will commit it sometime this weekend, but it is not up on the master branch yet.

[facelessuser]

Hex Viewer plugin is now in testing phase. If you can help identify bugs (if any), it would be very helpful. No new features are currently planned before release. Changes are on the master branch.

Current commit added a lot of stuff.
-Add Hex Inspector panel
-Add Hex Inspector styling
-Adjust format of hex output to allow for better isolation of ascii table; this helps to quickly identify ascii to byte index referencing.
-Adjust hex output styling for new format
-Remove Hex view panel (only toggle to hex view now)
-Adjust defaults settings
-Bug Fix: address view was one byte off
-Add menu items for settings
-Remove debug code

[facelessuser]

The viewer is pretty much done. I got the float and double added to the hex inspector, some bug fixes and general cleanup. I added key bindings etc.

I am going to give it a bit before I add it to package manager. I am trying to get some form of editing going on in an experimental branch. If it isn’t going well by the end of the week, I will go ahead and add the viewer to package manager and continue the endeavor in the weeks ahead. If the edit coding is going great, I will try and get it in before the official release. I do have some rough “edit” code in right now, but no “export to bin” code yet. It looks promising though.

[facelessuser]

Played around with the code a little bit on my lunch break today; editing is going to be coming before official release.

Currently on the experimental branch (which is has not been committed yet), I have basic editing.

You can only make changes on one line at a time. You simply select the number of bytes you want to edit and press the edit shortcut key and you can edit the bytes directly. If you want to write an ascii string directly, you would simply give the prompt the string specifier “s:” like so:

s:this will be written as hex

As long as the hex bytes returned are actual hex and are no greater or lesser in number than what is selected, the buffer will be updated. Edits will be highlighted to show you there are changes. ( I would use the dirty indicator in the tab, except it is misleading; the dirty indicator is really referring to the formatted hex output, and not the actual file it is tied to, and if I enable it, it will keep prompting the user when the tab is closed to save the file. Saving the hex view tab mealy saves the formatted hex output as a file, and then you loose the association with the actual file. I cannot use the save method either, I need to export the data directly. If the Sublime Text API allowed for the canceling of “save” and “save as” on presave, I could directly tap into that and export instead of save giving the experience a more integrated feel, but that is not possible right now.)

After that you can use the export shortcut to write it back to file. I will see if I can come up with an “export as” method as well.

I have already done some basic testing of editing a file and writing the bin. It is looking pretty good. Block pasting will not be allowed, at least not in the first official release.

[facelessuser]

What’s that? Editing in Hex Viewer? You don’t say…

You can see edited bytes highlighted with red underlines. Also, clips of the UI elements.

The experimental branch is here if you want to try it out.
I still need to add some knobs in the settings to tweak and do more testing.

[facelessuser]

FYI, there was a regression with exporting to a new file; changing the export path when dumping the bin file does not currently work. The export path is not getting updated.

Use caution, with the experimental branch. All things will be in flux until I can get back to the main branch. The experimental branch is just to get feedback on the general UI and such while I am polishing it up.

[facelessuser]

Didn’t have as much to do as I thought. Hex editing is now on the Master Branch. I made sure the exporting to a different file works. I also added a number of confirmation panels to make sure you don’t change the view if you have edits. I can’t really protect the hex edits if you accidentally close the file. ST2 doesn’t really allow you to cancel events like closing etc (I wish it did). If I leave the dirty indicator, it can be confusing to users when they are prompted to save; save does not export the bin file, but simply saves the text representation of it to a file. I might implement a failsafe to export to a temp file if a close is invoked while the hex view is dirty and then alert the user and ask them if they want to keep it or not, but that will be later in the week.

I did add overwrite protection, so if the file exists, it will make you confirm you want to overwrite. The master branch should be safe to use.

At this point, I am just trying to shake out any bugs left before release, that and maybe add the failsafe I talked about earlier.

You may need to restart ST2 if you drop this in while ST2 is running. This is due to dependencies.

Let me know if anyone runs into issues.

[mah]

[quote=“facelessuser”]

3) An ability to add extension-specific representation. For example, if the extension is BMP, rather than just displaying a simple set of words and their ascii, display the BMP structure header and display the body in standard hex format. Perhaps this is well beyond the intention of Hex Viewer.

Can you provide more in depth explanation to BMP structure headers? Is there an hex editor that does this sort of thing that I could view as an example? I would like to take a look into this before I give a yes or a no.[/quote]

http://www.hexworkshop.com/ is the only editor I’ve seen that does this. The screenshot they have on the front page sort of shows the feature – the lower left is displaying the contents of some pre-defined structure. Looking at their screen shots page at http://www.hexworkshop.com/screen_shots.html I see they’ve done a lot since the last time I eval’d their tool.

[quote]

Is there a way to bind a key to the “Hex Viewer: Toggle Hex View” command?

Yes, you just use the standard key binding methodology. I will post an example in a bit.[/quote]

Got it; thanks!