Sublime Forum

Preprocessing files on load

#1

I’m currently trying to workaround a problem of ST2/3 with huuuuuge text files. And I’m not talking about a 10000 lines of code file, I’m talking about hundreds of megabyte of ascii, without any linebreaks. Such files a pretty usual on mainframes and I currently need to work with them.

Some are structured (EDIFACT for example) some are record based (fixed intervals of fixed length text).

Currently I preprocess these files for viewing externally (i.e. split EDIFACT by specific patterns and split the record based files into separate lines according to the record length).

Now it would be awesome if I could do that as a ST plugin so it happens magically. I imagine something like when opening a file beyond a certain size (>= 10 MB maybe) a popup asks me how to proceed and if split is intended, how to do it (by length or by pattern). Popup and splitting aren’t the problem, however I can’t find a way to intercept the file loading before ST loads it completely - because that takes ages. So I have to “fix”/preprocess the buffer before ST tries to do anything with it (parsing, or whatever else it does while loading a file).

Any ideas?

BTW. just as a side note: vim, scite, notepad++, gedit, kate, … all struggle with these kind of files. The only editor I have so far that loads them instantly is UltraEdit. So I don’t blame ST for not handling them well, I just want to fix it :smiley:

0 Likes
#2

You won’t get anything before “on_load” I’m afraid. The only thing you could possibly do is listen on “open_file” command calls in the “on_window_command” callback, but most files will be opened using the window.open_file() API or some other internal magic so you won’t get to use that in many cases.

0 Likes
#3

You could create a bash script and pre-process the file in your prefered language before calling subl xyz. if you want to do it in sublime, you could create a new view, parse the file async and fill the view buffer by yourself (of course that would only be a shallow view of your file)

0 Likes
#4

Loading it completely by myself sounds like the only way currently. Too bad. I had hoped to get it “generic” so it also works with plugins like SFTP.
Then again, I probably won’t open 2gb files on the fly via SFTP plugin.

I will have to think about this a bit more.

Thanks for your ideas :smile:

0 Likes