Sublime Forum

File save on 7000 file find&replace: ST3 fast, ST4 very slow

#1

Hello,
I have been using my licensed ST3 during the past about three years, for doing long regexp based search&replace over >7.000 code files, totalling about 7 MLOC of 1-byte-character text in Windows 1252 encoding.

I moved to ST3 when I verified that with it the complete cicle (find-in-files, replace, save and close files) speeded-up from the about 15’ in my previous editor, to <2’ . It has been great on the task all this time.

But now I was thinking on changing to ST4, just as I have changed to Windows 11. And have been testing it (in safe mode, to ensure nothing else is being tested). And the performance falls to >120’ of the same task that ST3 performs in 2’ .

I have tested in two workstations, my newer Windows 11 with 16 core, 32Gb. And the previous Windows 7, 8 core, 16Gb. In both, same test (ST3 and ST4), and the results are the same (the Windows 7 machine both test slower than in the Windwos 11 one, but ST4 very much slower than ST3 on both).

I tested with a global replace of ^\h+ with nothing. Loading the 7.000 files to memory takes well less than 1’ in ST3, applying the change more or less the same, and saving takes the most part, to total around 2’.

In ST4, load & change takes about 2’, but saving takes more than 2 hours.

Just in case it may be useful, may it be related to the messaging and logging?:

  • When ST4 it writes the names of the files being saved (in the bottom status bar), I can read one by one. THe change from the circle image (“save pending”) to x (“no pending chanes”) is also easy to follow tab by tab.
  • ST3, OTOH, writes faster the names and the changes in image from circle to x, it evens stops and ceases to respond to Windows events … but it ends up the task much faster.

But this can perfectly be a consequence, not a cause …

I doubt that there is really a performance decrease in this so basic part of ST. May there be some config otion that the defaults in ST4 differ from ST3? Or something I am missing …

I write to ask for help in getting the performance back, and then expedite the movement to ST4.

Thank you!
Ray

0 Likes
#2

In general “Find in Files” should be faster as searching and replacing content is performed concurrently in background worker threads, instead of doing so one-by-one.

ST4 saves files files is asynchrounously to prevent GUI from blocking, which ST3 doesn’t.

I wonder wheter that’s related with long saving times.

To verify, you could try to add a “Save All” command, which saves synchronously like ST3 to Packages/User/Default.sublime-commands and run it from Command Palette.

[
    { "caption": "File: Save All (synchronous)", "command": "save_all", "args": { "async": false } },
]

Note: GUI may freeze, if it still takes too long.

As bugs are tracked via GitHub, filing an issue at https://github.com/sublimehq/sublime_text/issues would be useful.

If the files you are experience issues with are open source, publishing them might help to reproduce and debug the reason for that slow-down.

1 Like
#3

Thank you for the answer!
These are the result’s of this weekend’s tests with your suggestion. The benchmark has been replacing \A(.) with \1 in 300 files.

  • ST3: instant save-all/close-all
  • ST4 safe-mode with the extra command (File: Save All): cannot do the test, as when launching in safe mode, the C:\Users\Admin\AppData\Roaming\Sublime Text (Safe Mode)\Packages\User folder is emptied.
  • ST4 with the extra command: slow save. I can read each file’s name in the status-line as they are saved. No perceivable difference with the normal “Save All” command.

It looks like the status report of the multithreaded file save may be the reason. May it be a FIFO receiving the status report of the saving workers, and introducing the delay with the report and change from dot to x in the tab icon?

I will post the issue to the Github list you suggested.

Thank you!

0 Likes
#4

ST4 safe-mode with the extra command (File: Save All): cannot do the test, as when launching in safe mode, the C:\Users\Admin\AppData\Roaming\Sublime Text (Safe Mode)\Packages\User folder is emptied.

Simply add the Default.sublime-commands file after ST has started. ST hot-reloads resources.

0 Likes