Sublime Forum

Performance of automatic file reloading

#1

Hi there! I’m hoping to use Sublime’s build system to run an automatic code formatter for individual files. In this case, I’m using hackfmt, a command-line formatter for the Hack language. This tool takes the name of a file as its argument, and overwrites the same file with the formatted text.

When run directly from shell, hackfmt takes around 0.03 seconds, i.e., nearly instantaneous. However, when I execute it using Sublime’s build system, it takes 0.7 seconds, and there is a noticeable, distracting blip. As a reflex, I save at almost every edit I make, which means this delay is pretty significant.

I’m curious if there’s a way to profile this operation to see what might be going on here. It’s possible other plugins of mine are interfering, or maybe that’s just how long it takes Sublime to reload a file when its contents have changed on disk. Similar tools like the automatic Golang formatter run via the GoSublime package feel nearly instantaneous.

For reference, I’m running on MacOS High Sierra, and have the following build config file:

{
  "shell_cmd": "/path_to_hackfmt/hackfmt $file"
}
0 Likes
#2

… which is best practice for many programs, but not for ST as it even keeps unsaved changes save.

The build system is not made to run such kinds of background tasks. It needs to read the whole environment variables and merge it with settings from the *.sublime-build file, than starts some threads to read from stdout of the finally created process asynchronously. Quite a lot of stuff to do, which is not needed when running an app from the terminal directly.

But running in 0.03 but 0.7s is not the question when running builds as they might take much longer anyway.

What you’d need was a plugin, which runs your formatter directly. Many plugins do so by either passing the view’s file path to an external app doing something with it or copy the content of the unsaved view to a temporary file, which is than passed to an external app. The later one would imply to read the temporary file afterwards and replace the whole view’s content with it. This way the autoformat would even work without the need to save the file bypassing the whole auto-reload mechanism.

0 Likes
#3

Thanks for the response! I took your advice and wrote a plugin, and after trying various angles, it seems like the latency comes mostly from Sublime’s auto-reload. Regardless of whether the context is a build system or a plugin, there is noticeable latency when invoking the command-line formatter on the filename in view after a save.

However, the latency goes away if I use a pre-save listener that performs the following:

  1. Copy the view buffer into a tempfile
  2. Run the in-place formatter over the tempfile
  3. Replace the view buffer with the updated tempfile contents
  4. Delete the tempfile

Despite the extra file manipulation, the entire workflow runs about three times as quickly.

0 Likes
#4

… which relies on the performance of the filesystem’s notification API.

0 Likes
#5

… which relies on the performance of the filesystem’s notification API.

I think in this case, that’s not the full story. I’m using MacOS, which has a pretty fast FS notifications.

Take this as a simple example. I have a short notification-watching script called fswatch.js:

const fs = require('fs');

fs.watch('./', (e, path) => {
  console.log(`\nB: ${Date.now()} - ${path}\n`)
})

Then I ran this:

$ node fswatch.js &
[1] 69105
$ node -e 'console.log(`A: ${Date.now()}`)' && echo 'test' >> foo
A: 1533492857110
$
B: 1533492857128 - foo

According to those numbers, there’s an 18 millisecond delay (probably worst-case, since time A is sampled within a process that strictly precedes the file modification) between the file system change and the ability for another process to receive a notification. Node’s fs module doesn’t do anything fancy; it uses whatever native FS event system is available.

I think the takeaway here is that the latency reported in the parent message is coming from somewhere within Sublime Text, although I don’t think this test is super conclusive as to whether it’s intrinsic to the app itself. It’s possible that this is configuration-specific, or introduced by an unrelated plugin that is doing something with high overhead in a synchronous manner on file load events.

0 Likes