ST4 - high cpu loads while saving php files on Win* (possible issue in PHP package?)

[Orlmente]

ST4 has a strange behaviour and high cpu loads while saving php files.

Basically, when saving a php file, ST4 will spawn a new process that goes at around ~30% (25-35%) CPU load for about ~20 seconds, after that time the new process ends and CPU load turn back to original state.

• The issue persists even in a clean install.
• Actually, the .php file doesn’t even need to have some real php code in it.
• Saving any other file extension doesn’t trigger the issue, in the repo you’ll find html, css and js.
• Changing the syntax to anything else doesn’t trigger the issue.
• Whith bigger projects the problem is obviously more noticeable (higher cpu loads and longer duration).
• ST3 doesn’t seems to have this issue, even though there are some CPU load spikes upon saving the are usually both smaller (around +10%) and faster (~1 second).

I setted up a testing repository to replicate the issue.

[deathaxe]

The additional process is an indexer instance, which is triggered by saving the file as it modifes it on disk. It actually triggers for all files, whose syntaxes define indexed symbols, such as JS/CSS. Indexing them just finishes too quickly to notice. ST3 behaves very much the same way.

The only difference indeed is the amount of time, the indexer of ST4 needs to finsih.

It must be related with change syntax definitions. While ST3’s PHP “cache” is just 3MB, ST4’s is 19MB of size.

PHP includes or injects itself into CSS/JS/HTML syntaxes, which have undergone quite massive changes and extensions. Some of those syntaxes also use new features like “branching”, which are slightly more expensive.

What you see is probably just the result of higher syntax complexity.

[Orlmente]

Thx deathaxe for the indepth explanation.

Now my question is, is there a way to mitigate this, apart from disabling indexing (that whould be like removing the engine because your car uses too many gas)?
If it takes such load and time with just an empty file, you can imagine having a real project, and a normal workflow where you may hit save more often.
At the moment working with php is just a useless stress on the hardware (high cpu > higher temps > higher noise from the fans etc), and it’s what is keeping me from using ST4 as main editor (still work on st3), and that’s a shame

[deathaxe]

Not much an end user can do about it.

First we’d need to learn about the reasons in detail

1. increased effort to load and setup all syntaxes (per indexer process)
2. increased effort to parse files due to more complex syntaxes

First point may need some investigation from core devs as it might require some debugging tools to find out the reasons about what’s going on.

Second point may require some experiments with regards to changing PHP syntax definition itself. But my hunch is it is caused by much more complex JS syntax, which PHP injects its tags into. Not sure how to mitigate that, as all those changes are needed to support the weird/complex JS syntax and JSX/TS/TSX.

[bschaaf]

Check the indexer status and verify how many files are being indexed when you save only one. Are you using a networked filesystem perhaps?

[deathaxe]

I pulled the debug repo and tested locally. It doesn’t take 20 seconds to index the file but about 5-8secs with 100% on one core on a Ryzon 5600.

Other files, pure HTML, CSS and JavaScript files are indexed in less than a second without task manager even realy displaying CPU usage increasing.

It even happens when using old ST3 syntaxes (PHP, HTML, CSS, JS), even though indexing finishes slightly faster due to smaller syntaxes then.

The repo contains only one file which is indexed.

Even tried to comment out some bigger parts of PHP without noticible effect.

There’s something weird going on.

[Orlmente]

@deathaxe, exactly, even a project with a single .php file with trigger the issue, that’s why I was pointing out the PHP package, is the only package/language that presents the issue (an empty php file with only <?php phpinfo(); ?> was my first local test case)…
In my case, Intel i7 8700 doesn’t spike at 100%, always stable around 30% (as you can see from screenshots in the repo readme) but lasts 20/30seconds, that, in my normal workflow overlaps several saves from time to time (sometimes I hit save three or four times in half a minutes if I have small changes or I change my mind on what I’m doing).
If, as you said, is related with php inclusion in jsx/ts/etc would be nice to have some day a “clean” php package that doesn’t include such user cases, even though those languages are in hype nowdays, a lot of projects don’t rely on them, so a package without that “overhead” would surely be a nice thing to have.
If, in the other hands is something realted to the core indexer then, I hope this thread will lead to a solution.
@bschaaf, as @deathaxe wrote, just 1 file is being indexed, and no, no network fs is involved, just local files in a pro nvme ssd if that could helps.

[bschaaf]

If, as you said, is related with php inclusion in jsx/ts/etc would be nice to have some day a “clean” php package that doesn’t include such user cases, even though those languages are in hype nowdays, a lot of projects don’t rely on them, so a package without that “overhead” would surely be a nice thing to have.

There is no added overhead from that, as embedded syntaxes are lazy loaded in ST4. Considering the files presumably load plenty fast then they’ll index fast too.

Is the indexer actually running for the full 20s? What gets logged in the indexing status?

[Orlmente]

Yes, I presume.

When I hit save in console prints:
indexing [job 8]: spawning 1 workers to process 1 / 1 files
and only after the 20 seconds, when the new process goes away it prints:
indexing [job 8]: indexed 1 files

this screen was captured around halfway of the “saving process”, let’s say 10-12 seconds after hitting the save cmd.

[deathaxe]

Here’s a screencast.

It appears the indexer finishes quite fast but than hangs around stressing CPU for a while.

See also the small “collated in” time. … ok, this time is always very small even with very large projects, so I don’t think it works properly anyway or at least doesn’t count something useful.

Doing the same for html finishes quite instantly. As soon as the progress bar hits 100% it’s done.

Also tried with my rewritten JSP syntax which uses Java with heavy branching and complex rules. Works perfect and quick.

[bschaaf]

What is X:\?

[Orlmente]

For who may be intrested,
the conversation with bschaaf moved to discord chan, here what is missing.
“X:” is just a secondary internal sdd where I store working projects.
Even in “safe mode” the issue persists, but with a different output from the indexer, where it indexes all 6 files instead of just one in “regualr mode”.

[Orlmente]

Is the issue been addressed or there is any ETA on this?
I’m still on ST3 because of this, I want to upgrade but with the issue still present is a real no-go.

[bschaaf]

There’s a number of fixes in the latest build that may have fixed this, but I’m not certain.