Sublime Forum

**mattst** · January 19, 2019, 3:45pm

How should I go about replacing the default paste_from_history.py plugin?

I’ve written my own version of paste from history called PasteFromHistoryEnhanced, its changes are listed at the bottom of the post for interest’s sake only (no relevance to my question).

After making my alterations I realized that both the TextCommands, PasteFromHistoryCommand and my PasteFromHistoryEnhancedCommand, have an EventListener derived ClipboardHistoryUpdater class monitoring the clipboard, processing the display name, and storing the last 15 clipboard’s contents in memory.

So I renamed my plugin file as paste_from_history.py (same name as in the default package), placed it in my ST config User directory, and renamed my PasteFromHistoryEnhancedCommand class as PasteFromHistoryCommand (again the same as the default).

BUT… looking in the console I find this among the rest:

reloading plugin Default.paste_from_history
reloading plugin User.paste_from_history

There is NO unloading plugin Default.paste_from_history line.

I’m pretty sure this means that both the default and my paste from history plugins are running, wasting time and memory.

What setup do I need to ensure only my plugin is running?

I don’t want to ‘physically’ replace the paste_from_history.py in the install directory’s Packages/Default.sublime-package archive.

Thanks.

FOR INTEREST’S SAKE ONLY…

The changes made so far by PasteFromHistoryEnhanced are:

User chooses the paste to text from an overlay, show_quick_panel(), instead of a pop-up menu.
Ensures that text copied or cut with my own copy and cut plugins gets added to the clipboard history list.
is_enabled() replaced by a ‘clipboard history empty’ status message.
Allows operation from widgets, i.e. the console and input panels.
Various changes made to the display text; e.g. the addition of a line count (if >1) and a longer display length.

**addons_zz** · January 19, 2019, 4:26pm

Your best solution

You can put a copy of your paste_from_history.py with the command named PasteFromHistoryCommand into the loose packages directory Default/paste_from_history.py. However, just this by itself, will not cause the default paste_from_history.py to be reloaded/overridden automatically by Sublime Text. You still need to manually reload the Default/paste_from_history.py after all plugins are loaded.

This happens because on newer versions of Sublime Text, it does not accept packages overrides for the Default package files on the user loose packages directory. When loading the loose packages directory Packages/Default, it only loads the plugins which are not an override. For example, if you put a new plugin named my_cool_new.py on Packages/Default, it will load correctly on start up. But if you put your paste_from_history.py there, it will be ignored, because it is an override of a Default package file.

In order to make Sublime Text stop ignoring your paste_from_history.py on Packages/Default, you need to manually call sublime_plugin.reload_plugin("Default/paste_from_history") (without the .py extension) from another plugin, after all packages being loaded.

With this, you can at least remove the original plugin paste_from_history.py from memory, replacing it by your enhancement. You package will be required to copy the paste_from_history.py to Packages/Default and reload it after all plugins are loaded. You do not need to call sublime_plugin.unload_module("Default/paste_from_history") because if you inspect the code, you will notice sublime_plugin.reload_plugin("Default/paste_from_history") already does that your you, it just do not say so on the console.

After writing the last paragraph, I think you can get away with less efforts. Instead of coping your paste_from_history.py to Default/Packages, you should be able to just call sublime_plugin.unload_module("Default/paste_from_history") after all plugins are loaded. This should remove the paste_from_history.py from memory, and only keep your version of it, which can be named as you which, and does not need to be placed in Packages/User or Packages/Default.

Your worst solution

I have no idea about any other mechanism to stop Sublime Text from originally loading the default paste_from_history.py, unless trying to modify the Default.sublime-package. Which will probably fail in most Linux installations because the Sublime Text default files should not be able to modify due permission restrictions. Nonetheless, this kind of hack should work fine on Windows portable installations of Sublime Text, where no permissions restrictions apply, other than the traditional file lock Windows apply to its files like:

This is another problem which will complicate your life if you try to “monkey patch” the Default.sublime-package, because all .sublime-package files are locked while Sublime Text is running. And you cannot add Default to the user setting ignored_packages because Sublime Text ignores you, and does not ignore the Default package. This used to work on older builds of Sublime Text as build 3114, however on newer builds as 3176, you cannot do this anymore.

Even if you cannot add Default to ignored_packages setting, you can do specific platform hacks like using handle.exe tool on windows to unlock the Default.sublime-package file while Sublime Text is running, but it would require you to ship that executable with your package.

**ThomSmith** · January 19, 2019, 5:08pm

Do you have reason to believe that this is non-negligible?

As @addons_zz mentioned, it’s possible to unload a plugin via sublime_plugin.reload(), but that’s an undocumented private API. Python’s module system isn’t really designed for module unloading, and it’s very easy to break things in subtle and confusing ways. I’ve been doing work on AutomaticPackageReloader for a few weeks, and although I think I have a reasonably solid understanding of the module system’s internals and how Sublime uses them, I still occasionally break things. The risk is especially high for a Default plugin, because other packages may reasonably rely on that code being loaded. Unloading Default.paste_from_history would modify global state that is shared between all packages.

Unless there is a measurably significant performance impact, I would leave Default.paste_from_history where it is.

**mattst** · January 20, 2019, 6:48pm

Thank you both for your replies.

I had been hoping for a solution along the lines of:

Place your paste_from_history.py file in ../PATH_TO/CONFIG_ST3/Packages/Default/ and the default plugin will be overridden.

No, the effect of both plugins running will be, as you suggest, negligible. I tested by copying a 1 MB file into the clipboard and there was no noticeable effect, while a test of 10 MB of text caused the cursor to stop blinking for (best guess) 1-2 seconds which I suspect came from the default plugin running the display name regexes on 10 MB of text, rather than my plugin which limits the text size that the regex operations are run on to the first DISPLAY_LEN (60 in my plugin’s case) number of non-whitespace characters. Clearly copying 10 MB worth of text into the clipboard is a v. rare event (don’t think I’ve ever done so before that test), even copying a 25,000 byte file into the clipboard would be a rare event for me.

So I’ll probably leave both plugins running since there’s no easy way to override the default one. BUT…

How would you go about calling sublime_plugin.unload_module("Default/paste_from_history") after all the plugins have finished loading from within another plugin? I can not see any convenient method to do this with in the API, i.e. nothing like on_plugins_all_loaded() or on_application_finished_startup().

**addons_zz** · January 21, 2019, 4:14am

After reading @ThomSmith post I see that calling sublime_plugin.unload_module("Default/paste_from_history") could not be a good idea because some other package could depend on Default/paste_from_history being loaded into memory (as it is a default plugin), then, if I both packages at the same time, the other package would not work.

This is usually done with some non-very elegant way as Package Control does. When plugin_loaded() is called, it create a thread which runs the overall packages maintenance. For you, I would suggest the following code:

import time
import threading

def plugin_loaded():
    threading.Thread(target=_background_bootstrap).start()

def _background_bootstrap():
    time.sleep(1)
    # do the work here, not blocking others

I wrote and tested the following Python plugin for you to use. It is a copy and paste, and it will start working. On your package, you can create file a called anything you like as paste_from_history_installer.py and put this as its contents. Its only requirements are that you need to name your override for the paste_from_history.py as paste_from_history.py and the paste_from_history.py file has to be on the same directory as the paste_from_history_installer.py plugin file is on.

#!/usr/bin/env python3
# -*- coding: UTF-8 -*-

import sublime
import sublime_plugin

import os
import time
import threading

def plugin_loaded():
    threading.Thread(target=_background_bootstrap).start()

def sublime_join(*paths):
    return "/".join(paths)

def _background_bootstrap():
    time.sleep(1)
    custom_paste_from_history = sublime_join( "Packages", __package__, "paste_from_history.py" )
    default_paste_from_history = sublime_join( "Packages", "Default", "paste_from_history.py" )
    custom_paste_bytes = sublime.load_binary_resource( custom_paste_from_history )

    try:
        default_paste_bytes = sublime.load_binary_resource( default_paste_from_history )

    except IOError:
        default_paste_bytes = ""

    if custom_paste_bytes != default_paste_bytes:
        original_package = os.path.join( sublime.packages_path(), "Default" )
        default_paste_from_history = os.path.join( sublime.packages_path(), "Default", "paste_from_history.py" )

        if not os.path.exists( original_package ):
            os.makedirs( original_package )

        with open( default_paste_from_history, 'w', newline='\n', encoding='utf-8' ) as output_file:

            if isinstance( custom_paste_bytes, str ):
                output_file.write( custom_paste_bytes )

            else:
                output_file.write( custom_paste_bytes.decode('utf-8') )

    sublime_plugin.reload_plugin( "Default.paste_from_history" )

This will execute the friendly strategy I suggest to override the default paste_from_history.py in my first post. The only requirement for not breaking any other Sublime Text plugin which depends on paste_from_history.py, is that your paste_from_history.py version has to be backward compatible the the original paste_from_history.py.

It does put a copy of your paste_from_history.py with the command named PasteFromHistoryCommand into the loose packages directory Default/paste_from_history.py. And automatically call sublime_plugin.reload_plugin("Default/paste_from_history") after all packages being loaded. With this, you can remove the original plugin paste_from_history.py from memory, replacing it by your enhancement.

**mattst** · January 27, 2019, 9:31pm

Thanks so much for writing this installer addons_zz. V. kind of you. Sorry to take a week to get back to you I’ve had an unbelievably busy time.

I’m hesitant to use code I don’t fully understand, but will take a look soon and work out exactly how your installer plugin works.

It seems strange that the Default packages can not be easily over-ridden in the same way ST config files can be, i.e. just by placing paste_from_history.py in the ../PATH_TO/CONFIG_ST3/Packages/Default/ directory but perhaps that will be changed at some point.

I don’t think the default paste_from_history.py plugin is particularly well written, it applies a regex to clipboard text which could potentially be many MB is size, when only the first 45 non-whitespace characters will ever be subsequently used. Nor is the display name well designed, multi-line clipboards with lots of indentation are shown with all the non-newline whitespace intact (excepting the 1st line). My display naming is far neater. Also a shame that Sublime HQ on GitHub don’t accept pull requests for Default plugins, only syntaxes.

When I have time I’ll submit my PasteFromHistoryEnhanced plugin to Package Control, in the meantime it is in this gist: PasteFromHistoryEnhanced.py Note: Currently a work in progress, there are a couple more things to change.

**OdatNurd** · January 27, 2019, 9:37pm

I was surprised to see this because I have an override on Default/paste_from_history.py and it works just fine as far as I’ve been able to see (mine just replaces the pop up menu with a quick panel and shows more lines, for context).

Is there some written reference to this being changed? I don’t remember this ever being mentioned in a change log or anything.

**addons_zz** · January 27, 2019, 11:40pm

They are quite simple, just a few lines of code which copy things when they are updated, and call sublime_plugin.reload_plugin( "Default.paste_from_history" ) in the end.

I just talked by my experience. Once, sometime ago, I had unpacked Default.sublime-package in the loose packages, and none of them were loaded, except the third part ones. May be this was particular of the build I was using and changed right on the next one. Or may be it works fine if it is only one replacement. If so, then you can simplify edit my code and remove the last line sublime_plugin.reload_plugin( "Default.paste_from_history" ).

**mattst** · January 28, 2019, 7:32pm

Yes, you’re right it does. Clearly the same filename and the same TextCommand class name (i.e. PasteFromHistoryCommand) must be used with the location ../PATH_TO/CONFIG_ST3/Packages/Default/paste_from_history.py but then the console happily reports reloading plugin Default.paste_from_history instead of reloading plugin User.paste_from_history.

I’m running version 3176 (latest stable), are you referring to the beta versions that are more recent than that?

Although I agree this could theoretically be a problem, all another plugin could do is run the paste_from_history command in which case my version would be run and, since it just does the same thing slightly differently, the same effect would be achieved. Also, while many of the Default plugins are potentially useful to call from another plugin, paste_from_history is not one of them and I’m struggling to think of a reason why anyone would want to call it. [I am reasonably sure about this analysis, but have I missed anything?]

**addons_zz** · January 28, 2019, 10:24pm

I do not use it because I use a system manager clipboard application https://hluk.github.io/CopyQ/ and I do not see much sense in having a exclusive clipboard history only for Sublime Text while it can be available system wide. Unless we are talking about having 2 distinct and unrelated clipboards histories, i.e., which does not track the same records, for example, while the system wide clipboard history tracks everything, the Sublime Text clipboard history, would only track what I tell it to track without affect the system wide clipboard. This would be similar to vim, which does not mix the system clipboard with his own clipboard.

Just noting, the problem I mentioned, happens on Sublime Text start up. After Sublime Text had completely “booted”, everything reloads fine on Packages/Default. Then, if you edit the paste_from_history.py after Sublime Text had start up, it will reload correctly/normally, but if you restart Sublime Text, your changes to paste_from_history.py would be ignored and not loaded until you explicitly call sublime_plugin.reload_plugin( "Default.paste_from_history" )

When I said old builds, I was talking more about builds close to build 3114, and when I said with newer builds, I was mentioning builds up to 3176. Nowadays, I am also using build 3176 because the newer development builds are crashing:

Since I had stopped using development builds, I had no new crashes so far.

**mattst** · January 29, 2019, 4:40pm

You misunderstood, I thought it clear from the context that I meant: I’m struggling to think of a reason why anyone would want to call 'run_command("paste_from_history")' from within another plugin. ThomSmith had stated that by replacing 'paste_from_history.py' with my own version the integrity of other plugins might be adversely affected “…because other packages may reasonably rely on that code being loaded”.

Actually I am now certain there is no problem and that by placing the new code in the 'ST3_CONFIG/Packages/Default/paste_from_history.py' file the module gets loaded into memory replacing the original file of that name in the OS installation archive 'Default.sublime-package'. It does not matter what plugin class names are used in the new file, the whole module is replaced by the new one, i.e. all the plugin classes in the original file are unloaded and all the new plugin classes are loaded.

The only restriction is that 'ST3_CONFIG/Packages/Default/paste_from_history.py' must be in place when ST starts. After that the file can even be edited in situ and, on saving, the module gets reloaded.

To be certain the the original package was not also running alongside the new one, I created a new 'Default.sublime-package' and temporarily replaced the original one in the OS system files, i.e. on Linux '/opt/sublime_text/Packages/Default.sublime-package'.

In my test archive I added a line in the paste_from_history.py' file’s 'EventListener' class’s 'on_post_text_command()' method to print a message to the ST console every time a clipboard ‘cut’ or ‘copy’ command was run and g_clipboard_history.push_text() is triggered. It worked fine, console messages were printed, but when I recreated my 'ST3_CONFIG/Packages/Default/paste_from_history.py' file and restarted ST the console messages stopped being printed, that way I could be sure that the whole module had been replaced by the new one.

I hope this clarifies things for anyone looking to replace a 'Default.sublime-package' Python plugin with their own code/version. BUT to be clear, as ThomSmith stated, it is reasonable to expect that other plugins might call default packages for their own purposes, e.g. 'sort.py', 'delete_word.py', 'duplicate_line.py', 'open_in_browser.py', etc., etc., so you should be cautious if replacing any default package with your own module and preserve class names and functionality.

**ThomSmith** · January 29, 2019, 5:01pm

Just to clarify, my concern is about using the undocumented internals of sublime_plugin to manually unload the module. I wouldn’t want to do this unless absolutely necessary. In some cases, it is absolutely necessary; I just don’t think that this is such a case.

Adding a patch in Packages/Default/paste_from_history.py, on the other hand, is perfectly sane and reasonable. It’s possible that something could break as a result, but the level of potential weirdness is much lower.

Replacing the default "paste_from_history.py" plugin?

Your best solution

Your worst solution