Sublime Forum

[SOLVED] Where and how do I store internal package settings?

#1

Hi.

I am just starting out with Sublime Text Package Development, so I have some basic questions. In a package that I am trying to write, I need to collect some paths of installed applications. Now, once I collect these paths I want to store them somewhere, so they are available once I run a command that needs them or even the next time I start Sublime.

I tried to store them in a sublime-settings file, but that results in having them written to a sublime-settings file in the user folder. These are settings though that should not be altered by the user, so they should rather be stored somewhere internally within the Package directory itself.

What is the recommended way of doing this?

Thank you!

0 Likes

#2

The settings mechanism in Sublime allows a sublime-settings file to appear in any package that wants one (and anywhere inside of the package as well), but using save_settings() always stores the result in the User package.

The general idea is that since multiple settings files of the same name are automatically loaded and combined together, the package should carry a settings file that has the default values for any settings that it supports, and the user can apply their changes to a file in their User package to override only the settings they require.

It’s a little unclear from your description why it’s a bad thing that the user be able to modify the settings that you’re gathering (for example, what if your automatic detection doesn’t work or finds the wrong thing?).

Presuming you want to keep your data away from prying eyes (easily), technically speaking you can write a file anywhere you want. Assuming you don’t want to write it into the User package, you could choose to put it into the Cache folder instead (with the proviso that you’re prepared for the file to be missing and regenerate it), or if your package was named say Bob you could create a Bob folder in Packages (if it doesn’t already exist) and write the file into there instead.

I don’t think there’s a recommendation per se expect for what works for you and the goal you’re trying to achieve, though.

1 Like

#3

That’s a fair point. While I don’t want to hide the files from the user, I still would prefer to let the user change these settings through dedicated commands. So have an auto-detect command in place and then offer a command with a file picker to add application paths manually in case the auto detect fails.

So, I do something like

with open(os.path.join(packageDir, 'appPaths.json'), 'w') as file:
      file.write(json.dumps(appDict))

? And then read from that json file again in my commands? Is that what you mean?
Or is there a dedicated Sublime API method to achieve the same thing in a more native manner? I am also new to python, so I am somewhat still in the dark when it comes to using it.

Thanks!

0 Likes

#4

Something to keep in mind is that it’s somewhat of a “power user” kind of thing in Sublime to seek out and edit a sublime-settings file yourself (say with a file browser or via View Package File from the command palette) and generally that would be exposed by adding an item to the menu and/or the command palette that opens the appropriate file for you.

So in that sense, it may be enough to have two settings files; one that’s “hidden” and one that’s "public’; the hidden one would be a settings file stored in the User package but not exposed and the other one would be made more traditionally available. In that scenario you’re more or less setting up a power user with the ability to easily tweak the file (which they could probably figure out anyway) without going to any extra work.

On the flip side, something in favor of not storing the file in the User package is that it’s fairly common to syncrhonize your User package across multiple computers (including multiple operating systems) to keep all the copies of Sublime you use running the same. In such a case, storing configuration like the one you’re mentioning here may be a bad thing if the data it contains is somehow machine specific.

More or less, yep. Presuming that you obtain packageDir as something like os.path.join(sublime.packages_path(), "MyPackageName") (basically, you don’t want to write directly to sublime.packages_path()).

An alternative is to use sublime.cache_path() instead; that keeps things outside of the Packages folder but still associated with Sublime itself.

If you use packages_path(), you should definitely use a folder named for your package, since that’s essentially your “namespace” and is an area that nothing else should be touching. In the case of cache_path() it’s generally a good idea to do the same, for similar reasons.

In either case you also need to take the step of making sure the folder exists. If you’re working on your package locally, then that’s definitely the case. However if you distribute your package via Package Control at some point, it generally ends up getting installed as a sublime-package file instead (unless you tell Package Control not to do that), in which case the folder won’t be there unless you create it first,

The sublime.load_resource() method can be used to load the content of a package (wether it is in a folder or inside of a sublime-package file), so you could use that in order to load the file when you need to. There’s no real analoge for saving files into your package though, since generally speaking the most common things to be saved into a package would be settings (and that happens automatically).

To use it, you could do something like sublime.load_resource("Packages/MyPackage/appPaths.json") to give you the content of the file in one call (or load_binary_resource(), if the file is not text). It will throw an exception if the file can’t be found. You can also use sublime.find_resources() in order to determine if the file exists; sublime.find_resources("appData.json") would return a list of every file by that name that exists in a package, for example.

Something to note is that load_resource() will work with files from the Cache folder, but that’s sort of an implementation detail that may or may not go away, so it may not be a good idea to rely on it.

One last thing is that rather than loading the file in your commands, you could define a plugin_loaded() function in your plugin and load the file there. That method gets called every time a plugin reloads, which is the first time it’s loaded at startup, whenever you modify it, and when the package that it’s in gets removed from the ignored_packages setting. Assuming the data doesn’t change very often, that would let you load the file once instead of every time a command that needs it is executed.

0 Likes

#5

Thank you so much @OdatNurd for taking the time and reply in such detail. This is very valuable information for me.

Yes, having a public and a hidden file was my plan. Reading the pros and cons, it seems to make sense to me to not store this hidden file in the User folder, as it – as you correctly point – contains very machine specific data.

Yes, that’s how I did it.

Yes, I want to eventually distribute it as a sublime-package file.

I don’t quite understand, can I still write into files within this (zip packaged) sublime-package file or would I have to keep it unpackaged in this case?

This sounds good, but I also don’t quite understand where to this would load the data then. Would I just load it into some global top level variable that all the commands then have access to?

Thanks!

0 Likes

#6

Packages in Sublime can be installed as both sublime-package files and folders in the Packages folder at the same time. Files in Something.sublime-package and Packages/Something are both considered to be coming from the same place, which is generally transparent to you.

That’s why sublime.load_resource() exists and takes a file name of the sort Packages/Something/somefile.txt. As far as that method is concerned it doesn’t care which location the file comes from. Using that allows you to be sure that no matter how your package is installed, it will still find the correct file to open.

So in that regard, you can have Package Control install your package as a sublime-package file (which is the default unless you tell it otherwise) and still create the folder in Packages manually if you need to store extra files of any sort. Since the files there are considered as being part of your package, it’s a folder that’s “owned” by your package, if you will.

You generally only need to have your package be installed unpacked if it contains something that needs to be externally accessible, such as if it contains an executable that you’re going to launch, a needed library for something outside of the package, etc. All else being equal, being installed as a sublime-package file is better because it allows you to override a resource if you need to.

Yeah, pretty much. You can structure it how you like, but the general gist is that plugin_loaded() is guaranteed to be called for any plugin as it’s loading, and at a point where accessing the Sublime API is allowed, so it’s a good place to include initialization like that.

Keep in mind that in Python a global is really only global to the file that it’s defined in; it doesn’t “leak out” to be visible elsewhere unless something explicitly tries to get at it,

So, plugin_loaded() could set up a global that has the loaded data if the file exists, and your command to find and create the file can both save the file there and update the global to contain the data that it found. Then all commands that need access to it can get at it directly.

1 Like

#7

Perfect, I think I could follow all your explanations.

Again, thanks so much for taking the time to explain all this to me. Also, just to let you know, your YouTube channel was what got me started with Sublime Text Plugin development in the first place. Absolutely fantastic resource. Discovered it a few weeks ago and pretty much binge watched the entire channel already. Now I feel ready to tackle my first package!

2 Likes

#8

Hi @OdatNurd, it’s been a while since I have asked this question, but now I am back at the point where I need to implement this settings setup.

I have one more question about this specific point from your second reply. The “hidden” file in the User folder, how exactly would that be hidden/not exposed? And why would I place this in the User folder and not in Packages/MyPackageName? And would that be a .sublime-settings file or some sort of traditional .json file?

0 Likes

#9

I think the idea of “hidden” that I was going for there was just the idea that you would store the file in the User package (or anywhere really) but not provide a command to modify it directly in any way (the way you might say add an item to the Preferences menu to edit your package settings).

In that case, the file would essentially be invisible except to people that are power-user enough to know to look for it and manually get at it for editing, in which case they probably know enough to do it safely.

Technically you have the power to put any file you want in any location you want, but it’s probably bad form to drop files inside of packages you don’t own (except the User package), and users might come at you with a sack of doorknobs if you venture outside of the area set aside for Sublime files. :slight_smile:

Generally in a plugin if you’re creating some sort of data file that you expect to be around you’d put it into a package folder, whereas if it was a temporary file that you could recreate if it was missing and it doesn’t matter if it goes away you could use sublime.cache_path() to get the Cache folder location and store it in there.

What package you put it in is pretty much up to you. I suppose conventionally you would use the User package because it has to always exist and it’s the place that stores information specific to the current user. A potential downside to that is that people sync their User packages across machines, which might be an issue if what you put in the file is machine specific. In that case you might need to structure the file to be able to hold information for multiple machines.

Failing that I think the next best place would be in your own package’s folder, yes. Note however that if your package is installed as a sublime-package file, the folder won’t exist and you’ll have to create it.

If you tell package control not to install your package as a .sublime-package then it will put the contents directly into the package folder, but it will also remove the whole folder and replace it when the package upgrades (I think).

If you’re not planning on being distributed by Package Control that’s not really an issue, though.

This all applies to any sort of file that you’re needing to store, whether it’s a sublime-settings, json or anything else.

It’s also important to note that sublime-settings files in the User package need to be in the root of the package and not in a sub-folder. That only applies to the User package though; it’s a side effect of how sublime.save_settings() takes a file without a path and always writes changes into the User package at the root.

For what it’s worth, none of these sorts of things are really set in stone at all (except for things that rely on something technical in nature).

0 Likes

#10

Thanks a lot. I think I understood your points now, one thing that I still wonder though: If decide to put this settings file into my own package’s folder (so not in the User folder) that I have to create first, can this still be a .sublime-settings file that takes advantage of the sublime settings mechanisms or do I then have to pick another format?

0 Likes

#11

Sublime has a sort of “virtual resource file system” related to packages; things that exist in the Packages folder and the contents of sublime-package files all appear to plugins to be coming from the same place in a seamless way.

For example MyPackage/somefile.txt might be in $DATA/Packages/MyPackage/ or it might be in $DATA/InstalledPackages/MyPackage.sublime-package (there’s also a third folder where the packages that ship with Sublime are stored; those sublime-package files are also included).

So in your code, sublime.load_resource('Packages/MyPackage/somefile.txt') will return the contents of the file back no matter which place it’s stored (so long as it exists).

So in that regard, if you use a sublime-settings file it doesn’t matter as long as it’s inside of the package (with the exception of what I noted above regarding settings only working in the root of the User package).

1 Like

#12

Ok. Will try to sort this all out (it’s quite complex for a beginner like me) step by step and try to use a sublime-settings file then. Might post a follow-up question if I get stuck at some point. Thanks a lot!

0 Likes

#13

@OdatNurd Sorry, I yet again will have to ask one question for clarification:

As described above, I want to keep my settings entirely out of the User folder. Now when I started to implement the whole thing with a .sublime-settings file, set a key on that and do sublime.save_settings('my_settings.sublime-settings') it saves the key in the User folder, which is not desired in my case.

I am starting to understand the mechanism of sublime-settings now, the load_settings() command does not take an absolute path since it loads settings that are the result of User settings and package settings combined.

So. If I want to keep this whole mechanism out of the User folder, am I right in assuming that I cannot use a sublime-settings file and need to work with a .json file instead and manage that with vanilla python commands instead of Sublime methods? Or am I overlooking something?

Edit: Or could I maybe use a sublime-settings object, take advantage of its Sublime API methods and just avoid the save_settings() command to keep it out of the User folder?

0 Likes

#14

That would indeed work for you; it’s the save call that’s generating the file in the user package, so as long as you don’t do that (and write the settings manually instead), it would do what you want.

That said, one of the main draws the the settings mechanism is how it allows for multiple settings files to combine to allow the user to control how things work. If that’s not what you want and you have to manually save the file manually anyway, it may make sense to just use your own custom json file held elsewhere instead.

1 Like

#15

Ok, got it. Thanks!

0 Likes