API Suggestions

[bizoo]

1. Select the lines you want to hide
2. Menu Edit -> Code Folding -> Fold

You can use RegReplace to do it using REGEXP

[braver]

Menu Edit -> Code Folding -> Fold

Nope:

those lines would be excluded from search and edit operations.

[FichteFoll]

More consistent output panels

Output panels have really odd behaviors currently. In exec.py, there is this code:

if not hasattr(self, 'output_view'):
    # Try not to call get_output_panel until the regexes are assigned
    self.output_view = self.window.create_output_panel("exec")

# [...]

self.output_view.settings().set("result_file_regex", file_regex)
self.output_view.settings().set("result_line_regex", line_regex)
self.output_view.settings().set("result_base_dir", working_dir)
self.output_view.settings().set("word_wrap", word_wrap)
self.output_view.settings().set("line_numbers", False)
self.output_view.settings().set("gutter", False)
self.output_view.settings().set("scroll_past_end", False)
self.output_view.assign_syntax(syntax)

# Call create_output_panel a second time after assigning the above
# settings, so that it'll be picked up as a result buffer
self.window.create_output_panel("exec")

Observations:

1. Window.create_output_panel is run twice. But only at the first time.
2. calling .assign_syntax on the output panel turns it from a widget into an actual view. This causes it to infer its settings from the usual Preferences.sublime-settings files (or more likely, they are not overridden from some widget-specific thing). As a result, the output panel will have its gutter and line numbers visible, which exec.py has to disable explicitly. (more on this)
3. On each call of Window.create_output_panel, its entire contents are erased. Settings are not reset, though.

Priority: minor (it’s “always” been like this and changing behavior of create_output_panel especially wrt 3. would be a breaking change)

Related: API Suggestions

[mlf]

Putting scopes to work

So far, scopes use is limited: syntax coloration and some limited hard-coded features (‘go to symbol’, mostly).

What about extending them, in two ways:

1. being able to specify a scope for the search/replace api (so that we could search/replace for a given expression in comments only, or not in comments, extract all test fragments with a given scope, like we currently extract the list of symbols, jump to the next or previous string (or any scope), and so on);
2. being able to new ‘styling’ options for scopes, with styling being ‘editable/readonly’, or ‘searchable/notsearchable’, and so on.

Priority: minor

[gwenzek]

Speaking about putting scopes to work:

I’d like to have scope based folding instead of just indent based folding.

Similar to the tmPreferences files used by the symbol list,
a setting file would allow to control which scope can be folded.

Importance: Major

[gwenzek]

Also I’d like scope based auto-indentation.

Now the Indentation.tmPreferences is doing regex matching while all the information we need is already extracted by the syntax.

Importance: Minor (since there is already a working mechanism albeit limited)

[FichteFoll]

I also wanted to suggest an indentation system that is based on scopes (and the powerful syntax lexing that’s happening all the time anyway), but hold back onto creating an issue for it since I haven’t yet drafted out a specific suggestion on how I would imagine this feature.
I also wonder if it should be part of the syntax definition or utilize a separate scope-selector-oriented mechanism like tmPreferences do.

[gwenzek]

Another thing that I’d like is improved parsing of the output panel.
For now errors have to be extracted through two regexes, one for the file and one for the line, character offset and message.

So here are 3 suggestions in increasing order of complexity.

1. the output of the C# compiler doesn’t print full path to files, only the name of the file.
   This prevent the “next_result” command to open the correct file.
   Proposed fixed: if the matched path doesn’t exist try to open a file in the current project with the same name.

2. Sometimes the error message is on different line than the line number (eg in Python). So there is no way to capture the error message.
   Proposed fix: add a third regex for the error message.

3. There is no distinctions between errors and warnings. You have to choose when you create the build system if you want to catch both or only errors. Ideally the build system would provide a way to extract both, and the user could choose which one to display.

[FichteFoll]

I hate it that I always find something to comment on in these threads, but I just can’t help it. If that’s not desired, I’ll continue replying to posts in new threads, but I’m on mobile atm.

To 1.: relative file paths are a possibility (and by default relative to the active view’s file’s directory. It can be configured with a setting that you can inspect from exec.py.

To 2.: you can capture the message in the regular expressions, but nothing is really done with it. I think it shows in the status bar on some action.

[braver]

To 2.: you can capture the message in the regular expressions, but nothing is really done with it. I think it shows in the status bar on some action.

If you capture the message in a build systems, it is shown in phantoms. I think there is a build-error tooltips package out there that also uses the messages. So, in 3118 this works as described here.

[gwenzek]

To 2.: you can capture the message in the regular expressions, but nothing is really done with it. I think it shows in the status bar on some action.

I wasn’t aware of that so I updated my feature request.
The problem is that it must be one regex matching both line number and error message.
But for python where the error message is on a different line than the line number, you can’t capture it (confirmed by Jps in the build 3118 thread).

[315234]

Syntax-specific option for case-insensitive goto-definition

There was some discussion here:

Sublime Text 3 "goto definition" case sensitivity

Absolutely love ST3. Got a question tho. I write mostly SQL which is a case insensitive language. The goto defintion feature workes wonderfully (once I did some light hacking on the definition file). The matching regexes work properly: they do case insensitive matching when requested. However, when I f10 on a token that is different in terms of case, it will not find the definition: create table foo(); Foo <- f12 will not match the definition foo <- f12 will match the definition Is…

Essentially, for languages that are case-insensitive (Fortran is the example I care about) it is desirable that goto-definition is case insensitive (e.g. you define MYFUNCTION but then call MyFunction, currently goto-definition doesn’t work for this case). This is already partly solved by the ability to perform a symbol transformation to convert all indexed symbols to lowercase, but goto-definition will then not work on any word that is not lowercase. I see at least three possible solutions:

1. Add a caseInsensitive option to the tmPreferences. This works for case insensitive languages but is the least general approach. It has the advantage that the original capitalisation would be preserved in the symbol list.

2. Apply symbolTransformation to the current word before looking it up in the index. This might break something else, I’m not sure.

3. Add a lookupSymbolTransformation to the tmPreferences that would be applied to the current word before looking it up in the index.

Importance: major for case-insensitive languages

[315234]

Full set of character codes in minihtml

For arbitrary text, the python function html.escape often produces HTML codes that minihtml doesn’t understand, such as ' for ':

import html
view.add_phantom("id", sublime.Region(0,0), html.escape("'"), sublime.LAYOUT_BLOCK)

Obviously this makes it difficult to display arbitrary strings in Phantoms or Popups.

Importance: minor

Edit: It seems that minihtml currently works reliably with only the following substitutions (I have tested with all ASCII characters):

def to_html(s):
    s = s.replace('&', '&')
    s = s.replace('<', '&lt;')
    return s

[tito]

It would be nice to have “onpopupshowing” event on menus. And to allow programmable creation of the menuitems for that menu. This will allow:

1. Infinite nested menus and menuitems with variable amount of menuitems or menus.
2. avoid all processing that are shared in the menuitems, instead or processing stuff for each menuitem of a menu in a single command as we do now, you just do the processing on the onpopupshowing event and then reuse the calculations on all the menuitems that need it.

[315234]

API to get theme/scheme colours etc

Currenly, I have some code to get the popupCss background/foreground colours for the current view so that I can use these to generate an image that will not look out of place with the colour scheme. My current code looks like this (thanks to @FichteFoll for getting me started) :

    def parse_popupCss(css):
        words = css.split()
        i = 0
        bg = None
        fg = None
        while words[i] != "html":
            i += 1
        while words[i] != "}":
            if words[i] == "background-color:":
                bg = words[i+1]
            if words[i] == "color:":
                fg = words[i+1]
            i += 1
    
        # Defaults if not found
        if bg == None:
            bg = "#FFFFFF"
        if fg == None:
            fg = "#000000"
    
        # Remove leading # and trailing ;
        bg = bg[1:7]
        fg = fg[1:7]
    
        return bg, fg
    scheme_path = view.settings().get("color_scheme")
    scheme_content = sublime.load_binary_resource(scheme_path)
    scheme_data = plistlib.readPlistFromBytes(scheme_content)
    css = scheme_data["settings"][0]["settings"]["popupCss"]
    bg, fg = parse_popupCss(css)

This is not ideal:

• finding, loading and parsing the PList again seems hacky, when Sublime presumably has it in memory already

• the structure of the nested lists/dicts seems potentially fragile

  • scheme_data["settings"][0] is necessary to avoid searching through the entire list for the dict who’s only key is "settings", but is this always guaranteed to be the first in the list?
  • scheme_data["settings"][0]["settings"] doesn’t give the reader any clue about why I am doing this

• some of the colours are in dicts, others are in dicts inside of dicts, and some are now in chunks of CSS code inside of dicts

• Colour strings can be found in a variety of different formats (FFF vs #FFFFFF vs #FFFFFF00, for example)

• There is no CSS parser that ships with Sublime (that I know of), so I had to implement my own basic one. It is likely to break, depending on how schemes write their CSS.

• The colour I’m looking for may not even have been defined in the end, and the code above doesn’t even check for this but is already quite long

The situation with themes further complicates matters:

• There are potentially multiple resources (i.e. if a user overrides a theme, or uses a theme addon) that have to be searched through in the right order to find the final colour or setting

• The file format and dict structure are both different which requires different logic to handle

Finally, in trying to reproduce all of the internal logic that Sublime uses, we are likely to make mistakes and end up being inconsistent, leading to hard-to-track-down bugs.

Since all of this is already being done by Sublime, I would prefer to see an API exposing this functionality that looks something like this:

# Each of these returns a flat dict with a predefined set of keys, all set

# the main "settings" section of a scheme
d = view.appearance()
#     {'background': '#FFFFFF',
#      'caret': '#7C7C7C',
#      'foreground': '#000000',
#      'invisibles': '#B6B6B6',
#      'lineHighlight': '#EFFCA68F',
#      'selection': '#E3FC8D'}

# how text with the given scope string would be drawn in the current view
d = view.scope_appearance("some.combination.of.scopes", selection=False)
#     {'background': '#FFFFFF',
#      'foreground': '#000000',
#      'italic': false,
#      ...
#     }

# get CSS properties
d = view.html_appearance(where="popup", what="h3")
d = view.html_appearance(where="phantom", what="html.background")
#     {'background': '#FFFFFF',
#      'foreground': '#000000',
#      'fontsize': 10,
#      ...
#     }

# get theme properties
d = sublime.theme_appearance("tab", current=True)
#     {'background': '#FFFFFF',
#      'foreground': '#000000',
#      'height': 10,
#      ...
#     }

The exact API could probably be better, but the important things are

• Consistent with Sublime’s internal representation of these things

• Automatically selects the most appropriate resource to read from

• Guarantees existence of values by substituting in the defaults

• Consistent form for colour strings (maybe even a Color type?)

Importance: minor I suppose

[facelessuser]

Getting the colors by scope is possible now. I’ve been meaning to take what I have and make it a dependency. Maybe I’ll do that in the next couple of days.

But with that said, an API would be cool for this.

[315234]

Getting the colors by scope is possible now.

Do you mean using one of your packages, or in the base install? From what I understand you have implemented a lot of the stuff I talk about above in a package.

[gerry]

I extract the styles similar to your example for my ColorSchemeUnit package.

An API would be good.

Testing color schemes

okay, here is the same script rewritten to use the built in plistlib module instead of the third party lxml module: import sublime, sublime_plugin import plistlib class GetStylesAtCursorCommand(sublime_plugin.TextCommand): def run(self, edit): self.parse_color_scheme_file(self.view.settings().get('color_scheme')) scope_at_position = self.view.scope_name(self.view.sel()[0].begin()) print(self.get_styles_for_scope(scope_at_position)) def parse_color…

The current relevant code is (needs optimising):

class Style(object):

    def __init__(self, view):
        self.view = view
        self.parse_color_scheme_file(self.view.settings().get('color_scheme'))

    def at_point(self, point):
        return self.get_styles_for_scope(self.view.scope_name(point).strip())

    def parse_color_scheme_file(self, color_scheme_file):
        color_scheme_content = sublime.load_resource(color_scheme_file)
        color_scheme_dict = plistlib.readPlistFromBytes(bytes(color_scheme_content, 'UTF-8'))
        self.selectors_in_scheme = color_scheme_dict['settings']

    def get_styles_for_scope(self, scope):
        styles = dict()

        for scheme_selector in self.selectors_in_scheme:
            if 'scope' not in scheme_selector:
                styles.update(scheme_selector['settings'])

        matched_style = {'settings': {}, 'score': 0}
        for scheme_selector in self.selectors_in_scheme:
            if 'scope' in scheme_selector:
                score = sublime.score_selector(scope, scheme_selector['scope'])
                if score:
                    if score >= matched_style['score']:
                        matched_style['score'] = score
                        matched_style['settings'].update(scheme_selector['settings'])

        styles.update(matched_style['settings'])

    return styles

[facelessuser]

Do you mean using one of your packages, or in the base install? From what I understand you have implemented a lot of the stuff I talk about above in a package.

Yes. I have a couple of plugins that use some of the same color libs: ExportHtml, ScopeHunter, ColorHelper, and the mdpopups dependency. The idea is to create one dependency they use and make it available to anyone else as well. Though mdpopups will be harder move to a color lib dependency as it is a dependency itself, and Package Control can’t specify dependencies for dependencies. Getting people who use mdpopups to migrate to using that color dependency will be more slow as I have to give people time realize the change is occurring and adapt.

Anyways, the idea is to clean up the code and provide a more sane API. I would just bundle the various color stuff as a dependency lib. That way my plugins will just use a shared dependency instead of all shipping the same/similar libraries as they do now.

I would change unnecessary things like ColorSchemeMatcher.guess(view, pt, scope) to something like you suggested above ColorSchemeMatcher.guess(scope, selected=True) (which I like), and it would provide you with the a style object with colors, emphasis, transparent colors converted to non-transparent colors (since the Sublime CSS engine doesn’t really handle the color mixing stuff), etc. And you could then also request special colors like gutter, selection, foreground, and background directly like this: ColorSchemeMatcher.get_special_color('selectionForeground', simulated_transparency=True). Not sure if I care about including access to stuff like activeGuide etc. as I don’t find that stuff useful, but maybe.

I think I would also throw in the library to adjust the colors, brightness, saturation, etc. I would throw in the colorbox library that can generate color boxes for sublimes HTML content. Maybe even provide access to a call to convert a scheme to CSS using standard TextMate scopes. Stuff like that. Then if I add more, everyone using the lib would get access to those things too.

---

I’m not sure yet how I feel about implementing a CSS parser though. I have no problem with providing access to the CSS though. People could put anything in there. And you still don’t get what Sublime is providing under that layer. Luckily, in its currently implementation, you have to keep things simple. You can’t really specify div.class or .class1 .class2 (or you couldn’t last I checked – I don’t know if things changed). All you can really do right now is element and .class and .class1.class2. But if any of that changes, things break and then get more complicated to parse.

I don’t feel like it is practical to scan for everything in the CSS. The only time I really care about scanning for weird scopes in a scheme is when I am replicating a Sublime View in HTML, and I only need to that for creating code blocks from Sublime in HTML, and for something like ScopeHunter. I don’t really need to replicate CSS in my sub CSS for popups and phantoms; I should already get that stuff for free as long as I don’t override it. I don’t want to scan for available classes either. I just want to know what standard classes are reasonably expected to be there. Whatever non-standard class you are looking for could be missing if you install another scheme. From a plugin standpoint, I would only be interested in what standard class names I can use, and maybe the body element back-ground color for when I insert images. I would prefer that themes keep that stuff simple so I don’t have to do elaborate analysis.

The reason why I wrote mdpopups was to standardize everything for myself and the plugins I write to spare myself from analyzing scopes in every plugin for popups; it’s all done for me. I just give it scopes and don’t worry about. If I need something specific, I inject a little additional CSS using known scopes. I know I can only expect standard TextMate scopes and .foreground and .background. I know that mdpopups will make the background and foreground color the same as the scheme background and the foreground. So it is a simple list I can reference. I now know that schemes are likely to include colors and font styling under the classes .error, .deleted, .success, .inserted, .warning, and .modified. I don’t care if a theme inserts other classes as those will vary from theme to theme and can’t be relied on. I am only interested in standard colors. I’d rather give up a little control to have things consistent.

[kblomqvist]

See also https://github.com/SublimeTextIssues/Core/issues/134