Sublime Forum

Modify Markdown.sublime-syntax file to add hashtag

#1

I have been a long time ST user and have recently decided to get into customizing it to better meet my needs.

I used the package resource viewer to extract a copy of the Markdown.sublime-syntax file.

I constructed the regex I want: “(#\S+)”

I am still learning how the syntax system works I added this context to the bottom:

  hashtags:
    - match: '([\#](?!\#)\S+)'
      captures:
        1: invalid.illegal.expected-eol.markdown
      pop: true

Basically, I modelled it after the after-link-title context. I wanted to see if I could at least getting it highlighted in red before I ventured and learned how to modify the color scheme to show a yellow highlight around the the tags.

I am stuck and do not know where I would need to modify this file to get it working.

Also, is there any recommend “toy” files to play with to learn how the syntax highlighting works?

I did find a bug in the way sections are handled i.e. “# Section Header” works and this also works “#Section Header” notice the spacing. I was under the impression that there had to be a space after the #.

  atx-heading:
    - match: '(#)(?!#)\s+(?=\S)'

I replaced the * with a + and it seems to work better now.

In any case, I would appreciate some pointers on what to look for to get hashtags high lighting.

0 Likes
#2

You’d want to - include: hashtags, likely in the inline context. If this is your first time making or modifying syntax definitions (welcome!), allow me to recommend:

  • the official documentation,
  • the “New Syntax…” command palette item (Ctrl+Shift+P “syn”), which creates just such a “toy” syntax,
  • and the PackageDev package which has excellent highlighting for syntax definitions themselves.

The ATX header requiring a space depends on which flavor of Markdown you use, and Sublime Text elected to use the more permissive version. See, for reference, Stack Overflow mentioning how that specific thing is changing with their move to CommonMark.

1 Like
#3

I did try an “include” in the inline context, but nothing seemed to have happened. I’ll play with the “toy” and see what happens as per your recommendation. I’ll post back if I can’t get it to work.

I do have PackageDev installed. It would be more difficult if the syntax were not nicely highlighted.

I guess my problem is, is that I don’t have a mental model of how the syntax rules are applied to the text. I have read the official docs a couple of times. These things usually sink in after I have had some hands-on. Do you have some simply walk throughs?

For the ATX header, the only reason I noticed/care is with regards to #hashtags starting as the first character of the line. It then gets highlighted as if it were a section. The changes you linked too would work for me.

I am working on pushing my plain text journal/time capture system to the next level by automating ST more than I have in the past. My notes are in markdown format. I have made the decision to go with the common markdown variant. I will have to add something to handle #HashTags and my custom time range format “T: 0845 (America/Toronto - 1745 (America/Denver)”. When I can get those two things working in it I will be quite happy.

Is there a way to tell if the system version of the Markdown syntax file has changed? Or can I assume that it will change with each version of ST?

0 Likes
#4

i found this link: Syntax Highlighting for Plain Text [noobie asking] and I am making progress decoding this! My next steps are to move matches to separate contexts and figure out how the include works.

In the post, it mentions that the example is using the markdown scope, is that the scope: source.txt? How do I access this to see what it is?

Also I noticed the use of these: markup.bold. Where do I find the references to these so I can see what I can use?

0 Likes
#5

I am getting further!

I have the following:

%YAML 1.2
---
# https://forum.sublimetext.com/t/syntax-highlighting-for-plain-text-noobie-asking/31232
name: My Plain Text
file_extensions: [txt]
scope: source.txt
contexts:
  main:
    - include: all_caps
    - include: starred_lines
    - include: bold_words
    - include: hashtags

  all_caps:
    # Line starting in all caps
    - match: '^[A-Z_]+\b.*'
      push:
        - meta_scope: markup.heading
        - match: '$\n?'
          pop: true

  starred_lines:
    # Line starting with **
    - match: '^\*\*.*'
      push:
        - meta_scope: meta.seperator
        - match: '$\n?'
          pop: true

  bold_words:
    # Words surrounded by *
    - match: '(\*)(\w+)(\*)'
      captures:
        1: punctuation.definition.bold.begin
        2: markup.bold
        3: punctuation.definition.bold.end

  hashtags:
    # Match hashtags
    - match: '([\#]{1}[^\s|\#]+)'
      captures:
        1: invalid.illegal.expected-eol.markdown

Using this text it seems to work as expected:


# Hello

## Hello World


#StartTag - Starts with a Hashtags, but can #find. This works!

a #Hastag-1 #HashTag2 #Hashtag_3 has the rings.  IR29815 - When slicing, sometimes the drill direction would be pointing towards each other, the user has to then manually define the azimuth of each drill #AnotherTag

The last two tags are found #HashTag-4 #HelloWorld

#Hastag-1 #HashTag2 #Hashtag_3 All three tags are highlighted

 #ThisIsATest - seems to work

Cool.

My goal of putting it back into the Markdown syntax is getting closer. I am having a couple of problems.

With this line:

#StartTag - Starts with a Hashtags, but can #find the others

The first tag is highlighted, the other tags are ignored.

If I do this:

a #StartTag - Starts with a Hashtags, but can #find the others

It finds all of the tags in the line. If I had a paragraph that started with a hashtag. It would only highlight the first one ignoring the rest. If I used something else (not a space, it doesn’t like that either).

I can post my changes to the Markdown.sublime-syntax, I would have but I am not sure of the policy on that sort of thing.

Another minor issue I am having is: It seems that the view/syntax is not remembered when I pick the different markdown, it forgets when I save the file. Not sure what I am doing wrong here.

0 Likes
#6

The syntax is not updated every time. The GitHub repository has tags that correspond to (some) Dev releases. If you’re on a 3xxx series build of Sublime Text, make sure you’re branching off of a tag that is 3xxx. There are extra features used in the Markdown syntax in 4xxx series, so branching off of master will not work if you’re on ST3.

The official guidance is here, and Brian Reilly has a webpage that parses the default syntaxes for a summary of scopes used. Ultimately, highlighting is provided by your color scheme, not the syntax definition, but if you follow the guidelines then good color schemes will look good.

I’ll return to this later, but I have to go for now. GL!

1 Like
#7

I think I just realized why you might be asking this. @OdatNurd has you covered with the OverrideAudit package. In addition to the run-on-demand reports, it will detect a ST version upgrade and tell you if one of your overridden files is out of date, let you diff it, and decide to ignore or deal with the change.

Is this with the syntax you have posted? Because that’s working fine for me on your sample notes. In the real Markdown syntax, I think you’d have trouble with this line thinking you’re starting an ATX title.

Did you click the bottom-right of the window where you select syntax and choose the top option with “Open all with current file extension as…” ? Is that what you’re looking for?

You can open a PR if you like. I’d guess 70/30 against it getting merged, but others may disagree. I don’t know that there’s an official policy, but in practice these seem to be the major tenets:

  1. New languages are exceedingly rare (make your own package instead)
  2. Objective improvements are welcome (e.g. the old behavior doesn’t match the official spec, but now we’re closer to it)
  3. Sometimes, unofficial features are acceptable if
    • Widely valuable
    • Low impact (This is where I think your changes would run afoul. The ATX Header change will negatively impact people who used the other style, even if it was working in their Markdown flavor.)
    • Some examples: GitHub usernames in commit message highlighting; My own PR for GitHub checkboxes to Markdown.
0 Likes
#9

Thanks Michael for all your help! The webpage defining the markdown is just what I was looking for!

I wasn’t planning on push my changes up stream at this point. I was more interested in determining if the file had changed in newer versions of ST3. I think that OverrideAudit package might be what I am looking for.

I plan on putting my changes in a custom package I can pull from a git repo so I can use it on all my machines.

I suck at color schemes and would rather not do one myself. I would rather rely on the scopes and someone else’s better eye. Do you know of any scopes from here (http://brianreilly.me/sublime-syntax-dashboard/#/syntaxes/Markdown) that would highlight text like “invalid.illegal.expected-eol.markdown” I really hate to co-opt invalid or illegal char colors to get a background highlight. At some point I’ll probably get into syntax coloring, but only for the files I need.

Thanks!

0 Likes
#10

I figured out how to get my changes work, w00t! Thanks for your help!

One last issue. I have the syntax file called Markdown.sublime-syntax and I have it in my custom package folder. I changed the name in the yaml file to “name: Markdown (Bluebill)” so I could see it in the view/syntax menu. When I load a *.md file, my tester file. It loads using the regular Markdown syntax. I can use view/syntax and apply my syntax to the file. As soon as I save it, it reverts back to regular Markdown. If I go to view/syntax/open all with current extension as… and choose my dialect, I can see the highlighting take effect. If I save it reverts to the old system.

Basically, I can apply my new syntax and see the changes, but it won’t remember the changes. I think it has something to do with the file extensions. It seems to pass it to the system copy of Markdown.sublime-syntax and not my version first.

Any thoughts?

0 Likes
#11

Don’t co-opt bad scopes for your purpose. Follow the scope guidelines, use your favorite color scheme and choose “Edit Current Color Scheme” from the command palette (available with PackageDev) and use it to add a new rule for your new scope, including a background color if that’s what you want.

Your rules will be combined with the color scheme in your interface. I use the built-in Mariana color scheme, but I have a couple tweaks to it for my purposes. For example, I have this rule to slightly tint the background for strings:

{
    "rules": [
        {
            "name": "Strings",
            "scope": "string, constant.character.cs",
            "background": "color(var(blue3) l(+ 2%))",
        },
    ],
}

I don’t know what’s doing that. It could be a package interfering, or (this is just a guess?) is there a Packages/User/Markdown.sublime-settings with an extensions key that is getting hit first?

0 Likes
#12

I did have the Markdown.sublime-syntax in the user folder, doh! However, after deleting it, the problem remained. I noticed that when I set the syntax view to default to my new one it created a Markdown.sublime-settings file with a list containing the extension. I figure ST did not know which syntax file to open. I renamed mine and deleted any traces in the user folder. Now it works!

Are there rules around naming syntax files (in the file system)?

Thank you for all your help. I appreciate the information about how to edit the current color scheme. It works perfectly, much easier than taking the source file and editing it.

0 Likes
#13

That makes sense. Congrats.

Not that I’m aware of, but it seems you’ve encountered one anyway: Don’t reuse a package path. There are definitely syntax files with spaces in the name, and I’ve seen supporting files with parens in the name. I haven’t tried unicode or usually reserved ascii characters.

1 Like