Sublime Forum

Markdown syntax highlighting - indentation

#1

I’m no Markdown guru but have been using it recently for technical documentation in an effort to streamline my git workflow re. code & associated docs. I’ve stumbled into a little issue with Sublime’s syntax for Markdown and am looking possibly for a fix or just some guidance :wink:

Working on potentially large docs I’m after the ability to use ST’s code folding as a kind of ‘outline view’ of my document. This depends upon indentation, so under my headings I’m indenting everything by a couple of spaces to allow me to collapse them. However, the syntax highlighting seems to break with lists (numbered and unordered) even if I have blank lines terminating the list. Eg.:

# Title {#title}

## Heading 1

  Lorem ipsum dolor sit amet, consectetur adipisicing elit. Sint distinctio, pariatur, doloribus iure delectus vero odit. Rerum maiores vero alias voluptate omnis? Aliquid consequatur voluptatum vitae blanditiis eveniet perferendis suscipit!
 
  1. Some text
  2. Some more text
  3. Even more text

  More paragraph text...

## Heading 2

The problem being that in Sublime, ‘More paragraph text…’ (indented by 2 spaces) is still considered as part of the list. It only appears correctly if I remove its 2-space indent at the start. This is even so if the list is further indented than the text below, say by 4 spaces.

My understanding is that if there’s a blank linespace after a list and the following line doesn’t begin with a number (ordered) or special bullet (unordered), markdown considers the list ended, and it appears that way when I preview my markdown using github or python-markdown via the MarkDownPreview package.

So my question is this: Am I missing something here, or is there room to improve the bundled Markdown syntax package? Possibly resident markdown guru @deathaxe can offer some advice?

TIA :slight_smile:

0 Likes
#2

I realize I’m not the Markdown expert you requested, but maybe I can shed some light on the situation. Markdown has a concept called loose lists, and it means that list items can have blank lines in them. Sublime Text’s lexer is limited to looking at one line at a time, and also there is no easy way to correctly handle indentation when list item numbers become double or more digits. i.e. a list with 10 or more items.

I can’t be entirely concrete right now as I am on mobile, but iirc it means that lining up text like:

9. blah
10. more text

    continued list item

is especially hard for the grammar to detect, even without the leading indentation on the whole list, as in some cases it may think the list ended and an indented code block appeared etc.

out of curiosity, what effect do you notice regarding the incorrect/missing detection of the list ending? does it affect your keybindings, or color scheme?

once ST gains better support for syntax aware folding, I guess you’d no longer have as much benefit from indenting everything under the headings by two spaces, in case that is any consolation :wink:

that said, maybe there is a way to fix this concrete scenario, I don’t have any time to investigate though…

1 Like
#3

Hi @kingkeith of course I appreciate any input however knowledgeable haha :wink: Thanks for the link re. loose lists, I’d already read that and was left slightly confused between the spec and realworld behaviour, and having re-read it hasn’t really cleared things up.

In the example from my original post, all three MarkDownPreview renderers behave as I would expect; ie. ‘More paragraph text…’ is not considered part of the list and is rendered as a plain paragraph in HTML. This may (or may not) contradict the spec. Furthermore, I created a little example using the ‘Try me’ from the commonmark site, and that works as I expect too. Here it is:

https://spec.commonmark.org/dingus/?text=%20%20-%20foo%20 %20%20-%20bar %20%20-%20baz %20%20Hello

The above case is a tight list. But it works correctly with loose lists too:

https://spec.commonmark.org/dingus/?text=%20%20-%20foo %20 %20%20-%20bar %20%20-%20baz %20%20Hello

That leaves a couple of questions. First, in the case of lists what determines their end? What I gleaned from the spec examples, though it’s not entirely clear, is that the bullet type (or numbering) is a factor in determining continuation of the list, together with indentation. This could explain why the above examples render correctly ie. line ‘Hello’ doesn’t start with a bullet. The second question is that the above examples also render correctly with github/gitlab/python markdown, which would suggest that they’re behaving correctly. So I’m led to believe that the Sublime Markdown syntax has room to improve, though I accept there might be difficulties with this edge-case.

The colour scheme. Using Base16 Tomorrow Night, lists appear all red where plaintext is white. I quite like this, but plaintext paragraphs following lists remain red which is annoying. Scopes markup.list.numbered.markdown and meta.paragraph.list.markdown are being applied.

I suppose one workaround could involve setting these scopes to white text in the colour scheme, then adding specific colour overrides for

markup.list.numbered.bullet.markdown
markup.list.unnumbered.bullet.markdown

That way, the bullets and numbers will be coloured to indicate lists, but their text will not. Though the scopes are wrong, that would make things much more pleasant on the eye :slight_smile:

That would be a consolation and would indeed obviate the need for indenting. Do you know something I don’t? :wink:

Appreciate your input, thanks :slight_smile:

0 Likes
#4

A paragraph which whose indention is smaller than the beginning of the content of the list item. In the following example More paragraph text ends the list as its indention is smaller than column number of Even more text.

 1. Some text
   
   list content

20. Some more text

    list content

300. Even more text

  More paragraph text...

In other words: To correctly detect list content, the syntax would need to count the spaces in front of the list numbers + the length of the list number + dot + space and than use that information to check whether the next none-empty line belongs to the list or not.

We count spaces to detect such things in YAML or PackageDev’s sublime-syntax syntax, but as a list in markdown can be indented by spaces and tabs, I guess it would be quite hard and hacky to implement.

0 Likes
#5

Counting indentation characters in YAML’s syntax is actually pretty naive, as we only do it for block scalars and because the effective indentation must be at least as high as that of the opening line but the indentation for the block is based on the first content line, we actually ignore the minimum indentation and only track the indentation of the first content line.

E.g. the following is highlighted as a string in ST but is actually invalid:

a:
  b:
    c: |-
  sdsdsadasdsad

TL;DR Sublime Text’s limitations on syntax definitions make features like this infeasible or outright impossible to implement properly.

1 Like
#6

I’ve had a closer look both at the markdown spec and sublime-syntax definition to learn more, concurring with @FichteFoll that edge-cases such as this are going to be difficult (aka. hacky) to cater for. Furthermore, markdown appears to have some quirks with regard to indenting so, for example, any whitespace preceding headings isn’t allowed.

For now, the best workaround I can find is to modify the colour scheme so that text for list items is coloured the same as plain text. It’s a fairly simple override and I still get the benefit that bullets (numbered or otherwise) are coloured, clearly indicating lists. The only downside is that the scope for paragraphs like ‘More paragraph text…’ is wrong - but it looks right enough and that’s what matters.

Part of the problem here is my wanting to use code folding, which needs whitespace indentation to work. In this regard the best solution is syntax aware code folding, a longstanding ask hopefully on the roadmap for Sublime.

Thanks to all for your thoughts :slight_smile:

0 Likes
#7

You might try the SmartMarkdown package and add these keybindings.

//SmartMarkdown Package
// Fold/unfold current heading cursor is in 
{
    "keys": ["ctrl+shift+["],
    "command": "smart_folding",
    "context": [{
        "key": "selector",
        "operator": "equal",
        "operand": "markup.heading.markdown"
    }]
}
// Fold/unfold all
,{
    "keys": ["ctrl+shift+]"],
    "command": "global_folding",
    "context": [{
        "key": "selector",
        "operator": "equal",
        "operand": "text.html.markdown"
    }]
}
1 Like
#8

Here’s my solution: Open the file in Atom, with its MarkdownPreview package installed, and hit Ctrl-Shift-M to run the Markdown Preview: Toggle command. This opens a window with beautifully rendered text, right next to your source code, updated in real time. The only way to fly. Check it out.
P.S. There’s a setting to use Github CSS styles, if that’s where you’re going to be putting your code.
P.P.S. This one of the few things I do in Atom, at the moment.

0 Likes