Hi Keith
Often times I use docblocks and I even created macros to add/remove lines from them, the issue is that sublime keeps indenting after the block when I try to reindent:
The ideal would be for the variable declaration to be aligned with the /** and not the */, is there a solution for this case?
Thank you
Hi, great question - thanks for asking it 
yes, this is solvable - in fact by the idea I posted above if you can let me know which language syntax you are using, I’ll provide some step by step instructions on how to do it
also, while trying this out, it revealed something I hadn’t quite realized previously. I turn off preserveIndent for comments, because otherwise the reindent command doesn’t touch them and it messes up the rest of the indentation. But with it off, it unaligns the * in docblocks… I will need to explore to see if there is a suitable solution for both at once 
I am using MQL4 which is close to C++ but with less features and I am trying to simplify it and fix some issues like the indentation.
And yes, I tried disabling preserveIndent, but it was leaving the doc blocks unaligned too. Thanks for the article, it helped me understand some of the errors I was having before.
step 1 will be to edit the syntax definition to include a meta scope for when only whitespace follows the block comment:
contexts:
main:
- match: \/\/
scope: punctuation.comment.begin.mql.mql4
set: line
- match: \/\*
scope: punctuation.comment.begin.mql.mql4
set: block
line:
- meta_scope: comment.line.mql.mql4
- meta_content_scope: comment.content.mql.mql4
- match: $
pop: true
block:
- meta_scope: comment.block.mql.mql4
- meta_content_scope: comment.content.mql.mql4
- match: \*\/
scope: punctuation.comment.end.mql.mql4
set: # added to set a meta scope for when only whitespace follows the end of the block comment
- match: \s*$\n?
scope: meta.whitespace-after-comment-block-end.mql.mql4
pop: true
- match: ''
pop: true
step 2 will be to create a new .tmPreferences file with the following content:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>scope</key>
<string>comment.line.mql.mql4, comment.block.mql.mql4, meta.whitespace-after-comment-block-end.mql.mql4</string>
<key>settings</key>
<dict>
<key>unIndentedLinePattern</key>
<string>.</string>
</dict>
</dict>
</plist>
and it should now keep the variable declaration aligned with the /** instead of the */ when you reindent
Thank you @kingkeith for this elaborate topic. I tried to create a tmPreferences file for the language Omnis Studio, but I do not know where to put the file. I googled, came across this page a couple of times, but could not find any documentation about this. Can you tell me where to put this file? And do you have to do more then creating this file to get the indention to work or do I also have to add meta_scope lines to the tmLanguage file?
good point, I didn’t really cover that. If you, from ST, go to the Tools menu -> Browse Packages and then:
.tmPreferences file thereabout scopes, it depends what you are trying to achieve specifically and whether the existing scopes are suitable, but quite often it isn’t necessary as regex is powerful enough without specific scopes
Hi kingkeith,
thanks for this very informative thread, I stumbled across it since I’m trying to improve automatic indentation for vhdl. I’m very unhappy with the current state of the auto-indent, mainly because it only supports one-line patterns and decreasing one level of indentation.
I tried playing around with scopes (and adding . indent patterns for specific scopes) but it turned out to be rather tedious and didn’t even work in some cases (strange behaviour when including or not the line ending character in the scope…)
Anyway, I wanted to debug and potentially edit the auto-indent functions but I couldn’t find the code for it. Many commands are implemented in the Default packet but I couldn’t find indentation. Do you have any idea where to find the code that is run when using “reindent” command?
thanks a lot!
jk
that code is part of ST core, and not implemented in Python, so it is closed source and we can’t see it.
It’s partially the whole reason for this thread, as we can’t see the code - we rely on guesswork as to what it does, so I tried to document its behavior as best as I could, along with it’s limitations, in the hope that a future build of ST will have a better indentation engine (IMO, ideally decoupled from core and implemented as a Python plugin)
It’s not using the ‘autoindent’ feature, however my VHDL package does have a beautification function which if you tear it apart may be of some use to you. Basically it left justifies everything and then reindents based on language rules. It’s not perfect, but it has done a pretty good job for me at least so far.
In retrospect, I may have recreated the indentation file to some extent because I based it around a ‘current_indent’ and ‘next_indent’ model, but I have space for custom behavior. For instance after an instantiation, we usually indent, and then indent again for port map. We’d like the closing parens for port map to be aligned with the port map statement, but the following statement should be on the line aligned with the original instantiation. It seems to do that pretty well. Also had to create a custom ability for when a line ends with the closing symbol (parenthesis usually for VHDL) and when that’s put on a separate line. Different code styles treat this differently (K&R versus Allman versus Lisp).
Another thought, it’d be kind of cool if indent patterns could be included in a syntax file. The two are very related and it might be possible to get more fine grained control.
EDIT 2: Actually working on some fresh code (as opposed to maintaining existing code) and the need for a good auto-indenting model for the language is becoming apparent. I can tell that I’m having to hit tab a lot more than I remember having to with vhdl-mode in Emacs.
Hi Remi,
thanks for the remark, I will certainly take a look into your vhdl package.
I started working on an indentation tool that will allow the user (or language package) to define some opening and closing patterns (supporting multiline pattern) between which the code will be indented. It should easily handle situations like instantiation since the closing semicolon of the port map can close (in terms of indentation region) the port map itself as well as the instantiation. Thus also the problem of switch/case statements can be solved.
At first it’s going to be an “offline” tool like yours to unindent and reindent the complete code. But I’ll try to get the automatic indentation on a newline out of it as well, shouldn’t be too hard…
I’ll keep you updated if anything usefull comes out.
It’s actually a in-editor function and works better in-editor. I did create a standalone beautifier wrapper that uses the same method, but since it doesn’t have access to scope information, the alignment routine gets slightly more stupid. It was mainly for debug though because it’s easier to go through a huge file of stdout logging than try to parse through the ST console.
Anyway, good luck! Always interested in work done for the language.
What about simply indenting based on scope nesting?
Well I was under the impression (from the discussion at the top of the page) that the auto indent was based on those Textmate Preferences flags. These do rely on the lexical scope though, which is why I thought it might be interesting to put them in with that (if desired). But honestly I’ve not looked into the auto-indent functions though I need to probably for the next feature stage and try to make sure there’s something sane in place. Currently there’s no auto-indent information for VHDL that I was able to notice, so it’ll just return to the point of last indent after a CR/LF.
For my own beautifier, I do a lot of pre-process work (removing whitespace, aligning assignment operators, etc.) on a list of the lines in the file and I rapidly lose connection with the original lexically scoped source.
I’m trying to figure out a better indentation syntax for Erlang. Is it possible to de-indent a line based on the previous line? In Erlang, function syntax looks like so:
foo(A, B) ->
line1,
line2
line3.
bar() ->
line1,
line2.
A function is a header, a list of statements separated by commas (,) and concluded with a dot (`.). I want the de-indent to happen on the line after the dot, otherwise Sublime Text indents the next function at the same level as the body of the previous function. Is this possible at all?
Great question!
I’ve been experimenting, and unfortunately, I can’t see a way to do it. I had the idea that it would be possible, but not purely from the .tmPreferences metadata file - I thought it might work if the syntax definition were to scope the line after the dot, so that it could be targeted specially and use different rules for that line than for the rest of the Erlang syntax (and those rules would just always tell ST to de-indent the line). My attempt:
--- Shipped Packages/Erlang/Erlang.sublime-syntax 2017-10-10 16:52:54
+++ Packages/Erlang/Erlang.sublime-syntax 2017-10-24 16:27:15
@@ -239,11 +239,20 @@
1: entity.name.function.definition.erlang
push:
- meta_scope: meta.function.erlang
- match: \.
scope: punctuation.terminator.function.erlang
- pop: true
+ set:
+ - meta_scope: test
+ - match: \s*$\n?
+ scope: meta.after-function-terminator.erlang
+ set:
+ - match: ^\s*
+ scope: meta.unindent-me.erlang
+ pop: true
+ - match: ''
+ pop: true
- match: ^\s*+([a-z][a-zA-Z\d@_]*+)\s*+(?=\()
captures:
1: entity.name.function.erlang
- match: (?=\()
push:
--- Shipped Packages/Erlang/Indentation Rules.tmPreferences 2017-04-12 09:40:46
+++ Packages/Erlang/Indentation Rules.tmPreferences 2017-10-24 16:10:40
@@ -8,9 +8,9 @@
<key>settings</key>
<dict>
<key>decreaseIndentPattern</key>
<string>^\s*\b(end)\b</string>
<key>increaseIndentPattern</key>
- <string><![CDATA[^[^%]*(\b(if|case|receive|after|fun|try|catch|begin|query)\b(?!.*\b(end)\b.*))|(->(\s*%.*)?$)]]></string>
+ <string><![CDATA[^[^%]*(\b(if|case|receive|after|fun|try|catch|begin|query)\b(?!.*\bend\b)|->(\s*%.*)?$)]]></string>
</dict>
</dict>
</plist>
New file: Packages/Erlang/Indentation Rules - Unindent After Function Dot.tmPreferences:
<?xml version="1.0" encoding="UTF-8"?>
<plist version="1.0">
<dict>
<key>name</key>
<string>Indentation Rules</string>
<key>scope</key>
<string>source.erlang meta.unindent-me.erlang</string>
<key>settings</key>
<dict>
<key>decreaseIndentPattern</key>
<string>.</string>
<!-- <key>unIndentedLinePattern</key>
<string>.</string> -->
<key>increaseIndentPattern</key>
<string></string>
</dict>
</dict>
</plist>
But, because the line between the end of the function and the next function is blank, it seems to be ignored by the indentation engine when calculating the indentation for the next function (bar) in your example. Not having a blank line between the two (assuming its even valid Erlang syntax) means that ST has no way to add the new “unindent me” scope, so that’s not a solution/workaround either.
I think the best one can do at the moment is create a macro so that when you press Enter after the . that marks the end of a function, it would automatically perform a left_delete after the newline is added, to unindent one level. But this of course doesn’t help for formatting existing (parts of) documents.
Thanks for such an in depth investigation!
This is my conclusion as well.
Working with the indentation system in Sublime, I can only conclude that it is very limited. I continuously wanted all kinds of variants of the existing options, such as “decreaseIndentPreviousLinePattern”, “increaseIndentCurrentLinePattern” etc.
I’m not sure if it is feasible, but something based on the scopes defined in the actual syntax file would be great. Having to work with very cumbersome and limited regex options inside an XML settings file is far from ideal.
I’ve experimented with just such a system. You’d want to do something like:
meta.*)string.*).This should work for languages with comprehensive meta scopes. It wouldn’t work for languages that don’t have those meta scopes, probably including a lot of languages with old tmLanguage syntaxes and definitely including most indentation-based languages.
Really useful article, thanks! I raised an issue for the problem you already mentioned “lines that don’t conform to the tab stop size” - perhaps you could link it?
I also raised github issues 2029 and 2030 for new problems “Reindent forces indent on line that should have none” and “Indendation wrong for CSS close brace with comment” (I didn’t insert direct links as it seems that new users are only allowed two links in a post!)
I have some more observations on comment indentation after testing with PHP and Java.
After fixing Indentation Rules - Comments.tmPreferences as you suggested I also had to edit the language specific file Indentation Rules Annex.tmPreferences. I removed // from unIndentedLinePattern which allows single-line comments to be re-indented.
In PHP and Java, the re-indent command does not corrupt DocBlock comments. The fix comes from the same setting as in item 1, which excludes lines with ’ *’ and ’ */’ from indenting.
This combination works pretty well and I would recommend it for all languages: