Hi all, just wanted to share my ST3 plugin for showing interactive code map for the content in the active view: https://github.com/oleg-shilo/sublime-codemap.
It currently supports building the code tree for Python syntax. Support for C# is in the pipeline. You can also add custom parsing algorithm to build the code tree for any alternative syntax.
Enjoy.
Looks pretty cool, but I am wondering why donāt you use the symbols as parsed by ST to build the map - if the syntax definitions are all consistent enough (which they should be for color scheme purposes) then you could support all languages at once without any extra work 
Yes it is an attractive option. However it would mean that the code tree would reflect not the content itself but its decoration (highlighting). And while indeed this approach would cover all languages it would be limited by the nature with no room for extendibility.
Instead of this, I wanted to give a completely working infrastructure with the default mapping that anyone could integrate with a custom tree building algorithm even for the content that has no associated highlighting.
The plugin comes with:
Python mapper. Very simple (~70 lines) indent based mapper that deliberately limits mapping depth to 2-levels of nesting. While syntax highlight would go all the way, producing extra mapping noise.
Text mapper. A sample for building a pure content based mapper, which has no associated highlighting. This mapper builds the map of text lines. Similarly one can adjust it to map the paragraphs or logical sections (header vs footer)
C# mapper. A comprehensive mapping that goes beyond regex analysis and relies on true Type System analysis (Roslyn). This mapper as an adapter only (integration layer) and will fully function only when I release CS-Script plugin (in a week or so). This plugin is a port of Notepad++ plugin for C# intellisense and C# script execution.
Anyway, ideally I see CodeMap as a shell, which is used by others to integrate other custom (e.g. domain specific) mappers. One of the best examples of which would be a universal mapper based on your idea - āthe symbols as parsed by STā with a possibility to switch to dedicated mappers via settings when required.
I would be happy to include any mappers contributed by the community directly to the package.
A mapper is an incredibly simple generic Python routine that takes a file path as an input, analyeses the file content and returns a map text where each line represents a map item in the following format:
<item title>:<item position in source code>
This is the mapper sample: https://github.com/oleg-shilo/sublime-codemap/blob/master/custom_mappers/code_map.txt.py
This plugin is great, but it has a problem with some python files not showing the map (try commands.py from sublimelinter package)
Also, I prefer a more compact view, I added these in the codemap.py (class show_code_map) to make it look better
code_map_view.settings().set("line_numbers", False)
code_map_view.settings().set("gutter", False)
code_map_view.settings().set("fold_buttons", False)
code_map_view.settings().set("font_size", 8)
Maybe you could add some configurable settings for how youād like the map to look. It would be even cooler if it created the map on a new vertical group of its own, instead of adding the tab to the current group.
An excellent suggestion.
The release is published: v1.0.6
And I will have a look at parsing problem. Thank you for your feedback.
Very cool, I also like that it closes the pane if you close the map, not so obvious :).
I still canāt get the map for that file (my other UTF-8 files show the map). Also, about my suggestion I found out later that disabling just gutter would also disable line numbers and fold buttons, so maybe itās better just
code_map_view.settings().set(āgutterā, False)
so one can enable/disable them in one go, if preferred.
Thanks and good work!
ā¦it closes the pane if you close the mapā¦
Yes I thought it would be a good user experience. Itās all controlled by these settings:
{
"close_empty_group_on_closing_map": true,
"show_in_new_group": true
}
I decided that maintaining font size different to other views is a bit an overkill. But the rest makes perfect sense.
ā¦disabling just gutterā¦
Will consider putting this one into settings file too. Currently hiding is hard-codded.
As for that ānaughtyā codemap.py, can you just sent it to me. May be mine somehow is different to yours. I will have a look at its encoding.
This looks very similar to something Iāve seen in another editor. It appears to only map for the current file. Would it be readily extensible to following structure across several files?
The plugin is extensible, meaning that you can easily supply your custom routine (mapper) that would parse a file (or files) and produce the map. It actually comes with the mapper for MARKDOWN syntax. Thus showing the multi-file map is not a problem at all.
Navigating to the clicked item is a challenge.
The current implementation simply opens the document that is associates with the active map and navigates to the line with the same number as the last word in the clicked map item (line). Hooking into on_click event is not a problem. I can extend it and the custom handler can be taken from the same custom mapper module.
However, if you want to achieve multi-file model you will need to think how to encode multiple documents paths into the map item. This is something that you will need to solve if you are keen to get it working.
Thank you. There is another outliner style plugin that I discovered when pointed to a feature request for permitting API access to the sidebar. Once Iām finished with my current plugin development phase, I may take up the lance and attempt to extend this for a full source tree.
Thanks, this is similar to what Iāve been looking forā¦
Iāll need to do some reading in order to create an option for GMod Lua - does is the code_map..py where syntax is supposed to be the extension or the nameā¦ ie I have it as Lua but the syntax is GMod Lua - and from what I can see you load the language file ( and I pointed to that ).
Bookmarkedā¦
For those having trouble
This is what I have so far for adding GMod Lua to CodeMap:
Note: For some
user\codemap\custom_languages\lua.sublime-syntax:
%YAML 1.2
---
# http://www.sublimetext.com/docs/3/syntax.html
name: CodeMapGModLua
file_extensions:
- lua
scope: text
contexts:
main:
- match: \b(break|do|else|for|if|elseif|return|then|repeat|while|until|end|function|local|in|continue)\b
captures:
1: keyword.control.lua
functions:
- match: \b(function)\s+([a-zA-Z_.:0-9]+[.:])?([a-zA-Z_]\w*)\s*(\()([^)]*)(\))
captures:
1: keyword.control.lua
2: entity.name.function.scope.lua
3: entity.name.function.lua
4: punctuation.definition.parameters.begin.lua
5: variable.parameter.function.lua
6: punctuation.definition.parameters.end.lua
Basically reading the Lua.tmLanguage and altering the format to match the sublime-syntax styleā¦ Not done yet, but it is a start. and Iām using code_map.py.py with a few changes for nowā¦ renamed to code_map.lua.py
There are a lot of improvements that could be made the the mapper system - ie you have the python one hardcoded, just as the md one is hardcodedā¦ It could be altered to create a nice output by using classes, etcā¦ This way you can really control the output and you wouldnāt need to create the same script for each and every language ( unless you want to do something differentā¦ )
But, as it is, with some work this looks like what Iām looking forā¦ If I create a dynamic mapper, Iāll probably submit it to the github project or post it hereā¦ Then you can define the language in python by calling the class and a few additions, then you donāt have as much code being repeatedā¦
When you are happy with you mapper you can share it and I can include it into the default CodeMap package.
Sounds good - Iām making a lot of changes - this one will be for GMod Lua, although itād work for Lua too - Iām slightly altering how you have it set up, helper functions / classes, etcā¦
Also, because I created my own coding standard, I have predefined variable names which always mean something specific so I replace _p with _p as helpful output in the function args list, for exampleā¦
Things like that will be easily definableā¦
I may add a different system for adding, ie supporting categories ( so instead of everything added to one single table - added to table[ category ][ type ] or simply table[ category ][ entry key ] = entry or similar with sorting optionsā¦
Additionally, because I use ānamespacingā in my Lua code the output is quite largeā¦
ie:
Classes / function Acecool.C.
function Acecool.C.timekeeper:New( <N> _duration, <Timestamp> _start, <N> _delay, <*> _mode ):150
function Acecool.C.timekeeper:FromTable( <*> _data ):201
would automatically remove the prefix so theyād show up as:
Classes / function Acecool.C.
function timekeeper:New( <N> _duration, <Timestamp> _start, <N> _delay, <*> _mode ):150
function timekeeper:FromTable( <*> _data ):201
to shorten the data being displayedā¦
The goal in the actual mapper script is to get rid of all of the repetitions:
# -- This is here so I can have a separate category for functions defined in my namespace sub Classes rather than it be under functions..
if code_line.startswith( 'function Acecool.C.' ):
last_type = 'function Acecool.C.'
last_indent = indent_level
_entry = ( line_num,
'Classes / function Acecool.C.',
line.rstrip( ),
indent_level, 1 )
# -- Different category than above
elif code_line.startswith( 'function ' ):
last_type = 'function'
last_indent = indent_level
_entry = ( line_num,
'function',
line.rstrip( ),
indent_level, 1 )
elif code_line.startswith( 'local function ' ):
last_type = 'local function'
last_indent = indent_level
_entry = ( line_num,
'local function',
line.rstrip( ), #
indent_level, 1 )
would become
SetupNewWatcher( string_search_for = 'function Acecool.C.', string_category = 'Classes / function Acecool.C.', boolean_strip_search_pattern_from_line = 1, string_substitute_replaced_search_pattern_with = 'function' )
or something similarā¦ ie all of it becoming 1 line definition ( and then that defs file could be read from, in the following order fall-through: Project Root Folder > User Language Folder > User CodeMap Folder > Default CodeMap Folder > Default Language folder - or similarā¦
which would make using this plugin that much easier for people that arenāt familiar with python, or other languages but use ST3 for other purposes, etcā¦
I have this project on high priority because itāll increase my efficiency on the other projects Iām working on so you can likely see a post soonā¦
Oh, questionā¦ At the top of the code_map..py file, you declare ignored, installed, and map_syntax but they arenāt used anywhere in the fileā¦ Do you use it in another file? I havenāt looked through the entire plugin yet - but Iām guessing you do use it to load certain dataā¦
Current example of output: https://www.dropbox.com/s/oejte8c9vjed26y/sublime_text_unregistered__screenshot_20170914003027_7648.png?dl=0
Is there a way to hide the minimap for the CodeMap group? My function definitions with args are longer than Iād like the group to be - and Iāve done things to shorten it for example the primary classes group Acecool.C. is removed by adding it as a category and nested components are indentedā¦ But I added data-types to the arguments field ( although I may alter that too )ā¦
Additionallyā¦ Is there a way to add icons ( specific ico files or png etcā¦ ) to the gutter, or to the file output? And, since the gutter is hidden, is there a way to enable it? I have yet to look at the main plugin code, so it could all be right in front of me but since you created it, itād take less time for you to simply reply than it would with me familiarizing myself with itā¦
The mapper is almost ready - Iām still putting a few things in place ( for example in GLua there are realms - ie where code is executedā¦ Clientside or Serverside or shared / both which is typically known by if statements but some also add the designation to the file name or folder-nameā¦ Right now for REALM switches in the context of the code I add a new category and inject additional data before the prefix so if specified in a file, it is known for those lines )ā¦
I still need to add built-in support for detecting localized functions and so on and use a special character to show it belongs to a parent instead of simply being tabbed - I will probably shorten the realm system too so it doesnāt take up important real-estate by having the category show the realm and have the same L char pointing that it is owned by a parentā¦ I am going to re-add Alias support but possibly integrate them into the output where the original definitions are which would mean changing the storage system from the original mechanismā¦ among other thingsā¦
Hereās an example of the output:
Basic idea of alias support
I am a bit confused about your post. CodeMap is designed in such a way that it is completely decoupled from the mappers. Thus the level of consistency (or inconsistency) between coding styles in both a mapper and plugin is completely irrelevant.
The same comes for your preferred naming conventions. And the ability of people to understand CodeMap internals.
The plugin extensibility model is extremely simple. You donāt need to know anything about CodeMap but just implement an analyzer for the syntax of your choice, with a very simple set of responsibilities. That analyzer is expected to produce a list of the file ābookmarksā in the form of <bookmark_name>:<line_number>. Thatās it. Nothing more nothing less.
Feel free to format the whole entry as you want or to control the overall width of the entry. You can even place some visual separators:
Category A
function_1: 20
function_2: 21
Category A
function_1: 30
function_2: 31
-----------------
Globals
function_1: 40
function_2: 41
All this is up to you and all this has no affect on CodeMap as double-clicking in the map view will ignore any decoration line and only navigate on clicking a valid bookmark entry <bookmark_name>:<line_number>.
May be I misunderstood your post but I interpreted it as a prompt for changing rather the actual plugin implementation than the developing Lua mapper. Please correct me if I am wrong.
Forgotā¦
Is there a way to hide the minimap for the CodeMap group?
Yes, I would live to be able to do that. Though donāt know if it is possible.
Additionallyā¦ Is there a way to add icons ( specific ico files or png etcā¦ ) to the gutter, or to the file output? And, since the gutter is hidden, is there a way to enable it? I have yet to look at the main plugin code, so it could all be right in front of me but since you created it, itād take less time for you to simply reply than it would with me familiarizing myself with itā¦
This goes well beyond the intended functionality of the plugin. Sorry.
The ST3 philosophy is to keep views extremely simple and to prevent developers from creating any complex (non-Text) views. I personally would prefer ST to allow me to do complex views as I can do for VSCode:
This way CodeMap would be much better suited for the purpose, butā¦ we have what we have. Despite multiple requests ST team does not want to allow custom views or custom toolbars. And I am not sure itās gonna change.
Thatās why any attempt to mimic comprehensive visual experiences with pure text is going to be extremely costly. And, just considering a simple āeffort vs. benefitā ratio makes me think that evolving CodeMap visual complexity is very unlikely.
Ironically the CodeMap itself is an attempt to mimic a TreeView with a pure text. Yes it works but there are only so much that can be done. Just compare it to my CodeMap for Notepad++. Itās so much better visually:
Thanks - Iām aware of the
<bookmark name>:<Jump-To-Line>
format - and the mapper Iām writing is a simplified way to add search / replacers and methods to format the output to make it cleanerā¦ Think - a mapper could be made as a definitions file which includes a base mapper - ie
# Create the Categories we want to show ( Category Name, Category-Wide Actions to be applied to the entry, other flags )
<ENUM / Key> CATEGORY_FUNCTIONS = CodeMap.AddCategory( "Functions", { LINE_STRIP_RIGHT }, { CODEMAP_FUNCTIONS_CATEGORY } );
<ENUM / Key> CATEGORY_ENUMS = CodeMap.AddCategory( "ENUMeration" );
# Set the order in which the categories show in the output
CodeMap.SetCategoryOrder( CATEGORY_ENUMS, CATEGORY_FUNCTIONS );
# Other Config
# Indentation will be 2 spaces despite what is used in the file
CodeMap.SetIndentationChars( ' ' );
# Should we hide functions, etc... declared within other functions or show them indented?
CodeMap.HideNestedDeclarations( False );
# How should we display nested declarations? Examples of using the CharacterMap tree characters to clean up the look - Obviously there will be more config options, ie which char to show when it is the last in the list, the first, or in-between, and which to show when there are others ie double-line with 1 line going to it and the left one going underneath and so on...
CodeMap.SetNestedDeclarationsChars( ' ā' );
CodeMap.SetNestedDeclarationsChars( ' ā' );
CodeMap.SetNestedDeclarationsChars( ' ā' );
CodeMap.SetNestedDeclarationsChars( ' ā' );
CodeMap.SetNestedDeclarationsChars( ' ā' );
# May set up args to look like: ( FirstWithoutOthers, FirstWithOthers, Center, End ) which would look like:
CodeMap.SetNestedDeclarationsChars( 'ā', 'ā', 'ā', 'ā' ); # But it could be shorted to Multiple, End ie the last 2 which would work with end being first without others and end, multiple being first with others and center meaning others...
# Adding a way to match code to a category ( Category Key, Match String, Match Start of Line or end of line or whatever ENUM so MATCH_LINE_REGEX MATCH_LINE_STARTSWITH MATCH_LINE_ENDSWITH, Flags varargs which can be MATCH_ or it can be things such as STRIP_LINE_LEFT STRIP_LINE_RIGHT STRIP_LINE LINE_TO_LOWER LINE_TO_UPPER ) or ( Category Key, Matching Text, Matching Options as a single option or an array, Entry Options as a single option or an array )
CodeMap.AddCategoryMatch( CATEGORY_FUNCTIONS, 'function ', MATCH_LINE_STARTSWITH, { LINE_STRIP_RIGHT, LINE_STRIP_MATCH_TEXT ( This will remove function from the entry although this option could be applied to the category add code } );
# So we know to skip things in comments - more specifically to skip things in block comments where string.startswidth would be "fooled"...
CodeMap.AddCommentLineMatch( '//' )
CodeMap.AddCommentLineMatch( '--' )
CodeMap.AddCommentBlockMatch( '--[[', ']]' )
CodeMap.AddCommentBlockMatch( '/*', '*/' )
Etcā¦ In short, what I am doing is making it easier to create mappers for new languages - right now it doesnāt seem you can create the lua.sublime-syntax and have it do the work for you - it seems like you need to create the mapperā¦ And the mappers, right now, is a lot of repetitive codeā¦ if code_line.startswith: Add it elif re.find ā¦ elif blah blah blahā¦
What I am doing simply makes CodeMap available to a wider audience, quicker than would otherwise itād be available to them because it saves them the time of creating a long / elaborate or repetitive code mapper by letting them make a few simple function calls and options to define or by making it easier for you to add languagesā¦
Additionally, because there are built-in markers to detect when youāre in a comment-block, for example, it wonāt wrongly match functions within those blocks and it has built in functionality to enable those features ie what do you want to do when you come across a nested functionā¦ or what do you want to do for nested whateverā¦ what about classesā¦ etcā¦ In the category definitions function you simply define the behaviorā¦
Everything to make things easier - so far I am enjoying CodeMapā¦
IE some basic features in my eyes would be a way to sort the function list by name, ensure anything categorized is held together so if functions is a category name, and enumerations is another, then you can always force all enumerations to display at the top ( I code in that way, but if someone else doesnāt and they want to show that category at the top, then itād be available to them )ā¦
Things of that nature is what Iām doingā¦ Also the data-typing for languages which donāt have strict declarations in the function declaration - and for things that do such as C++ if a declaration is made at the top, and the real function is at the bottom then they can optionally be hidden or shown differently than in the functions categoryā¦ etcā¦
A few questions: Is it possible to have the CodeMap in a separate window ( I prefer to have more room to code and Iāve shortened a lot of the definitions quite a lot [ Although Iāll probably add other keywords so function can be shortened to func and the rest of the code be properly highlighted, and so on to get everything shorter ] but it isnāt short enough - When I move CodeMap to a different window it isnāt updated and when I move it back, itās fine )ā¦?
In Notepad++ I had my functions list ( and the snippets panel ) on my right monitor docked to the left side of the monitor, and on the left monitor I had my file-tree docked to the right side, and so onā¦
This is one of the main reasons not being able to have separate panels in ST3 is such a drawback - but if you know a way where it can be updated in a separate window, thatād be fantastic ( to simplify behavior, the last file opened / selected, as it currently is, would be what CodeMap displays if it has a mapper for it )ā¦
I agree - being able to have panels would increase efficiency as sometimes complex panels are neededā¦ Having code analysis graphs generated would be nice, ie how much of the file size is from spaces, tabs, strings ( which should be in a language definitions file ), etcā¦
Weāre incredibly limited - but ST3 is so much faster than Notepad++ because of the properly done threading system, and pre-indexing projects and so onā¦ But panels are a requirementā¦ They need to be addedā¦ because being limited to one window for your project is hell because the file-tree can easily take up 25 to 50% of a 1920x1080 monitor, then CodeMap can take up 15 to 50% easily ( Iām trying to shave off as much as possible while providing all of the necessary information I need at my fingertips ) and not being able to limit the search / replace thing at the bottom makes stretching the window across multiple monitors impossible and non-usable - if CodeMap and the file-tree could be extended all the way to the bottom, then using multiple monitors for 1 window wouldnāt be that bad but panels would solve all of these issuesā¦
I would switch to Atom but it has the same problems as Notepad++ so itās slow and so many things were done incorrectlyā¦
This is the current lua.sublime-syntax in user/codemap/custom_languages/ for GMod Lua - a few custom things marked Acecool can be removed along with the _ variables ( Should be everything starting at line 258 AcecoolDev_Framework Namespace
But even if the language is defined, it doesnāt seem to do anything - but again I havenāt looked through the package code to see what is done with itā¦
Thank you, I will try to find the time to go through your code with the attention it deserves. Though, did you also want to share the mapper?
And the mappers, right now, is a lot of repetitive codeā¦ if code_line.startswith: Add it elif re.find ā¦ elif blah blah blahā¦
Agree 100% 
And it is exactly why I am reluctant to see any ācode parsingā in CodeMap regrdless how good or bad it is. I want to push it completely to the mappers. Even if the parsing is done in some generic and very dev-friendly way. My argument is a pore IoC reasoning.
I would rather have some generic parsing util/module available for mapper creators instead. But it needs to be completely decoupled form the CodeMap codebase, even if they are distributed together. Though, as I said, I do not want to dismiss it completely without giving it a chance.
As for panels, you will not find a bigger supported then me. After creating quite a few plugins for Visual Studio, VSSCode, Notepad++ and ST3 I struggling to see how such a beautifully crafted creature, that ST3 is, can continue dismiss the request for custom panels support so arrogantly.
Is it possible to have the CodeMap in a separate windowā¦
I understand and fully agree your motivation for this feature. I think it makes sense to make this option available. Can you please log the feature request on GitHub: https://github.com/oleg-shilo/sublime-codemap