Sublime Forum

Embedded syntax highlighting in xml

#1

Hi, I’ve been trying to add python syntax highlighting withing certain xml tags to the general xml syntax highlighting. I copied the default tag definition in the main context, and changed it so it matches my target tags and applies the python highlighting:

- match: '(<)(exec|virtual|condition|validate)([^/>\s]*)'
  captures:
    1: punctuation.definition.tag.begin.xml
    2: entity.name.tag.localname.xml
  push:
    - meta_scope: meta.tag.xml
    - include: tag-stuff
    - match: '>'
      scope: punctuation.definition.tag.end.xml

      set: Packages/Python/Python.sublime-syntax
      with_prototype:
        - match: (</)(exec|virtual|condition|validate)
          captures:
            1: punctuation.definition.tag.begin.xml
            2: entity.name.tag.localname.xml
          push:
            - meta_scope: meta.tag.xml
            - include: tag-stuff
            - match: '>'
              scope: punctuation.definition.tag.end.xml
              pop: true
          pop: true

This highlights the python code fine, but I am running in the following problem - when both the opening and closing tags are on the same line, it sometimes fails to pop the python syntax. Here are some example with the the scopes on the closing tag:

<exec> a = 2</exec> <!--pops: text.xml punctuation.definition.tag.begin.xml-->
<exec> a = b </exec> <!--pops: text.xml punctuation.definition.tag.begin.xml-->
<exec> a = b</exec> <!--does not pop with space removed: text.xml source.python meta.qualified-name.python punctuation.definition.tag.begin.xml-->
<exec> import test </exec> <!--does not pop: text.xml source.python meta.statement.import.python punctuation.definition.tag.begin.xml-->

I think this has something to do with the meta scopes since the “meta.tag.xml” is not applied to the closing tag regardless of anything at this point. I don’t really know how to add that, maybe it tries to find it in the python syntax file and that fails? Anyways any help would be appreciated here…

0 Likes

#2

I didn’t actually try the following one. But use embed / escape will make it more easier and maintainable.

- match: '(<)(exec|virtual|condition|validate)([^/>\s]*)'
  captures:
    1: punctuation.definition.tag.begin.xml
    2: entity.name.tag.localname.xml
  push:
    - meta_scope: meta.tag.xml
    - include: tag-stuff
    - match: '>'
      scope: punctuation.definition.tag.end.xml
      embed: Packages/Python/Python.sublime-syntax
      escape: (?=(</)(exec|virtual|condition|validate))
    - match: (</)(exec|virtual|condition|validate)
      captures:
        1: punctuation.definition.tag.begin.xml
        2: entity.name.tag.localname.xml
      push:
        - meta_scope: meta.tag.xml
        - include: tag-stuff
        - match: '>'
          scope: punctuation.definition.tag.end.xml
          pop: true
      pop: true
2 Likes

#3

That works great, thanks! Looks like I missed this key part of the docs
“While similar to push, it pops out of any number of nested contexts as soon as the escape pattern is found.” so now the python syntax can’t mess things up :wink:

0 Likes

#4

… and the embed prevents possible recursions, which with_prototype suffers from.

One additional note: I’d suggest to always include/embed syntaxes by scope name rather than syntax file path.

   embed: scope:source.python
1 Like

#5

I wonder what happens if there are multiple packages creating syntax using scope: source.python as their default scope. Say I have python1.sublime-syntax, python2.sublime-syntax, …etc

0 Likes

#6

I think it will use the last one loaded.

0 Likes

#7

Hi all,

Can you please specify where exactly we should put the code above?

Thanks!

0 Likes

#8

The following syntax extends ST4’s builtin XML.sublime-syntax by prepending tag context.

%YAML 1.2
---
name: XML (Python)
scope: text.xml.python
version: 2

extends: Packages/XML/XML.sublime-syntax

# file_extensions:
#   - xml

contexts:

###[ XML TAGS ]###############################################################

  tag:
    - meta_prepend: true
    - include: python-tags

###[ PYTHON XML TAGS ]########################################################

  python-tags:
    - match: (<)(exec|virtual|condition|validate)(?=[/<>\s])
      captures:
        1: punctuation.definition.tag.begin.xml
        2: entity.name.tag.localname.xml
      push: python-tag-content

  python-tag-content:
    - meta_scope: meta.tag.xml
    - match: '>'
      pop: 1
      scope: meta.tag.xml punctuation.definition.tag.end.xml
      embed_scope: meta.embedded.python.xml source.python.embedded.xml
      embed: Packages/Python/Python.sublime-syntax
      escape: (?=(</)(exec|virtual|condition|validate)[/<>\s])
    - match: />
      scope: punctuation.definition.tag.end.xml
      pop: 1
    - match: \?>
      scope: invalid.illegal.bad-tag-end.xml
      pop: 1
    - include: tag-end-missing-pop
    - include: tag-attribute
0 Likes

#9

I’m using Sublime 3 and I don’t have this folder: Packages/XML/XML.sublime-syntax. What should be my embed?

0 Likes

#10

A variant for ST3 would probably look like:

%YAML 1.2
---
name: XML (Python)
scope: text.xml.python

# file_extensions:
#   - xml

contexts:
  main:
    - include: python-tags
    - include: scope:text.xml

###[ PYTHON XML TAGS ]########################################################

  python-tags:
    - match: (<)(exec|virtual|condition|validate)(?=[/<>\s])
      captures:
        1: punctuation.definition.tag.begin.xml
        2: entity.name.tag.localname.xml
      push: python-tag-content

  python-tag-content:
    - meta_scope: meta.tag.xml
    - match: (?=>)
      set: python-tag-body
    - match: />
      scope: punctuation.definition.tag.end.xml
      pop: true
    - match: \?>
      scope: invalid.illegal.bad-tag-end.xml
      pop: true
    - include: scope:text.xml#tag-stuff

  python-tag-body:
    - match: '>'
      scope: meta.tag.xml punctuation.definition.tag.end.xml
      embed_scope: meta.embedded.python.xml source.python.embedded.xml
      embed: Packages/Python/Python.sublime-syntax
      escape: (?=(</)(exec|virtual|condition|validate)[/<>\s])
    - match: ''
      pop: true

You have! It is one of the default packages shipped with ST. It’s located in a Packages folder in ST’s installation directory.

0 Likes

#11

Thanks, it works!

0 Likes

#12

Hey again,
Can we add syntax highlighting for more than one language, for example - for Python and CSS within XML? I need to highlight the Python (as given above) plus the CSS code as below:

<style name="question.header" mode="before">
        <![CDATA[
<style type="text/css">
.D3_maxdiff tr.maxdiff-header-legend {
    background-color: transparent;
    border-bottom: 1px solid #2e4053;
}
</style>
        ]]>
</style>
0 Likes

#13

Wouldn’t be a bad idea to also wrap python into CDATA as it may also contain < and > signs which would need to be escaped otherwise.

Your code below is actually HTML being embedded in XML. I’ve extended the example above and renamed tags to be a bit more descriptive.

%YAML 1.2
---
name: XML (Python & HTML)
scope: text.xml.python

# file_extensions:
#   - xml

variables:
  tag_name_break_char: '[/<>\s]'
  tag_name_break: (?={{tag_name_break_char}})


contexts:
  main:
    - include: embedded-python-tags
    - include: embedded-html-tags
    - include: scope:text.xml

  xml-tag-attributes-common:
    - match: />
      scope: punctuation.definition.tag.end.xml
      pop: true
    - match: \?>
      scope: invalid.illegal.bad-tag-end.xml
      pop: true
    - include: scope:text.xml#tag-stuff

###[ EMBEDDED PYTHON TAGS ]###################################################

  embedded-python-tags:
    - match: (<)(exec|virtual|condition|validate){{tag_name_break}}
      captures:
        1: punctuation.definition.tag.begin.xml
        2: entity.name.tag.localname.xml
      push: embedded-python-tag-content

  embedded-python-tag-content:
    - meta_scope: meta.tag.xml
    - match: (?=>)
      set: embedded-python-tag-body
    - include: xml-tag-attributes-common

  embedded-python-tag-body:
    # everything within <exec> and </exec> is highlighted as python code
    - match: '>'
      scope: meta.tag.xml punctuation.definition.tag.end.xml
      embed_scope: meta.embedded.python.xml source.python.embedded.xml
      embed: scope:source.python
      escape: (?=(</)(exec|virtual|condition|validate){{tag_name_break_char}})
    - match: ''
      pop: true

###[ EMBEDDED HTML TAGS ]######################################################

  embedded-html-tags:
    - match: (<)(style){{tag_name_break}}
      captures:
        1: punctuation.definition.tag.begin.xml
        2: entity.name.tag.localname.xml
      push: embedded-html-tag-content

  embedded-html-tag-content:
    - meta_scope: meta.tag.xml
    - match: '>'
      scope: meta.tag.xml punctuation.definition.tag.end.xml
      set: embedded-html-tag-body
    - include: xml-tag-attributes-common

  embedded-html-tag-body:
    # leave context if </style> tag is matched
    - match: (?=(</)(style){{tag_name_break_char}})
      pop: true
    # highlight CDATA content as HTML
    - match: (<!\[)(CDATA)(\[)
      captures:
        0: meta.tag.sgml.cdata.xml
        1: punctuation.definition.tag.begin.xml
        2: keyword.declaration.cdata.xml
        3: punctuation.definition.tag.begin.xml
      embed_scope: meta.tag.sgml.cdata.xml meta.embedded.html.xml source.html.embedded.xml
      embed: scope:text.html.basic
      escape: ']]>'
      escape_captures:
        0: meta.tag.sgml.cdata.xml punctuation.definition.tag.end.xml
    # the rest is XML
    - include: scope:text.xml

grafik

0 Likes

#14

How can I extend an embedded syntax? Embedding C++ works great, but now I want to escape <<tags>> with double angle brackets within the embedded C++ code.

I guess I could define a special C++ with Tags.sublime-syntax where I extend C++. Is there an easier way?

%YAML 1.2
---
name: noweb
scope: source.nw
version: 1
file_extensions: [nw]

contexts:

###[ DOCUMENTATION ]###############################################
  main:
    - meta_scope: comment.block.documentation
    - meta_prepend: true
    - include: code-chunk

###[ CODE ]########################################################

  code-chunk:
    - match: '^(<<)(.*)(?=>>=)'
      captures:
        1: punctuation.definition.generic.begin
        2: variable.other
      push: code-chunk-content

  code-chunk-content:
    - match: '>>=$'
      pop: 1
      scope: meta.tag.xml punctuation.definition.generic.end
      embed_scope: meta.embedded.cpp.xml source.cpp.embedded.xml
      embed: Packages/C++/C++.sublime-syntax
      escape: '^@($| )'

  cpp-code: # How do I get that into C++ code?
    - meta_scope: comment.block.documentation
    - match: '(<<)(.*)(?=>>)'
      captures:
        1: punctuation.definition.generic.begin
        2: variable.other
        3: punctuation.definition.generic.end

sample file:

Hi there! This is a cool program. Very cool.

<<example.cpp*>>=
<<includes>>
auto main(int argc, char *argv[]) -> int {
  <<the code>>
  return 0;
}

@
Here we document stuff

<<includes>>=
#include <stdio.h>

@ Here we document some other stuff

<<the code>>=
// Will the indentation work?
printf("Hello world!\n");
for (int i = 0; i < 10; i += 1) {
  <<hey this is cool>>
}
<<hey this is cool>>=
printf("wow");
printf("baby");
0 Likes

#15

An extended C++ would be the most advanced solution without loosing any advantages of embed.

This is the way how ShellScript is included into Makefile syntax with support to highlight Makefile’s variables within shell scripts.

This is also how Java Server Pages or Ruby on Rails is prepared to work.

see: https://github.com/sublimehq/Packages/pull/2654/files

An inherited CSS/JS syntax is created to add Java interpolation via prototype. This strategy keeps - meta_include_prototype: false working, which allows the inherited syntax to exclude your injected code from contexts, they might cause harm in.

The downside is your code might break at any time, if ST’s default syntaxes change - depending on how many contexts you modify.


The old way to solve the issue was to push scope:source.c++ using the with_prototype directive to inject your code and required escape patterns.

The downsides are meta_include_prototype: false not beeing respected at all. Injecting code in any context of the pushed syntax may cause unwanted results or at least cause the context limit to be exceeded depending on syntax’s complexity.

3 Likes

#16

Hi again,

In addition to the Python/CSS embedded syntax highlighting, can we add JavaScript highlighting as well?

Thanks very much in advance!

0 Likes

#17

Of course, you can. Already provided examples should be sufficent enough to realize how to do so.

0 Likes