Sublime Forum

Mis-parse language-formatting bundle

#1

Bug report: Sublime mis-parses a language-formatting bundle.

This report is about Sublime build 4107 on a Mac, to which has been added the TextMate PostScript bundle, as assisted in this forum (PostScript language support?, 28th Dec 2014). The same error is present in Sublime build 4107 on a PC.

Start with the small example file, www.jdawiseman.com/papers/bugs/20210618_postscript_dict_formatting.ps.

%!PS

% http://forum.sublimetext.com/t/mis-parse-language-formatting-bundle/59051

/TestForm1  % as might be invoked with execform
<<
	/FormType 1
	/BBox [ 0 0 72 72 ]
	/Matrix matrix identmatrix
	/PaintProc
	{
		% stuff
	}
>>

% Moving the comment repairs the formating
/TestForm2
<<
	/FormType 1
	/BBox [ 0 0 72 72 ]
	/Matrix matrix identmatrix
	/PaintProc
	{
		% stuff
	}
>>

/TestForm3  % as might be invoked with execform
	<<  % Start line with space or tab: repairs
		/FormType 1
		/BBox [ 0 0 72 72 ]
		/Matrix matrix identmatrix
		/PaintProc
		{
			% stuff
		}
	>>

TextMate 2.0.19 seems happy:

But Sublime build 4107 has mess:

Observe the red blocks, the green text, and mis-colouration of the dictionary boundaries = << and >>.

The problem seems to require both:

  • previous line ends in a comment = % ...;
  • line begins with a dictionary-start token = <<.

The problem disappears if the previous line doesn’t end in a comment; the problem disappears if the << is preceded by a tab or a space.

Is the error in the language bundle, or in Sublime? It could be that there is an error in the language bundle, and another error in TextMate, these two errors precisely cancelling. But one error, Sublime only, would suffice to explain. Hence Ockham’s Razor points to Sublime.

FWLIW, the problem is the same as the “Weird colouring bug” that was posted on 29th Dec 2014 (image). It can be seen in the wild: in the 12th June 2021 version of placemat.ps from line 3869; in subsequent versions perhaps search near there for “\n<<”.

There is no obvious reason to believe that my setup is quirky. Nonetheless, please could somebody else test the example PostScript file in their Sublime and TextMate, and confirm or deny the behaviour described in this post? Thank you.

0 Likes

#2

Are there any errors in the console? It may be that said syntax file is using newer text mate features that we don’t support.

0 Likes

#3

No error shown in the console with https://raw.githubusercontent.com/shpakovski/syntaxmate.tmbundle/a609c5cb7184638b15ea45cafbcf32a9f5222215/Syntaxes/Postscript.tmLanguage


Is it possible that ST doesn’t support \G in its regex engine?

0 Likes

#4

ST seems not supporting \G in its regex engine hence then I remove it and change a push into set.

Save the following file as Postscript.sublime-syntax:

%YAML 1.2
---
# http://www.sublimetext.com/docs/syntax.html
name: Postscript
version: 2
file_extensions:
  - ps
  - eps
first_line_match: ^%!PS
scope: source.postscript
contexts:
  main:
    - match: \(
      captures:
        0: punctuation.definition.string.begin.postscript
      push:
        - meta_scope: string.other.postscript
        - match: \)
          captures:
            0: punctuation.definition.string.end.postscript
          pop: 1
        - include: string_content
    - match: ^(%%(BeginBinary:|BeginCustomColor:|BeginData:|BeginDefaults|BeginDocument:|BeginEmulation:|BeginExitServer:|BeginFeature:|BeginFile:|BeginFont:|BeginObject:|BeginPageSetup:|BeginPaperSize:|BeginPreview:|BeginProcSet|BeginProcessColor:|BeginProlog|BeginResource:|BeginSetup|BoundingBox:|CMYKCustomColor:|ChangeFont:|Copyright:|CreationDate:|Creator:|DocumentCustomColors:|DocumentData:|DocumentFonts:|DocumentMedia:|DocumentNeededFiles:|DocumentNeededFonts:|DocumentNeededProcSets:|DocumentNeededResources:|DocumentPaperColors:|DocumentPaperForms:|DocumentPaperSizes:|DocumentPaperWeights:|DocumentPrinterRequired:|DocumentProcSets:|DocumentProcessColors:|DocumentSuppliedFiles:|DocumentSuppliedFonts:|DocumentSuppliedProcSets:|DocumentSuppliedResources:|EOF|Emulation:|EndBinary:|EndComments|EndCustomColor:|EndData:|EndDefaults|EndDocument:|EndEmulation:|EndExitServer:|EndFeature:|EndFile:|EndFont:|EndObject:|EndPageSetup:|EndPaperSize:|EndPreview:|EndProcSet|EndProcessColor:|EndProlog|EndResource:|EndSetup|ExecuteFile:|Extensions:|Feature:|For:|IncludeDocument:|IncludeFeature:|IncludeFile:|IncludeFont:|IncludeProcSet:|IncludeResource:|LanguageLevel:|OperatorIntervention:|OperatorMessage:|Orientation:|Page:|PageBoundingBox:|PageCustomColors|PageCustomColors:|PageFiles:|PageFonts:|PageMedia:|PageOrder:|PageOrientation:|PageProcessColors|PageProcessColors:|PageRequirements:|PageResources:|PageTrailer|Pages:|PaperColor:|PaperForm:|PaperSize:|PaperWeight:|ProofMode:|RGBCustomColor:|Requirements:|Routing:|Title:|Trailer|VMlocation:|VMusage:|Version|Version:|\+|\?BeginFeatureQuery:|\?BeginFileQuery:|\?BeginFontListQuery:|\?BeginFontQuery:|\?BeginPrinterQuery:|\?BeginProcSetQuery:|\?BeginQuery:|\?BeginResourceListQuery:|\?BeginResourceQuery:|\?BeginVMStatus:|\?EndFeatureQuery:|\?EndFileQuery:|\?EndFontListQuery:|\?EndFontQuery:|\?EndPrinterQuery:|\?EndProcSetQuery:|\?EndQuery:|\?EndResourceListQuery:|\?EndResourceQuery:|\?EndVMStatus:))\s*(.*)$\n?
      scope: meta.Document-Structuring-Comment.postscript
      captures:
        1: keyword.other.DSC.postscript
        3: string.unquoted.DSC.postscript
    - match: '(^[ \t]+)?(?=%)'
      captures:
        1: punctuation.whitespace.comment.leading.postscript
      push:
        - match: '%'
          captures:
            0: punctuation.definition.comment.postscript
          set:
            - meta_scope: comment.line.percentage.postscript
            - match: \n
              pop: 1
    - match: \<\<
      captures:
        0: punctuation.definition.dictionary.begin.postscript
      push:
        - meta_scope: meta.dictionary.postscript
        - match: \>\>
          captures:
            0: punctuation.definition.dictionary.end.postscript
          pop: 1
        - include: main
    - match: '\['
      captures:
        0: punctuation.definition.array.begin.postscript
      push:
        - meta_scope: meta.array.postscript
        - match: '\]'
          captures:
            0: punctuation.definition.array.end.postscript
          pop: 1
        - include: main
    - match: '{'
      captures:
        0: punctuation.definition.procedure.begin.postscript
      push:
        - meta_scope: meta.procedure.postscript
        - match: '}'
          captures:
            0: punctuation.definition.procedure.end.postscript
          pop: 1
        - include: main
    - match: \<\~
      captures:
        0: punctuation.definition.string.begin.postscript
      push:
        - meta_scope: string.other.base85.postscript
        - match: \~\>
          captures:
            0: punctuation.definition.string.end.postscript
          pop: 1
        - match: '[!-z\s]+'
        - match: .
          scope: invalid.illegal.base85.char.postscript
    - match: \<
      captures:
        0: punctuation.definition.string.begin.postscript
      push:
        - meta_scope: string.other.hexadecimal.postscript
        - match: \>
          captures:
            0: punctuation.definition.string.end.postscript
          pop: 1
        - match: '[0-9A-Fa-f\s]+'
        - match: .
          scope: invalid.illegal.hexadecimal.char.postscript
    - match: '[0-3]?[0-9]#[0-9a-zA-Z]+'
      comment: well, not really, but short of listing rules for all bases from 2-36 best we can do
      scope: constant.numeric.radix.postscript
    - match: '(\-|\+)?\d+(\.\d*)?([eE](\-|\+)?\d+)?'
      scope: constant.numeric.postscript
    - match: '(\-|\+)?\.\d+([eE](\-|\+)?\d+)?'
      scope: constant.numeric.postscript
    - match: \b(abs|add|aload|anchorsearch|and|arc|arcn|arct|arcto|array|ashow|astore|atan|awidthshow|begin|bind|bitshift|bytesavailable|cachestatus|ceiling|charpath|clear|cleartomark|cleardictstack|clip|clippath|closefile|closepath|colorimage|concat|concatmatrix|condition|configurationerror|copy|copypage|cos|count|countdictstack|countexecstack|counttomark|cshow|currentblackgeneration|currentcacheparams|currentcmykcolor|currentcolor|currentcolorrendering|currentcolorscreen|currentcolorspace|currentcolortransfer|currentcontext|currentdash|currentdevparams|currentdict|currentfile|currentflat|currentfont|currentglobal|currentgray|currentgstate|currenthalftone|currenthalftonephase|currenthsbcolor|currentlinecap|currentlinejoin|currentlinewidth|currentmatrix|currentmiterlimit|currentobjectformat|currentpacking|currentpagedevice|currentpoint|currentrgbcolor|currentscreen|currentshared|currentstrokeadjust|currentsystemparams|currenttransfer|currentundercolorremoval|currentuserparams|curveto|cvi|cvlit|cvn|cvr|cvrs|cvs|cvx|def|defaultmatrix|definefont|defineresource|defineusername|defineuserobject|deletefile|detach|deviceinfo|dict|dictfull|dictstack|dictstackoverflow|dictstackunderflow|div|dtransform|dup|echo|eexec|end|eoclip|eofill|eoviewclip|eq|erasepage|errordict|exch|exec|execform|execstack|execstackoverflow|execuserobject|executeonly|executive|exit|exp|false|file|filenameforall|fileposition|fill|filter|findencoding|findfont|findresource|flattenpath|floor|flush|flushfile|FontDirectory|for|forall|fork|ge|get|getinterval|globaldict|GlobalFontDirectory|glyphshow|grestore|grestoreall|gsave|gstate|gt|handleerror|identmatrix|idiv|idtransform|if|ifelse|image|imagemask|index|ineofill|infill|initclip|initgraphics|initmatrix|initviewclip|instroke|internaldict|interrupt|inueofill|inufill|inustroke|invalidaccess|invalidcontext|invalidexit|invalidfileaccess|invalidfont|invalidid|invalidrestore|invertmatrix|ioerror|ISOLatin1Encoding|itransform|join|kshow|known|languagelevel|le|length|limitcheck|lineto|ln|load|lock|log|loop|lt|makefont|makepattern|mark|matrix|maxlength|mod|monitor|moveto|mul|ne|neg|newpath|noaccess|nocurrentpoint|not|notify|null|nulldevice|or|packedarray|pathbbox|pathforall|pop|print|printobject|product|prompt|pstack|put|putinterval|quit|rand|rangecheck|rcurveto|read|readhexstring|readline|readonly|readstring|realtime|rectclip|rectfill|rectstroke|rectviewclip|renamefile|repeat|resetfile|resourceforall|resourcestatus|restore|reversepath|revision|rlineto|rmoveto|roll|rootfont|rotate|round|rrand|run|save|scale|scalefont|scheck|search|selectfont|serialnumber|setbbox|setblackgeneration|setcachedevice|setcachedevice2|setcachelimit|setcacheparams|setcharwidth|setcmykcolor|setcolor|setcolorrendering|setcolorscreen|setcolorspace|setcolortransfer|setdash|setdevparams|setfileposition|setflat|setfont|setglobal|setgray|setgstate|sethalftone|sethalftonephase|sethsbcolor|setlinecap|setlinejoin|setlinewidth|setmatrix|setmiterlimit|setobjectformat|setoverprint|setpacking|setpagedevice|setpattern|setrgbcolor|setscreen|setshared|setstrokeadjust|setsystemparams|settransfer|setucacheparams|setundercolorremoval|setuserparams|setvmthreshold|shareddict|show|showpage|sin|sqrt|srand|stack|stackoverflow|stackunderflow|StandardEncoding|start|startjob|status|statusdict|stop|stopped|store|string|stringwidth|stroke|strokepath|sub|syntaxerror|systemdict|timeout|transform|translate|true|truncate|type|typecheck|token|uappend|ucache|ucachestatus|ueofill|ufill|undef|undefined|undefinedfilename|undefineresource|undefinedresult|undefinefont|undefineresource|undefinedresource|undefineuserobject|unmatchedmark|unregistered|upath|userdict|UserObjects|usertime|ustroke|ustrokepath|version|viewclip|viewclippath|VMerror|vmreclaim|vmstatus|wait|wcheck|where|widthshow|write|writehexstring|writeobject|writestring|wtranslation|xcheck|xor|xshow|xyshow|yield|yshow)\b
      scope: keyword.operator.postscript
    - match: '//[^\(\)\<\>\[\]\{\}\/\%\s]+'
      scope: variable.other.immediately-evaluated.postscript
    - match: '/[^\(\)\<\>\[\]\{\}\/\%\s]+'
      scope: variable.other.literal.postscript
    - match: '[^\(\)\<\>\[\]\{\}\/\%\s]+'
      comment: stuff like 22@ff will show as number 22 followed by name @ff, but should be name 22@ff!
      scope: variable.other.name.postscript
  string_content:
    - match: '\\[0-7]{1,3}'
      scope: constant.numeric.octal.postscript
    - match: '\\(\\|[nrtbf\(\)]|[0-7]{1,3}|\r?\n)'
      scope: constant.character.escape.postscript
    - match: \\
      scope: invalid.illegal.unknown-escape.postscript.ignored
    - match: \(
      push:
        - match: \)
          pop: 1
        - include: string_content
0 Likes

#5

No errors in console. (Though, from later posts, it seems that this won’t surprise.)

0 Likes

#6

Please, if this small change will repair the problem and not introduce new errors, and is hence a good thing, would you be willing to propose it in github.com/textmate/postscript.tmbundle? So that the problem is fixed for everyone rather than just for me.

(I don’t want to make the proposal as I don’t understand what is being proposed.)

Thank you.

0 Likes

#7

Unfortuantely. I have no experience on .tmlanguage and I have no experience on TextMate.

It should be fixed on ST side if ST claims it support .tmlanguage imo (and that syntax file is at least 10 years old, not a new feature on TextMate side). Or at lease, emit an error on that.

0 Likes

#8

Done, as issue 4493, entitled tmlanguage misparse.

1 Like