python - Trouble using Vim's syn-include and syn-region to embed syntax highlighting

Question

I am trying to get Vim to syntax highlight any file that ends with extension .Rtex in the following way:

All top level text is highlighted as TeX
Exception: any text enclosed in \begin{python}...\end{python} is highlighted as Python

I am able to achieve each of these criteria individually, but unable to achieve both simultaneously. I think that somehow the TeX highlighting overrides my Python-highlighted regions, or prevents them from taking effect, and I am stuck trying to figure out how.

First step: edit .vimrc to give files with extension .Rtex the filetype rtex:

  au BufRead *.Rtex setf rtex

Second step: create ~/.vim/syntax/rtex.vim. It is the contents of this file that will determine how to highlight .Rtex files.

Third step: enable general top-level TeX highlighting, by making rtex.vim look like this:

runtime! syntax/tex.vim

If I now open a .Rtex file, the entire file is highlighted as TeX, including any text within \begin{python}...\end{python}, as expected.

Fourth step: follow the instructions in Vim's :help syn-include to include python highlighting and apply it to all regions delimited by \begin{python} and \end{python}. My rtex.vim file now looks like this:

runtime! syntax/tex.vim
unlet! b:current_syntax
syntax include @Python syntax/python.vim
syntax region pythonCode start="\\begin{python}" end="\\end{python}" contains=@Python

The unlet! b:current_syntax command is meant to force the python.vim syntax file to execute even though an existing syntax (TeX) is already active.

Problem: If I now open a .Rtex file, the entire file is still highlighted only as TeX. The \begin{python}...\end{python} region seems to have no effect.

Experiment: If I remove or comment out the runtime! command, I do get python highlighting, within the \begin{python}...\end{python} regions, exactly as desired, but of course no TeX highlighting in the remainder of the document. I therefore conclude that the TeX highlighting is somehow responsible for preventing the python regions from taking effect.

Can a Master of Vim offer me any suggestions? I am currently stumped. I have looked at several pages and stackoverflow questions that seem relevant, but none of them have so far led to a solution:

Maxy-B Maxy-B · Accepted Answer · 2011-03-03T15:27:04

After some more study of the manual, and much more trial and error, I have finally answered my own question (a simultaneously embarrassing and sublime accomplishment), which I now preserve here for posterity.

Basically, I think the problem is that the python highlighting wouldn't take effect because the pythonCode region was itself contained in a region or highlight group defined by tex.vim, so it wasn't top-level. The solution is to also include (rather than just runtime) tex.vim, giving it a name like @TeX, and then add containedin=@TeX to my python region definition. So syntax/rtex.vim now looks like this:

let b:current_syntax = ''
unlet b:current_syntax
runtime! syntax/tex.vim

let b:current_syntax = ''
unlet b:current_syntax
syntax include @TeX syntax/tex.vim

let b:current_syntax = ''
unlet b:current_syntax
syntax include @Python syntax/python.vim
syntax region pythonCode matchgroup=Snip start="\\begin{python}" end="\\end{python}" containedin=@TeX contains=@Python

hi link Snip SpecialComment
let b:current_syntax = 'rtex'

And this works! I'm not sure if all of those unlet b:current_syntax commands are strictly necessary. I also gave the python region delimiters a matchgroup (Snip) so that they can be highlighted themselves (with the SpecialComment color), rather than left just plain, which is apparently what happens by default, since they are no longer considered part of the TeX.

Now it is a trivial thing to add more highlight regions for different languages (e.g., \begin{Scode}...\end{Scode}), which is great if you're getting into literate programming -- the original motivation for my question.

Edit: here is a screenshot showing how it works with Python and R code embedded in a TeX document: Screenshot: Python and R embedded in TeX

python - Trouble using Vim's syn-include and syn-region to embed syntax highlighting

2 Answers