Using Unicode in fancyvrbs VerbatimOut

Question

Problem

VerbatimOut from the "fancyvrb" package does not play well with UTF-8 characters.

Minimum working example:

 \documentclass{minimal} \usepackage[utf8]{inputenc} \usepackage[T1]{fontenc} \usepackage{fancyvrb} \begin{document} \begin{VerbatimOut}{\jobname.test} é \end{VerbatimOut} \input{\jobname.test} \end{document} 

Error message

When compiling using pdflatex mini this gives an error

The file ended while scanning using \UTFviii@three@octets .

Another error occurs when a single occurrence of é above is replaced by something else, for example. é */ :

Package input error Error: Unicode char \u8:### not configured for use with LaTeX.

- indicating that in this case, LaTeX succeeds in reading the multibyte UTF-8 character, but does not know what to do with it (i.e. its incorrect character).

In fact, when I open the created .test file manually, it contains the é character, but in Latin-1 encoding !

Proof: when I open the files in a hex editor, I get the following:

• Source file: C3 A9 (corresponds to LATIN SMALL LETTER E WITH ACUTE in UTF-8)
• Written file: E9 (matches é in Latin-1)

Question

How to install VerbatimOut ?

filecontents* (from "filecontents") indicates that it can work. Unfortunately, I do not understand any code, so I can not fix the fancyvrbs code by manually replicating the logic from file resources.

I also cannot use filecontents* instead of VerbatimOut , because the former does not work inside \newenvironment , and the latter does.

(Oh, by the way: vanilla Verbatim instead of VerbatimOut also works as expected. An error occurs when writing a file, and not when reading verbatim input)

Answer 1

Is your ultimate goal to write characters and accents in Verbatim? Because you can do it like this:

 \documentclass{article} \usepackage{fancyvrb} \begin{document} \begin{Verbatim}[commandchars=\\\{\}] \'{e} \~{e} \`{e} \^{e} \end{Verbatim} \end{document} 

The commandchars parameter allows the characters \ { } to work as usual.

Source: http://ctan.mirror.garr.it/mirrors/CTAN/macros/latex/contrib/fancyvrb/fancyvrb.pdf

Answer 2

Is this still unfixed? I'll see it again. What exactly do you want: your package for using VerbatimOut or for it not to interfere with it?

Test

TexLive 2009 Xelatex compiles fine. With pdflatex version

This is pdfTeX, Version 3.1415926-1.40.10 (TeX Live 2009)

I get an error message, which is rather a more useful error message than you received:

 ! Argument of \UTFviii@three@octets has an extra }. \par l.8 é ? i \makeatletter\show\UTFviii@three@octets ! Undefined control sequence. \GenericError ... #4 \errhelp \@err@ ... l.8 é 

If I were to guess, I would say that inputenc with pdftex uses pdftex primitives to do some hairy storage and restoration of symbol tables, and in some table there are rarely errors somewhere.

Possibly related

I saw a message from Vladimir Volovich in the archives of the pdf-tex mailing lists, starting in 2003, which discusses the conflict between inputenc and fancyvrb, and posts a patch to "solve the problem." Who knows, maybe he faced the same problem? It might be worth sending it by email.

Answer 3

XeTeX has much better Unicode support. The following xelatex run creates "é" both in \jobname.test and in the output PDF file.

 \documentclass{minimal} \usepackage{fontspec} \tracingonline=1 \usepackage{fancyvrb} \begin{document} \begin{VerbatimOut}{\jobname.test} é \end{VerbatimOut} \input{\jobname.test} \end{document} 

fontspec downloads Latin Modern fonts that support Unicode. The standard TeX Computer Modern fonts do not have the correct tables to support Unicode.

If you use a character that does not have a glyph in the current font, by default XeTeX writes a blank space to PDF and prints a warning in the log, but not on the terminal. \tracingonline=1 prints a warning to the terminal.

Answer 4

At http://wiki.portal.chalmers.se/agda/pmwiki.php?n=Main.LiterateAgda they suggest using

 \usepackage{ucs} \usepackage[utf8x]{inputenc} 

in the preamble. I have successfully used this to insert unicode into a shorthand environment.

Answer 5

 \documentclass{article} \usepackage{fancyvrb} \usepackage[utf8]{inputenc} \usepackage[T1]{fontenc} \newenvironment{MonVerbatim}{% \count0=128\relax % \loop \catcode\count0=11\relax \advance\count0 by 1\relax \ifnum\count0<256 \repeat \VerbatimOut[commandchars=\\\{\}]{VerbatimText.tex}% }{\endVerbatimOut} \newcommand\test{A command producing accented characters éà} \begin{document} \begin{MonVerbatim} A little bit text in verbatim mode éà_]. \test \end{MonVerbatim} Followed by some accented character éà. \end{document} 

This code works for me with TeXLive 2018 and pdflatex. Yous, you should probably avoid modifying catcode if you are using 16-bit TeX (lualatex or xelatex).

You can use the iftex package to check the tex engine used.