Using BCP to create XML files creates invalid encoding files for non-ASCII characters

Question

Reproduction issue, SQL code:

CREATE TABLE dbo.FooTable(SomeString NVARCHAR(100));
INSERT INTO dbo.FooTable(SomeString)
VALUES('Degree symbol is °');
DECLARE @Code NVARCHAR(4000) = N'BCP "SELECT (SELECT SomeString FROM dbo.FooTable FOR XML PATH(''Foo''), ROOT(''BAR''), TYPE )"  QUERYOUT "F:\Output\File.XML" -c -C RAW -T ';

EXEC xp_cmdshell @Code;

DROP TABLE dbo.FooTable;

It creates a file with the following contents:

<BAR><Foo><SomeString>Degree symbol is °</SomeString></Foo></BAR>

Such files are not recognized as valid XML files using Internet Explorer and Firefox (Chrome gives an error error on line 1 at column 23: Encoding error). If I open them using Notepad and save it as Unicode (small end), it opens and is checked.

Update:

changing the bcpparameters -T -w -r -tseems to make XML valid for most XML users, but not for Internet Explorer.

Answer 1

Try this option -

IF OBJECT_ID('tempdb.dbo.##t') IS NOT NULL
    DROP TABLE ##t

SELECT x = (
    SELECT x
    FROM (
        VALUES (N'Degree symbol is °')
    ) t(x)
    FOR XML PATH('Foo'), ROOT('BAR'), TYPE
)
INTO ##t

DECLARE @sql NVARCHAR(4000) = 'bcp "SELECT * FROM ##t" queryout "D:\sample.xml" -S ' + @@servername + ' -T -w -r -t'
EXEC sys.xp_cmdshell @sql

, ...