Geek Answers Handbook

Problem with returning Cyrillic characters from MSSQL through unixODBC and FreeTDS

I am using django-pyodbc as a database on Ubuntu 12.04 LTS and MSSQL 2008 on a remote host. It works well, except for the returning Cyrillic characters. Instead, I see question marks - "?". I started an investigation into what might cause this problem.

As far as I understand, the MSSQL-django chain looks like this:

MSSQL ↔ FreeTDS ↔ unixODBC ↔ pyodbc ↔ django-pyodbc

So, I started with FreeTDS. When I run the query in tsql - it works well, I can see all the characters, including the Cyrillic alphabet.

The next one was isql - as far as I understand, I can check out a couple of FreeTDS ↔ unixODBC. And there I did not receive the correct data. In fact, when I run the query, the isql columns that contain Cyrillic characters are empty or consist of invisible characters. I assume that the problem is in communication between FreeTDS ↔ unixODBC. What can cause this problem? By the way, I also tried iusql - nothing has changed.

MSSQL mapping - Cyrillic_General_CI_AS.

Contents of freetds.conf:

[global] tds version = 4.2 dump file = /tmp/freetds.log debug flags = 0xffff timeout = 10 connect timeout = 10 client charset = UTF-8 text size = 64512 [egServer50] host = symachine.domain.com port = 5000 tds version = 5.0 [egServer70] host = ntmachine.domain.com port = 1433 tds version = 7.0 [rfxdigest] host = mssql-iis-1 port = 1433 tds version = 8.0 client charset = UTF-8

Content odbc.ini:

 [RFX] Description = Rfx digest server Driver = FreeTDS Database = RFXDB Servername = rfxdigest TDS_Version = 8.0

Edit1 08/15/12

In python using pyodbc, I get '?' instead of cyrillic characters - I tried both versions of python: UCS2 and UCS4.

+4
source share
3 answers

Ok, I created this whole chain of modules:

 MSSQL <-> FreeTDS <-> unixODBC <-> pyodbc <-> django-pyodbc

I just added "unicode_results": True in the DATABASES parameters in the django settings:

 DATABASES = { 'default': { 'ENGINE': 'sql_server.pyodbc', # Add 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'. 'NAME': 'name', # Or path to database file if using sqlite3. 'USER': 'user', # Not used with sqlite3. 'PASSWORD': 'pwd', # Not used with sqlite3. 'HOST': 'server-name', # Set to empty string for localhost. Not used with sqlite3. 'PORT': 'port', # Set to empty string for default. Not used with sqlite3. 'OPTIONS': { 'unicode_results':True, 'driver': 'FreeTDS', 'host_is_server': True, 'extra_params': 'TDS_VERSION=8.0' }

But pyobbc and isql are still not working correctly - maybe I missed other unicode specific options. To test how odbc and pyodbc use this unicode_results parameter later. In any case, the site can now display Cyrillic characters.

0
source

You are getting? for non-printable characters

Run the command below to find out which Unicode your Python setup supports: -

 python -c "import sys;print(sys.maxunicode<66000)and'UCS2'or'UCS4'"

Next, you need to install FreeTDS to use the same character set as Python. If FreeTDS does not support the Unicode format that you use in Python, you will need to change both Python and FreeTDS.

To rebuild Python from a source with UCS2 enabled, you need to do something like: -

 $ ./configure --enable-unicode=ucs2 $ make $ sudo make install
+1
source

I struggled with the problem of Cyrillic letters for a week. I find another solution, I just use microsoft mssql drivers for Linux, here is great how to install it on other than supported distributions (Suse, redhat): https://groups.google.com/forum/#!topic/shiny- discuss / AyFthz3UGwg

This driver returns normal utf-8 and everything that works with the package.

0
source

More articles: