How do you use python3 c api for command line application?

Question

I have been using custom assembly to replace virtualenv for a while, and it is brilliant. It will take longer, but it really works, and it never gets screwed.

Part of this is in a simple python shell that adds some specific folders to the library path, which I found very useful. The code for it is trivial:

#include <stdio.h> #include <n/text/StringUtils.h> #include <Python.h> int main(int argc, char *argv[]) { /* Setup */ Py_SetProgramName(argv[0]); Py_Initialize(); PySys_SetArgv(argc, argv); /* Add local path */ PyObject *sys = PyImport_ImportModule("sys"); PyObject *path = PyObject_GetAttrString(sys, "path"); /* Custom path */ char *cwd = nrealpath(argv[0]); char *libdir = nstrpath(cwd, "python_lib", NULL); PyList_Append(path, PyString_FromString(libdir)); free(cwd); free(libdir); /* Run the 'main' module */ int rtn = Py_Main(argc, argv); // <-- Notice the command line arguments. Py_Finalize(); return rtn; } 

So is switching to python3 good? So that...

I dutifully replaced the call to PyString_FromString () with PyByte_FromString () and tried to recompile, but it causes errors:

 /Users/doug/env/src/main.c:8:21: error: incompatible pointer types passing 'char *' to parameter of type 'wchar_t *' (aka 'int *') [-Werror,-Wincompatible-pointer-types] Py_SetProgramName(argv[0]); ^~~~~~~ /Users/doug/projects/py-sdl2/py3/include/python3.3m/pythonrun.h:25:45: note: passing argument to parameter here PyAPI_FUNC(void) Py_SetProgramName(wchar_t *); ^ /Users/doug/env/src/main.c:10:23: error: incompatible pointer types passing 'char **' to parameter of type 'wchar_t **' (aka 'int **') [-Werror,-Wincompatible-pointer-types] PySys_SetArgv(argc, argv); ^~~~ /Users/doug/projects/py-sdl2/py3/include/python3.3m/sysmodule.h:12:47: note: passing argument to parameter here PyAPI_FUNC(void) PySys_SetArgv(int, wchar_t **); ^ /Users/doug/env/src/main.c:24:27: error: incompatible pointer types passing 'char **' to parameter of type 'wchar_t **' (aka 'int **') [-Werror,-Wincompatible-pointer-types] int rtn = Py_Main(argc, argv); ^~~~ /Users/doug/projects/py-sdl2/py3/include/python3.3m/pythonrun.h:148:45: note: passing argument to parameter 'argv' here PyAPI_FUNC(int) Py_Main(int argc, wchar_t **argv); ^ 3 errors generated. make[2]: *** [CMakeFiles/python.dir/src/main.co] Error 1 make[1]: *** [CMakeFiles/python.dir/all] Error 2 make: *** [all] Error 2 

As you can see from the error, wchar_t is used instead of char *.

How should you use this api?

I see that there are several examples of this, for example: http://svn.python.org/projects/python/tags/r32rc2/Python/frozenmain.c

Really?

Should my 29-line program become a 110-line monster full of #ifdefs?

Do I really not understand, or is that python3 c api really getting so ridiculously difficult to use?

Have I really missed some obvious convenience feature that does this for you in a simple, portable, and cross-platform way?

Answer 1

The official recommended way to convert from char to wchar_t is to use Py_DecodeLocale . Like this:

 wchar_t *program = Py_DecodeLocale(argv[0], NULL); Py_SetProgramName(program); 

Answer 2

There seems to be no easy way to do this.

The closest I came below. I will leave the question open with vague hopes that someone will come and show me a very simple and easy way to do this.

 #include <stdio.h> #include <Python.h> #include <wchar.h> int main(int argc, char *argv[]) { /* These have to be wchar_t */ char *str_program_name = argv[0]; char **str_argv = argv; /* For ever stupid reason, these don't need to be wchar_t * */ char *_sys = "sys"; char *_libdir = "lib"; char *_path = "path"; char *_dot = "."; #if PY_MAJOR_VERSION >= 3 wchar_t **_argv = nstrws_array(argc, str_argv); wchar_t *_program_name = nstrws_convert(str_program_name); #else char **_argv = str_argv; char *_program_name = str_program_name; #endif /* Setup */ Py_SetProgramName(_program_name); Py_Initialize(); /* Add local path */ #if PY_MAJOR_VERSION >= 3 PyObject *sys = PyImport_ImportModule(_sys); PyObject *path = PyObject_GetAttrString(sys, _path); PyList_Append(path, PyBytes_FromString(_dot)); PyList_Append(path, PyBytes_FromString(_libdir)); #else PyObject *sys = PyImport_ImportModule(_sys); PyObject *path = PyObject_GetAttrString(sys, _path); PyList_Append(path, PyString_FromString(_dot)); PyList_Append(path, PyString_FromString(_libdir)); #endif /* Run the 'main' module */ int rtn = Py_Main(argc, _argv); Py_Finalize(); #if PY_MAJOR_VERSION >= 3 nstrws_dispose(argc, _argv); free(_program_name); #endif return rtn; } 

Using:

 /** Unix-like platform char * to wchar_t conversion. */ wchar_t *nstrws_convert(char *raw) { wchar_t *rtn = (wchar_t *) calloc(1, (sizeof(wchar_t) * (strlen(raw) + 1))); setlocale(LC_ALL,"en_US.UTF-8"); // Unless you do this python 3 crashes. mbstowcs(rtn, raw, strlen(raw)); return rtn; } /** Dispose of an array of wchar_t * */ void nstrws_dispose(int count, wchar_t ** values) { for (int i = 0; i < count; i++) { free(values[i]); } free(values); } /** Convert an array of strings to wchar_t * all at once. */ wchar_t **nstrws_array(int argc, char *argv[]) { wchar_t **rtn = (wchar_t **) calloc(argc, sizeof(wchar_t *)); for (int i = 0; i < argc; i++) { rtn[i] = nstrws_convert(argv[i]); } return rtn; } 

and for Windows users, if required:

 #include <windows.h> /** Windows char * to wchar_t conversion. */ wchar_t *nstrws_convert(char *raw) { int size_needed = MultiByteToWideChar(CP_UTF8, 0, raw, -1, NULL, 0); wchar_t *rtn = (wchar_t *) calloc(1, size_needed * sizeof(wchar_t)); MultiByteToWideChar(CP_UTF8, 0, raw, -1, rtn, size_needed); return rtn; } 

Answer 3

This is probably the wrong way to do this, though:

 Py_SetProgramName((wchar_t*)argv[0]); 

This fix stopped my code from complaining, didn't check it to find out how it handles args, but at least compiles it.

Answer 4

I found that this works when converting char * to wchar_t * to the main function:

 wchar_t progname[FILENAME_MAX + 1]; mbstowcs(progname, argv[0], strlen(argv[0]) + 1); Py_SetProgramName(progname); 

If you are using unix:

 #include "sys/param.h"