What is the appropriate library for binding Postgres server-side C functions

Question

I am trying to compile some C extensions on a platform with Ubuntu 14.04 for Postgres 9.5.

In my case, I want to write my C code and first compile it into a standalone executable (as you can see from my Makefile below). This is because I also use the NumPy API and write functions that convert Postgres ArrayType into NumPy PyArray objects, and then use some functions of the NumPy array. It’s very difficult to get the right data, to properly release NpyIter objects, etc. Therefore, I definitely need to compile, run, observe errors and test everything before finalizing the details of how the library is built for the last part, where I say CREATE EXTENSION in Postgres .

When compiling, I get several undefined links , for example:

 tmp.c:(.text+0x2d6): undefined reference to `get_typlenbyvalalign' tmp.c:(.text+0x346): undefined reference to `deconstruct_array' tmp.c:(.text+0x41f): undefined reference to `DatumGetFloat8' tmp.c:(.text+0x4ae): undefined reference to `pfree' tmp.c:(.text+0x4ba): undefined reference to `pfree' 

These are server functions from the Postgres C API, but with lots of Googling and lots of synergy pgxs I can’t figure out how to get the name or path in the Postgres library, which I can’t link to.

Almost all requests are referred only to libpq , but these functions are not defined in this client API library, so I'm looking for something else.

For reference, here is the Makefile that I am currently using. Including the library directory from pg_config --libdir should also be incorrect, since it does not cause any changes to the original undefined errors.

 INCLUDEDIRS := -I. INCLUDEDIRS += -I/usr/include/python2.7 INCLUDEDIRS += -I/home/username/anaconda/lib/python2.7/site-packages/numpy/core/include INCLUDEDIRS += -I$(shell pg_config --includedir-server) INCLUDEDIRS += -I$(shell pg_config --includedir) LIBS := -L$(shell pg_config --libdir) LIBS += -lpython2.7 tmp: tmp.c Makefile gcc tmp.c -o tmp $(INCLUDEDIRS) $(LIBS) 

The output of pg_config --libdir :

 user@computer :~/programming$ pg_config --libdir /usr/lib/x86_64-linux-gnu 

I also found libpgcommon in this library directory, and when I add it to the Makefile, some of the undefined links disappear, but not all. They still remain:

 tmp.c:(.text+0x2d6): undefined reference to `get_typlenbyvalalign' tmp.c:(.text+0x346): undefined reference to `deconstruct_array' tmp.c:(.text+0x41f): undefined reference to `DatumGetFloat8' 

So pfree was found by linking to libpgcommon , but nothing else.

Digging even further, inside postgres.h I see where the DatumGetFloat8 macro is DatumGetFloat8 (line 662):

 #ifdef USE_FLOAT8_BYVAL extern float8 DatumGetFloat8(Datum X); #else #define DatumGetFloat8(X) (* ((float8 *) DatumGetPointer(X))) #endif 

So, it should be that Postgres was installed in some way that used the USE_FLOAT8_BYVAL flag. Is this a standard? Do you expect from a simple installation of Postgres using batch repositories for a popular OS such as Ubuntu?

Given this, what other source code or library from which DatumGetFloat8 is extern 'd? For example, a Google search for "postgres DatumGetFloat8" sheds almost nothing shines on it. The best I can find is a message flow from 2011 , indicating (not sure if it is correct):

Missing DatumGetFloat8 reference implies server was built
with float8 pass by value and pljava was built with float8 pass by
Reference.

(the pljava bit doesn't matter to me).

Answer 1

The PostgreSQL extension is a shared library, not a standalone executable, so you should use -shared -fpic -o tmp.so in your Makefile. Do not mess with -lpq or -lpgcommon .

Then gcc will not complain about undefined links - they will be allowed with the postgres executable when the shared library is loaded into PostgreSQL either using the SQL LOAD or when the PostgreSQL C function defined with this library is called.

Of course, you have to use the same -D that is defined as the PostgreSQL server when it was created, so for example, USE_FLOAT8_BYVAL installed or not installed in both. Otherwise, your extension may not load or may be launched in an interesting way.
(As you asked: there is nothing wrong with passing float8 by value, it is actually more efficient if your architecture supports it. DatumGetFloat8 is defined and exported to postgres if USE_FLOAT8_BYVAL defined.)

To make it all simple, it is best to use the PostgreSQL PGXS extension build infrastructure. This will take care of all these issues for you.

All you have to do is create an appropriate Makefile , in your case it can be as simple as

 MODULES = tmp PG_CONFIG = pg_config PGXS := $(shell $(PG_CONFIG) --pgxs) include $(PGXS) 

Then run make and make install and you are ready to go!

Since your problem is verifying and debugging code using the NumPy API, you must modulate the PostgreSQL extension so that you can do it more easily. For example, you can write numpy.c , which contains all the code that accesses this API, and pgxs.c , which contains the PostgreSQL extension code and calls functions in numpy.c .
You can then write custom test code and link it to numpy.o to test your NumPy functions.

Answer 2

If you write the Postgres extension (the type where you say CREATE EXTENSION foo ), you don't need to worry too much about all the INCLUDEDIR tags, etc. It is easier than you think! I wrote a couple of very simple Postgres extensions, despite the fact that I did C only a few times a year, and this is actually quite easy. Your Makefile can be literally 6 lines long, for example:

 MODULES = myextensionname EXTENSION = myextensionname DATA = myextensionname--1.0.sql PG_CONFIG = pg_config PGXS := $(shell $(PG_CONFIG) --pgxs) include $(PGXS) 

You should also have a myextensionname.control file that looks something like this:

 comment = 'Something about your extension' default_version = '1.0' module_pathname = '$libdir/myextensionname' relocatable = true 

And you will need myextensionname.c :

 #include <postgres.h> #include <fmgr.h> #include <catalog/pg_type.h> #include <utils/builtins.h> // whatever other things you need to include here PG_MODULE_MAGIC; // your code here 

Then you write the file myextensionname--1.0.sql , which starts when someone says CREATE EXTENSION , and the one where you create the types / functions / whatever you need (in SQL).

If you want some really simple examples for copying, here are my two projects:

https://github.com/pjungwir/aggs_for_arrays

https://github.com/pjungwir/inetrange

If you need more advanced example extensions, look in the contrib directory for a local Postgres check .

Regarding your question specifically about this:

So, it should be that Postgres was installed in some way that used the USE_FLOAT8_BYVAL flag.

If you use a Makefile like the one above, you don’t need to worry about it, because your extension will be compiled with the same flags, so it will handle DatumGetFloat8 in the same way as your installed Postgres (both function and macro). I think you will find that USE_FLOAT_BYVAL is just the tip of the iceberg, so instead of setting individual flags, you should use include $(PGXS) to make everything match.

EDIT: I just saw your comment that you want to write standalone binary code for testing purposes. I think that I will do this after creating a working extension so that you can become better acquainted with the process. In addition, you can look in my own projects and contrib extensions for examples of how other people handle testing.