_dl_runtime_resolve - When are shared objects loaded into memory?

Question

We have a messaging system with high performance requirements. We recently noticed that the first message takes a lot longer than subsequent messages. A bunch of transformation and an increase in messages occur as it passes through our system, most of which are carried out through an external library.

I simply profiled this problem (using callgrind), comparing the "start" of only one message with the "run" of many messages (providing a baseline for comparison).

The main difference that I see is the do_lookup_x function, which takes a huge amount of time. Looking at the various calls to this function, they all seem to be called by a common function: _dl_runtime_resolve. Not sure what this function does, but for me it looks like the first time using different shared libraries, and then loaded into ld memory.

Is this a correct guess? That the binary will not load shared libraries into memory until they are ready for use, so we will see a significant slowdown in the first message, but not in any of the following?

How can we avoid this?

Note. We work with a microsecond scale.

Answer 1

On the ld.so(8) man ENVIRONMENT page :

  LD_BIND_NOW (libc5; glibc since 2.1.1) If set to a non-empty string, causes the dynamic linker to resolve all symbols at program startup instead of deferring function call resolution to the point when they are first referenced. This is useful when using a debug- ger. 

So LD_BIND_NOW=y ./timesensitiveapp .

Answer 2

As an alternative to the Ignacio Vazquez-Abrams runtime suggestion, you can do the same during the link. When you link your shared library, pass the -z now flag to the linker.