What is the difference between LD_LIBRARY_PATH and -L at connection time?

Question

I have problems with LD_LIBRARY_PATH on the link (this question has nothing to do with runtime).

The link line looks like this when I run make (this is a Linux system using g ++ version 4.1.x):

 g++ ao bo co -o myapp \ -L/long/path/to/libs/ \ -L/another/long/path/ \ -labc -ldef -lghi 

The -l options refer to shared libraries (such as libabc.so) that exist in the directories specified by the -l options. These directories are also displayed in LD_LIBRARY_PATH . With this configuration, the link failed, and I can start the application.

If I delete directories from LD_LIBRARY_PATH , then I will get one line of error, for example:

 /usr/bin/ld: cannot find -labc 

On the other hand, if I remove directories from the -l options list, then I get a lot of warnings, such as:

 /usr/bin/ld: warning: libabc.so, needed by /long/path/to/libs/libxyz.so, not found (try using -rpath or -rpath-link) 

and then many more errors, for example:

 /long/path/to/libs/libdef.so: undefined reference to `Foo::Bar<Baz>::junk(Fred*)' 

Can someone explain the difference between LD_LIBRARY_PATH and -l ? I would like to understand this material in depth, so the links are very much appreciated!

Also, what do I need to add to the link line to avoid using LD_LIBRARY_PATH ?

EDIT: When directories were missing in -l , the compiler suggested "try using -rpath or -rpath-link". I do not think that I saw these parameters in the make file before. You have? Not sure if this will help solve LD_LIBRARY_PATH problem.

Answer 1

The LD_LIBRARY_PATH settings have the highest priority, so when it is set, the set of directories mentioned in LD_LIBRARY_PATH is executed first even before the standard set of directories. Thus, in your case, the LD_LIBRARY_PATH setting affects the library lookup mentioned in the -l option. Without LD_LIBRARY_PATH some of the dependencies may have been resolved from a standard set of directories.

Although the LD_LIBRARY_PATH setting will help with debugging and trying out the new version of the library, its use in the general installation and deployment of the development environment is considered to be bad.

Also see this HOWTO from the Linux documentation for more information on shared libraries.

Answer 2

There are two answers to this question: part of the answer is to compile-time binding ( gcc -lfoo -L/usr/lib ... which in turn calls ld ) and looking for a run-time linker.

When you compile your program, the compiler checks the syntax, and then the linker ensures that the characters necessary for execution exist (for example, variables / methods / etc), among other things. LD_LIBRARY_PATH , as noted, has a side effect of changing the behavior of gcc / ld , as well as the way that the runtime linker behaves by changing the search path.

When you run your program, the runtime linker actually extracts the shared libraries (on disk or from memory, if possible) and loads into common characters / code / etc. Again, LD_LIBRARY_PATH affects this search path implicitly (sometimes this is not very good, as already mentioned).

The correct fix for this without using LD_LIBRARY_PATH for most Linux systems is to add the path that contains your shared libraries to /etc/ld.so.conf (or on some distributions, create a file in /etc/ld.so.conf.d/ using the path in it) and run ldconfig ( /sbin/ldconfig as root) to update the runtime linker binding cache.

Debian example:

 jewart@dorfl:~$ cat /etc/ld.so.conf.d/usrlocal.conf /usr/local/lib 

Then, when the program executes, the runtime linker will look in these directories for the libraries that your binary was associated with.

If you want to know what libraries the runtime linker knows about, you can use:

 jewart@dorfl:~$ ldconfig -v /usr/lib: libbfd-2.18.0.20080103.so -> libbfd-2.18.0.20080103.so libkdb5.so.4 -> libkdb5.so.4.0 libXext.so.6 -> libXext.so.6.4.0 

And, if you want to know what the binary libraries are associated with, you can use ldd as such, which will tell you which library your runtime linker will be selected in:

 jewart@dorfl:~$ ldd /bin/ls linux-vdso.so.1 => (0x00007fffda1ff000) librt.so.1 => /lib/librt.so.1 (0x00007f5d2149b000) libselinux.so.1 => /lib/libselinux.so.1 (0x00007f5d2127f000) libacl.so.1 => /lib/libacl.so.1 (0x00007f5d21077000) libc.so.6 => /lib/libc.so.6 (0x00007f5d20d23000) 

Answer 3

LD_LIBRARY_PATH designed to search for shared libraries at application startup. This is a side effect that affects your connection, and you should not rely on it.

As often an undesirable side effect, LD_LIBRARY_PATH will also search at the link (ld) stage after the directories specified with -L (also if no -L flag is given).

Why LD_LIBRARY_PATH is bad

Answer 4

If I could guess, I would say that the linker goes back to using LD_LIBRARY_PATH to resolve libraries that your direct links (like libabc.so , libdef.so and libghi.so ) are dynamically linked to, if you look at the libghi.so page for ld it looks like the .so link that was built with -rpath will affect how dynamically linked characters look.

Answer 5

Checking man for g ++, I found that the -lxxxxx parameter looks for libxxxxx.a in the provided -L path, so only the .a file will be downloaded with the link time. At run time, if the library is missing, then only the library will be loaded as a shared object, so .so will be loaded, and then it will look in LD_LIBRARY_PATH . In the executable I'm working on, I see that in some library directory there is a version of libxxxx.a and libxxxx.so , so I think this means that the library can be linked at link time or linked at run time in the form shared object.

If the library exists only as a shared object, this means that the library directory path must be LD_LIBRARY_PATH , which will be found at runtime. If the library exists only as archived as .a, then this means that it needs to be linked when building the executable file, and then -L directorypath and -lxxxxx must be provided in g ++ at compile time.

This is my understanding ... and at least it matches your observations