There are three stages of making your C code run: compilation, linking, and runtime.
gcc is mostly responsible for the compilation, while when linking, it uses a linker program (e.g.,
ld) with some additional flags.
- In compilation,
gccrequires only header (
.h) files to ensure that external function calls have correct parameters matched with the definition.
- Next, at the linking stage, the linker
ldconnects/links all compiled objects and libraries to an executable file by wrapping some appropriate headers after combining the objects. This stage produces the executable file or linkable library. In windows, it is PE (Portable Executables format: EXE, DLL, or OCX), in Linux, it is ELF (Executable and Linkable format: no extension,
- Runtime is when you request the OS to execute the executable file.
There are two types of libraries: static library (in Linux
.a, in Window
.lib) and dynamic library (
.ocx). The static libraries are copied and embedded directly into the executable file. In opposite, the dynamic libraries are not. The dynamic libraries are supposed to exist in the system where the executable file is executed at runtime.
So, where does
gcc look for the header files and the dynamic libraries?
Check the configuration
echo | gcc -x c -E -Wp,-v - >/dev/null
echo | gcc -x c++ -E -Wp,-v - >/dev/null
These are the outputs on my PC.
With ld command
ld's search directories:
ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012
The output from my PC.
In Arch Linux:
ld, use the
LD_DEBUG environment variable. See help with
The most useful value to debug library path searching is
LD_DEBUG=libs ld foo. Sample output on my PC:
With GCC command
gcc wraps some flags when calling
ld. Therefore, the directories list is different. To list them:
gcc -print-search-dirs | sed '/^lib/b 1;d;:1;s,/[^/.][^/]*/\.\./,/,;t 1;s,:[^=]*=,:;,;s,;,; ,g' | tr \; \\012 | tr : \\012
The output from my PC.
In Arch Linux:
gcc's build information with
In Arch Linux:
gcc looks for the libraries' names from left to right and stops finding when it matches the first library with the searching term.
To pass additional flags to the underlining
ld command when invoking
-Wl,<command separated flags>. The following command options have the same effect:
gcc -Wl,flag1,-flag2,flag3,flag4 # same as gcc -Wl,flag1,-flag2 -Wl,flag3,flag4 # same as ld flag1 -flag2 flag3 flag4
- You also can show the library search directories list by adding the verbose flag
gcc -v foo.o bar.o -o foo
There are many tools to analyze ELF files, namely:
ldd (Linux only),
otool (Mac only),
objdump (Linux only),
ldd command (Linux only)
To figure out which libraries are linked with a program and the libraries' full path, use
ldd. For example,
ldd is the most useful command because it recursively looks for required dynamic libraries and outputs their paths in the running system, while other commands only look for direct dependencies.
Sample output on my PC.
linux-vdso.so.1 (0x00007fff08bf3000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fbc85170000) libblas.so.3 => /lib/x86_64-linux-gnu/libblas.so.3 (0x00007fbc85110000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbc84f1f000) /lib64/ld-linux-x86-64.so.2 (0x00007fbc852e8000) libopenblas.so.0 => /lib/x86_64-linux-gnu/libopenblas.so.0 (0x00007fbc82d3a000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fbc82d17000) libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x00007fbc82a77000) libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007fbc82a2b000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fbc82a11000)
LD_DEBUG environment variable can be used to debug
ldd. In fact, setting
LD_DEBUG also outputs the debug information when executing a command, e.g.,
otool command (Mac)
otool -L foo. Sample output:
foo: mylib.so (compatibility version 0.0.0, current version 0.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.100.3)
objdump command (Linux)
objdump -p foo | grep NEEDED. Sample output:
readelf -d foo | grep NEEDED. Sample output:
How to add custom directories as a library search directory
Custom library paths, which are added to the compilation/linking, by the following method have higher priority than the default library search directories. In other words, if a library is found in a customized directory, it will be selected rather than the library in system default.
Method 1: use
CPATH (c or c++),
C_INCLUDE_PATH (c only),
CPLUS_INCLUDE_PATH (c++ only). For example:
CPATH=/home/transang/my_libs_headers gcc -c foo.c -o foo.o.
Method 2: use
gcc when compiling. For example:
gcc -I/home/transang/my_libs_headers -c foo.c -o foo.o.
Method 1: To add custom directories to the library linking search directories list, use
LIBRARY_PATH environment variable. For example:
LIBRARY_PATH=./mylib:/home/transang/my_libs gcc foo.o bar.o -o foo.
Method 2: add the flag
gcc when linking. For example:
gcc -L/home/transang/my_libs -L./mylib foo.o bar.o -o foo.
From GCC man page:
-Ldir Add directory dir to the list of directories to be searched for -l.
Method 3: add the flag
-rpath-link <dir_path> to
ld. For example:
ld -rpath-link /home/transang/my_libs or
add the flag
-rpath <dir_path> to
ld. For example:
ld -rpath /home/transang/my_libs or
gcc -Wl,-rpath,/home/transang/my_libs. Note: this affects the library searching path in runtime
LIBRARY_PATH environment variable's value does not affect the results of
ld --verbose and
gcc -print-search-dirs commands.
Note 2: method 3 and 4 do not work when I tried but the
ld manpage says it works so I leave them for reference.
The linker uses the following search paths to locate required shared libraries.
1. Any directories specified by
2. Any directories specified by
-rpathoptions. The difference between
-rpath-linkis that directories specified by
-rpathoptions are included in the executable and used at runtime, whereas the
-rpath-linkoption is only effective at link time.
3. On an ELF system, if the
rpath-linkoptions were not used, search the contents of the environment variable
4. On SunOS, if the
-rpathoption was not used, search any directories specified using
5. For a native linker, the contents of the environment variable
6. The default directories, normally `/lib' and `/usr/lib'.
For dynamic libraries, regardless of what you have configured for library searching paths. At runtime, the OS uses an independent configuration.
To add customized directories, use
LD_LIBRARY_PATH environment variable when executing the executable file. For e.g.
How to add dynamic libraries when linking
So far, I have introduced a way to figure out the current configuration and modify it to add more directories for library searching.
To link a library, add
-l<lib_name> flag to the
gcc command when linking. If the
lib_name does not start with
gcc will look for a library named
lib<lib_name>.so. Otherwise, the file name
lib_name will be searched.
For example: with
gcc looks for
libfoo.so file. With
gcc looks for
foo.so file (Look at it precisely,
-l: requires specifying library name extension,
lib prefix and extension suffix to the library name).
In some cases, you may want a clean build that isolates from the running machine. You can use the following command:
LDFLAGS= LIBRARY_PATH= LD_LIBRARY_PATH= CPATH= C_INCLUDE_PATH= CPLUS_INCLUDE_PATH= CPPFLAGS= CFLAGS= gcc <command>
Moreover, the system library header searching path is configured in
/etc/ld.so.conf refers to the content of all files in the directory
/etc/ld.so.conf.d. You can look into the directory to control the configuration.
Embedding library paths for runtime at linking
rpath/runpath can be configured at the linking stage to add library paths to the ELF file header which will be used to scan for dynamic libraries at runtime. See dynamic array tags table here: DT_RPATH (value 15) and DT_RUNPATH (value 29). More about rpath at its wiki page.
Compare RPATH and RUNPATH
runpath and rpath are nearly similar with runpath being newer and gradually replacing rpath.
Search order with LD_LIBRARY_PATH
The search order is as follows, from high to low:
DT_RUNPATH is used only for direct dependencies, while DT_RPATH is used for nested dependencies (ref).
Set rpath with ld
ld -rpath <path> or
gcc main.o -o main -l:mylib.so -Wl,-rpath,/home/transang/mylib -L.. Then at runtime, there is no need to specify
LD_LIBRARY_PATH, just use
Note: the correct flag to embed the search path to the ELF file is
gcc use RPATH. To use RUNPATH, use
--enable-new-dtags. For example:
gcc main.o -o main -l:mylib.so -Wl,-rpath,./libs,--enable-new-dtags -L. To explicitly use RPATH, use
Although, RUNPATH is considered newer, setting it as default caused many compilations failed (ref). So that if you want to use
-rpath, it is better to also explicitly specify its usage with
Set runpath with patchelf
patchelf --set-rpath <path>. For example:
gcc main.o -o main -l:mylib.so -L. patchelf --set-rpath . main ./main
patchelf use RUNPATH.
patchelf also clears existing RPATH/RUNPATH headers.
Get configured rpath/runpath
Use one of the following commands
objdump -x foo | grep RPATH
readelf -d foo | grep RPATH
objdump -x foo | grep RUNPATH
readelf -d foo | grep RUNPATH
Relative search path
rpath/runpath supports the following path patterns:
- Absolute paths: starting with
- Paths relative to the current working dir: starting with
- Paths relative to the executable file: the
$ORIGINvariable is expanded at runtime to the absolute path of the directory containing the executable file.
For example, you can use
patchelf --set-rpath '$ORIGIN/libs' main to add the
libs directory next to
main to the library search path.
Security concern with the setuid/setgid flag
For files with the setuid/setgid flag set (via
chmod a+s <file> command),
LD_LIBRARY_PATH is ignored, any rpath/runpath containing
$ORIGIN expansion are also ignored.
Also note that
ldd does not consider the setuid flag checking in its output.