Clang/LLVM¶

The 3.9 release of clang includes ThinLTO support. However, ThinLTO is under active development, and new features, improvements and bugfixes are being added for the next release. For the latest ThinLTO support, build a recent version of clang and LLVM.

Linkers¶

ThinLTO is currently supported for the following linkers:

• gold (via the gold-plugin): Similar to monolithic LTO, this requires using a gold linker configured with plugins enabled.
• ld64: Starting with Xcode 8.
• lld: Starting with r284050 (ELF only).

Basic¶

To utilize ThinLTO, simply add the -flto=thin option to compile and link. E.g.

% clang -flto=thin -O2 file1.c file2.c -c
% clang -flto=thin -O2 file1.o file2.o -o a.out

As mentioned earlier, by default the linkers will launch the ThinLTO backend threads in parallel, passing the resulting native object files back to the linker for the final native link. As such, the usage model the same as non-LTO.

With gold, if you see an error during the link of the form:

/usr/bin/ld: error: /path/to/clang/bin/../lib/LLVMgold.so: could not load plugin library: /path/to/clang/bin/../lib/LLVMgold.so: cannot open shared object file: No such file or directory

Then either gold was not configured with plugins enabled, or clang was not built with -DLLVM_BINUTILS_INCDIR set properly. See the instructions for the LLVM gold plugin.

Controlling Backend Parallelism¶

By default, the ThinLTO link step will launch up to std::thread::hardware_concurrency number of threads in parallel. For machines with hyper-threading, this is the total number of virtual cores. For some applications and machine configurations this may be too aggressive, in which case the amount of parallelism can be reduced to N via:

• gold: -Wl,-plugin-opt,jobs=N
• ld64: -Wl,-mllvm,-threads=N
• lld: -Wl,--thinlto-jobs=N

Incremental¶

ThinLTO supports fast incremental builds through the use of a cache, which currently must be enabled through a linker option.

• gold (as of LLVM r279883): -Wl,-plugin-opt,cache-dir=/path/to/cache
• ld64 (support in clang 3.9 and Xcode 8): -Wl,-cache_path_lto,/path/to/cache
• lld (as of LLVM r296702): -Wl,--thinlto-cache-dir=/path/to/cache

Cache Pruning¶

To help keep the size of the cache under control, ThinLTO supports cache pruning. Cache pruning is supported with ld64 and ELF lld, but currently only ELF lld allows you to control the policy with a policy string. The cache policy must be specified with a linker option.

• ELF lld (as of LLVM r298036): -Wl,--thinlto-cache-policy,POLICY

A policy string is a series of key-value pairs separated by : characters. Possible key-value pairs are:

• cache_size=X%: The maximum size for the cache directory is X percent of the available space on the the disk. Set to 100 to indicate no limit, 50 to indicate that the cache size will not be left over half the available disk space. A value over 100 is invalid. A value of 0 disables the percentage size-based pruning. The default is 75%.

• cache_size_bytes=X, cache_size_bytes=Xk, cache_size_bytes=Xm, cache_size_bytes=Xg: Sets the maximum size for the cache directory to X bytes (or KB, MB, GB respectively). A value over the amount of available space on the disk will be reduced to the amount of available space. A value of 0 disables the byte size-based pruning. The default is no byte size-based pruning.

  Note that ThinLTO will apply both size-based pruning policies simultaneously, and changing one does not affect the other. For example, a policy of cache_size_bytes=1g on its own will cause both the 1GB and default 75% policies to be applied unless the default cache_size is overridden.

• prune_after=Xs, prune_after=Xm, prune_after=Xh: Sets the expiration time for cache files to X seconds (or minutes, hours respectively). When a file hasn’t been accessed for prune_after seconds, it is removed from the cache. A value of 0 disables the expiration-based pruning. The default is 1 week.

• prune_interval=Xs, prune_interval=Xm, prune_interval=Xh: Sets the pruning interval to X seconds (or minutes, hours respectively). This is intended to be used to avoid scanning the directory too often. It does not impact the decision of which files to prune. A value of 0 forces the scan to occur. The default is every 20 minutes.

Clang Bootstrap¶

To bootstrap clang/LLVM with ThinLTO, follow these steps:

1. The host compiler must be a version of clang that supports ThinLTO.
2. The host linker must support ThinLTO (and in the case of gold, must be configured with plugins enabled.
3. Use the following additional CMake variables when configuring the bootstrap compiler build:

• -DLLVM_ENABLE_LTO=Thin
• -DLLVM_PARALLEL_LINK_JOBS=1 (since the ThinLTO link invokes parallel backend jobs)
• -DCMAKE_C_COMPILER=/path/to/host/clang
• -DCMAKE_CXX_COMPILER=/path/to/host/clang++
• -DCMAKE_RANLIB=/path/to/host/llvm-ranlib
• -DCMAKE_AR=/path/to/host/llvm-ar

1. To use additional linker arguments for controlling the backend parallelism or enabling incremental builds of the bootstrap compiler, after configuring the build, modify the resulting CMakeCache.txt file in the build directory. Specify any additional linker options after CMAKE_EXE_LINKER_FLAGS:STRING=. Note the configure may fail if linker plugin options are instead specified directly in the previous step.