GCC Compatibility¶

The number one goal of the driver is to ease the adoption of Clang by allowing users to drop Clang into a build system which was designed to call GCC. Although this makes the driver much more complicated than might otherwise be necessary, we decided that being very compatible with the gcc command line interface was worth it in order to allow users to quickly test clang on their projects.

Flexible¶

The driver was designed to be flexible and easily accommodate new uses as we grow the clang and LLVM infrastructure. As one example, the driver can easily support the introduction of tools which have an integrated assembler; something we hope to add to LLVM in the future.

Similarly, most of the driver functionality is kept in a library which can be used to build other tools which want to implement or accept a gcc like interface.

Low Overhead¶

The driver should have as little overhead as possible. In practice, we found that the gcc driver by itself incurred a small but meaningful overhead when compiling many small files. The driver doesn’t do much work compared to a compilation, but we have tried to keep it as efficient as possible by following a few simple principles:

• Avoid memory allocation and string copying when possible.
• Don’t parse arguments more than once.
• Provide a few simple interfaces for efficiently searching arguments.

Simple¶

Finally, the driver was designed to be “as simple as possible”, given the other goals. Notably, trying to be completely compatible with the gcc driver adds a significant amount of complexity. However, the design of the driver attempts to mitigate this complexity by dividing the process into a number of independent stages instead of a single monolithic task.

Internals Introduction¶

In order to satisfy the stated goals, the driver was designed to completely subsume the functionality of the gcc executable; that is, the driver should not need to delegate to gcc to perform subtasks. On Darwin, this implies that the Clang driver also subsumes the gcc driver-driver, which is used to implement support for building universal images (binaries and object files). This also implies that the driver should be able to call the language specific compilers (e.g. cc1) directly, which means that it must have enough information to forward command line arguments to child processes correctly.

Design Overview¶

The diagram below shows the significant components of the driver architecture and how they relate to one another. The orange components represent concrete data structures built by the driver, the green components indicate conceptually distinct stages which manipulate these data structures, and the blue components are important helper classes.

Driver Stages¶

The driver functionality is conceptually divided into five stages:

1. Parse: Option Parsing

   The command line argument strings are decomposed into arguments (Arg instances). The driver expects to understand all available options, although there is some facility for just passing certain classes of options through (like -Wl,).

   Each argument corresponds to exactly one abstract Option definition, which describes how the option is parsed along with some additional metadata. The Arg instances themselves are lightweight and merely contain enough information for clients to determine which option they correspond to and their values (if they have additional parameters).

   For example, a command line like “-Ifoo -I foo” would parse to two Arg instances (a JoinedArg and a SeparateArg instance), but each would refer to the same Option.

   Options are lazily created in order to avoid populating all Option classes when the driver is loaded. Most of the driver code only needs to deal with options by their unique ID (e.g., options::OPT_I),

   Arg instances themselves do not generally store the values of parameters. In many cases, this would simply result in creating unnecessary string copies. Instead, Arg instances are always embedded inside an ArgList structure, which contains the original vector of argument strings. Each Arg itself only needs to contain an index into this vector instead of storing its values directly.

   The clang driver can dump the results of this stage using the -### flag (which must precede any actual command line arguments). For example:

   $ clang -### -Xarch_i386 -fomit-frame-pointer -Wa,-fast -Ifoo -I foo t.c
   Option 0 - Name: "-Xarch_", Values: {"i386", "-fomit-frame-pointer"}
   Option 1 - Name: "-Wa,", Values: {"-fast"}
   Option 2 - Name: "-I", Values: {"foo"}
   Option 3 - Name: "-I", Values: {"foo"}
   Option 4 - Name: "<input>", Values: {"t.c"}
   

   After this stage is complete the command line should be broken down into well defined option objects with their appropriate parameters. Subsequent stages should rarely, if ever, need to do any string processing.

2. Pipeline: Compilation Action Construction

   Once the arguments are parsed, the tree of subprocess jobs needed for the desired compilation sequence are constructed. This involves determining the input files and their types, what work is to be done on them (preprocess, compile, assemble, link, etc.), and constructing a list of Action instances for each task. The result is a list of one or more top-level actions, each of which generally corresponds to a single output (for example, an object or linked executable).

   The majority of Actions correspond to actual tasks, however there are two special Actions. The first is InputAction, which simply serves to adapt an input argument for use as an input to other Actions. The second is BindArchAction, which conceptually alters the architecture to be used for all of its input Actions.

   The clang driver can dump the results of this stage using the -ccc-print-phases flag. For example:

   $ clang -ccc-print-phases -x c t.c -x assembler t.s
   0: input, "t.c", c
   1: preprocessor, {0}, cpp-output
   2: compiler, {1}, assembler
   3: assembler, {2}, object
   4: input, "t.s", assembler
   5: assembler, {4}, object
   6: linker, {3, 5}, image
   

   Here the driver is constructing seven distinct actions, four to compile the “t.c” input into an object file, two to assemble the “t.s” input, and one to link them together.

   A rather different compilation pipeline is shown here; in this example there are two top level actions to compile the input files into two separate object files, where each object file is built using lipo to merge results built for two separate architectures.

   $ clang -ccc-print-phases -c -arch i386 -arch x86_64 t0.c t1.c
   0: input, "t0.c", c
   1: preprocessor, {0}, cpp-output
   2: compiler, {1}, assembler
   3: assembler, {2}, object
   4: bind-arch, "i386", {3}, object
   5: bind-arch, "x86_64", {3}, object
   6: lipo, {4, 5}, object
   7: input, "t1.c", c
   8: preprocessor, {7}, cpp-output
   9: compiler, {8}, assembler
   10: assembler, {9}, object
   11: bind-arch, "i386", {10}, object
   12: bind-arch, "x86_64", {10}, object
   13: lipo, {11, 12}, object
   

   After this stage is complete the compilation process is divided into a simple set of actions which need to be performed to produce intermediate or final outputs (in some cases, like -fsyntax-only, there is no “real” final output). Phases are well known compilation steps, such as “preprocess”, “compile”, “assemble”, “link”, etc.

3. Bind: Tool & Filename Selection

   This stage (in conjunction with the Translate stage) turns the tree of Actions into a list of actual subprocess to run. Conceptually, the driver performs a top down matching to assign Action(s) to Tools. The ToolChain is responsible for selecting the tool to perform a particular action; once selected the driver interacts with the tool to see if it can match additional actions (for example, by having an integrated preprocessor).

   Once Tools have been selected for all actions, the driver determines how the tools should be connected (for example, using an inprocess module, pipes, temporary files, or user provided filenames). If an output file is required, the driver also computes the appropriate file name (the suffix and file location depend on the input types and options such as -save-temps).

   The driver interacts with a ToolChain to perform the Tool bindings. Each ToolChain contains information about all the tools needed for compilation for a particular architecture, platform, and operating system. A single driver invocation may query multiple ToolChains during one compilation in order to interact with tools for separate architectures.

   The results of this stage are not computed directly, but the driver can print the results via the -ccc-print-bindings option. For example:

   $ clang -ccc-print-bindings -arch i386 -arch ppc t0.c
   # "i386-apple-darwin9" - "clang", inputs: ["t0.c"], output: "/tmp/cc-Sn4RKF.s"
   # "i386-apple-darwin9" - "darwin::Assemble", inputs: ["/tmp/cc-Sn4RKF.s"], output: "/tmp/cc-gvSnbS.o"
   # "i386-apple-darwin9" - "darwin::Link", inputs: ["/tmp/cc-gvSnbS.o"], output: "/tmp/cc-jgHQxi.out"
   # "ppc-apple-darwin9" - "gcc::Compile", inputs: ["t0.c"], output: "/tmp/cc-Q0bTox.s"
   # "ppc-apple-darwin9" - "gcc::Assemble", inputs: ["/tmp/cc-Q0bTox.s"], output: "/tmp/cc-WCdicw.o"
   # "ppc-apple-darwin9" - "gcc::Link", inputs: ["/tmp/cc-WCdicw.o"], output: "/tmp/cc-HHBEBh.out"
   # "i386-apple-darwin9" - "darwin::Lipo", inputs: ["/tmp/cc-jgHQxi.out", "/tmp/cc-HHBEBh.out"], output: "a.out"
   

   This shows the tool chain, tool, inputs and outputs which have been bound for this compilation sequence. Here clang is being used to compile t0.c on the i386 architecture and darwin specific versions of the tools are being used to assemble and link the result, but generic gcc versions of the tools are being used on PowerPC.

4. Translate: Tool Specific Argument Translation

   Once a Tool has been selected to perform a particular Action, the Tool must construct concrete Commands which will be executed during compilation. The main work is in translating from the gcc style command line options to whatever options the subprocess expects.

   Some tools, such as the assembler, only interact with a handful of arguments and just determine the path of the executable to call and pass on their input and output arguments. Others, like the compiler or the linker, may translate a large number of arguments in addition.

   The ArgList class provides a number of simple helper methods to assist with translating arguments; for example, to pass on only the last of arguments corresponding to some option, or all arguments for an option.

   The result of this stage is a list of Commands (executable paths and argument strings) to execute.

5. Execute

   Finally, the compilation pipeline is executed. This is mostly straightforward, although there is some interaction with options like -pipe, -pass-exit-codes and -time.

Additional Notes¶

Relation to GCC Driver Concepts¶

For those familiar with the gcc driver, this section provides a brief overview of how things from the gcc driver map to the clang driver.

• Driver Driver

  The driver driver is fully integrated into the clang driver. The driver simply constructs additional Actions to bind the architecture during the Pipeline phase. The tool chain specific argument translation is responsible for handling -Xarch_.

  The one caveat is that this approach requires -Xarch_ not be used to alter the compilation itself (for example, one cannot provide -S as an -Xarch_ argument). The driver attempts to reject such invocations, and overall there isn’t a good reason to abuse -Xarch_ to that end in practice.

  The upside is that the clang driver is more efficient and does little extra work to support universal builds. It also provides better error reporting and UI consistency.

• Specs

  The clang driver has no direct correspondent for “specs”. The majority of the functionality that is embedded in specs is in the Tool specific argument translation routines. The parts of specs which control the compilation pipeline are generally part of the Pipeline stage.

• Toolchains

  The gcc driver has no direct understanding of tool chains. Each gcc binary roughly corresponds to the information which is embedded inside a single ToolChain.

  The clang driver is intended to be portable and support complex compilation environments. All platform and tool chain specific code should be protected behind either abstract or well defined interfaces (such as whether the platform supports use as a driver driver).