Since a young age I've learned that build systems and dependency management range from difficult to mind-bogglingly frustrating. My best experiences have been with Autotools, which might surprise many. However, I had never made anything serious from scratch using Autotools myself, but rather only modified things setup by others. Lately I had some positive experiences building C programs linking libguile
with Autoconf and this prompted me to try something bigger. I setup the build toolchain for a C++ program using Autotools. In this blog post I attempt to document what I learned along the way.
Autotools, Automake and Autoconf
The first question I wanted answered is if I should be using Autotools, Autoconf or even Automake. The answer is that Autotools is not a binary or similar, but rather it's an umbrella term for Autoconf, Automake and everything related. On the other hand, automake
is a binary that converts a Makefile.am
into Makefile.in
.
In my brand new program, I needed to learn very little about Automake. I created a top-level Makefile.am
containing a pointer to the related documentation and an instruction to recurse into the subdirectory src
:
$ cat Makefile.am
SUBDIRS = src
dist_doc_DATA = README.md
The file in src/Makefile.am
contains information of the sources to compile into objects and the resulting binaries:
$ cat src/Makefile.am
bin_PROGRAMS = test_algorithm
test_algorithm_SOURCES = \
algorithm.cpp \
exception.cpp \
job.cpp \
warning.cpp \
test_algorithm.cpp
You can run automake
as a standalone program. However, I found it far more convenient to call it using autoreconf
. autoreconf
is a program that runs automake
and autoconf
(and more) conveniently for you. Typically, I run autoreconf
with the following switches: -i
to generate missing auxiliary files, -v
for verbose output and -f
to force regeneration of all configuration files.
$ autoreconf -vif
But now I'm getting ahead of myself. Before we can run autoreconf
, we need to write the input for autoconf
, namely, configure.ac
.
Introduction to Autoconf
The role of Autoconf is to generate the configure
to ship with a software distribution. The input file configure.ac
is formatted in the macro language M4.
Now, this might be a bit confusing, but configure
(and hence autoconf
) is not meant to work with software dependency management. Instead, the main design principle of Autoconf is to work with features and capabilities of the installed system where the distributed software is compiled.
config.h
Although I didn't use it myself, I need to mention config.h
. (for what it is worth, I should probably try it out and update this part of the blog post later)
As a result of running configure
, a header file config.h
is typically generated. This header file contains detected capabilities of the system we are running the compilation on. For example, if we look for pthread
, config.h
will contain suitable #define
's to indicate if pthread
is found and linkable or not.
As stated previously, I did not utilize config.h
and therefore no such file is generated in my case.
Structure of configure.ac
The first statement of configure.ac
must be AC_INIT
and the last must be AC_OUTPUT
. The rest of the statements can be placed anywhere, except for when they depend on each other. This dynamic seems to have created a de facto standard structure for configure.ac
files. Even the Autoconf manual has a section Standard configure.ac
Layout. However, I find it quite confusing and instead I follow a mental picture, which is as follows:
- Initialization
- Compiler settings
- Configuration Files
- Dependencies
- Output
As previously stated, AC_INIT
is the first statement. It accepts parameters such as program name, program version, maintainer e-mail address etc. In my case it is:
AC_INIT([algorithm], [1.0])
As part of the initialization phase, I also put a statement to initialize automake
with AM_INIT_AUTOMAKE
. I found this in the automake
manual. Note that statements starting with AC_
are for Autoconf and statements starting with AM_
are macros for Automake. However, AM_INIT_AUTOMAKE
is the only statement for Automake in my case.
Now, this is really important: We give some parameters that look just like compiler flags, but they are for Automake when it generates Makefile.in
etc. I used -Wall
, -Werror
and foreign
. I needed foreign
, because otherwise Automake requires all the files in a standard GNU package, i.e. NEWS
, README
, AUTHORS
and ChangeLog
. My AM_INIT_AUTOMAKE
is as follows:
AM_INIT_AUTOMAKE([-Wall -Werror foreign])
Compiler settings
Next we need to select a compiler. When compiling a standard C program, we use AC_PROG_CC
. In my case, I'm compiling C++ so I use AC_PROG_CXX
instead. Note that the macro AC_PROG_CPP
is related to the C preprocessor and has nothing to do with C++ (for the C++ preprocessor command, use the macro AC_PROG_CXXCPP
).
The only parameter that AC_PROG_CC
and AC_PROG_CXX
accept are a compiler search list, but no compiler flags. Instead, if you want to specify compiler flags, you need to set CFLAGS
or CXXFLAGS
before calling the macro. As per the Autoconf manual, by default AC_PROG_{CC,CXX}
sets {C,CXX}FLAGS}
to -g -O2
(or -g
if the detected compiler is not a GNU compiler). But if {C,CXX}FLAGS
was already set, it's kept unchanged.
In my case, I call the macro AC_PROG_CXX
with the following construct:
: ${CXXFLAGS="-g -O2 -Wall -Werror"}
AC_PROG_CXX
Furthermore, as my source code uses C++17 features, I use an extra macro for that. But it's a little special and I will come back to it later in the section Autoconf Archive.
Configuration files
As explained earlier, Automake compiles an Makefile.in
from Makefile.am
. However, this Makefile.in
is not yet a Makefile
. To specify which makefiles need to be compiled, we use the macro AC_CONFIG_FILES
. In my case:
AC_CONFIG_FILES([
Makefile
src/Makefile
])
Note that we will still distribute the Makefile.in
and the Makefile
will be generated only on site. The instructions to build the file are contained in config.status
while the configure
will use.
Furthermore, the utility of AC_CONFIG_FILES
is not retricted to makefiles, but other files can be specified, too. However, I had no such use.
Dependencies
As explained earlier, Autoconf doesn't work with software dependency managers in the conventional sense. I'll call this section Dependencies anyhow, as that's how I think about it. There are various things you can do here, and in my case I was happy I needed 3 different ways to search for libraries and headers, as that really made me learn.
Dependencies: AC_CHECK_LIB
The most typical thing you would do is AC_CHECK_LIB
. To AC_CHECK_LIB
you give the library name, a symbol in the library, and, optionally, a code segment to execute on success and a code segment to execute on failure. For example, in my scenario I ended up doing
AC_CHECK_LIB([pthread], [pthread_create], [], [AC_MSG_ERROR([pthread is not installed.])])
My first surprise was that a symbol from the library is needed. In fact, the macro will expand into a C program that will be compiled. The source of the C program will contain the symbol we indicated, i.e. pthread_create
and the program will be linked against the library, i.e. with the flag -lpthread
. If compilation succeeds, the third parameter will be executed, but since its []
the default action will be taken. Note that to disable the default action you can do [ ]
. If compilation fails, we run the fourth parameter which is a macro to signal an error and halt operation.
The default action is to add the specified library to LIBS
, i.e. in my case LIBS="-lpthread $LIBS" (yes, I double-checked from the generated
configure). Furthermore, the define
HAVE_LIBlibrarywill be set. In my case, it's
#define HAVE_LIBPTHREAD 1`.
I think there's at least two takeaways here. First of all, we don't match libraries with any kind of versioning. Instead, we look for a symbol. I would assume this can cause headache with missing or deprecated features. Furthermore, the macro AC_CHECK_LIB
uses always C linking. This means that C++ libraries cannot be added using this construct, unless the library also has some symbols using C linking (i.e. extern "C"
).
Dependencies: Linking C++ libraries
Since we can't use AC_CHECK_LIB
for linking C++ code, what other options do we have? There's a macro AX_CXX_CHECK_LIB
, but that is not distributed as part of Autoconf and not even the Autoconf Archive, so I have decided against using it.
Instead, I found a blog post that suggests writing your own check. The basic idea is to use the macro AC_LINK_IFELSE
(this is the same macro as AC_CHECK_LIB
uses). To AC_LINK_IFELSE
you give a source to compile and a segment to run on success and a segment to run on failure. In my case:
AC_LINK_IFELSE(
<source here>,
[HAVE_<library>=1],
[AC_MSG_ERROR([<library> is not installed.])])
To generate the source code that is used to verify linking, the blog post suggests using the macro AC_LANG_PROGRAM
. The macro takes to parameters, a prologue and a body. In my case:
AC_LANG_PROGRAM([#include <gtest/gtest.h>], [EXPECT_EQ(1,1)])
We are still missing two pieces here. First, even though we set AC_PROG_CXX
, we have never set the language to use when running compilation tests. This is done using the macro AC_LANG
. By setting AC_LANG(C++)
, AC_LINK_IFELSE
will use a C++ compiler (and linker) rather than the default choice of a C compiler.
Furthermore, on success, AC_CHECK_LIB
will add the specified library to the variable LIBS
. Therefore, we have to do it manually. If we would want to be really fancy, we could unset the library from LIBS
, if the test fails. Since we exit on failure, we don't unset anything.
Here's the whole block of code:
AC_LANG(C++)
LIBS="-lgtest $LIBS"
AC_LINK_IFELSE(
[AC_LANG_PROGRAM([#include <gtest/gtest.h>], [EXPECT_EQ(1,1)])],
[HAVE_GTEST=1],
[AC_MSG_ERROR([gtest is not installed.])])
Dependencies: Headers
Some dependencies are merely headers. To check the availability of a header, there's the macro AC_CHECK_HEADER
. This macro takes the header as it would be used in a #include
preprocessor directive. Furthermore, you can specify an action to take on success and on failure, respectively. In my case, I'm using the header-only library nlohmann-json
and only headers from gmock
, and thus I'm using the following two statements:
AC_CHECK_HEADER([nlohmann/json.hpp], [], [AC_MSG_ERROR([nlohmann-json is not installed.])])
AC_CHECK_HEADER([gmock/gmock.h], [], [AC_MSG_ERROR([gmock is not installed.])])
Dependencies: pkg-config
The idea behind pkg-config
seems simple enough: it's a program that can be used to query for installed software and their versions, respectively. pkg-config
requires software to be distributed with package metadata contained in a .pc
file. The metadata file will contain various information of the package, such as CFLAGS
and LIBS
. Sounds like a replacement for AC_CHECK_LIB
, right?
The Autoconf Archives provides the macro AX_PKG_CHECK_MODULES
. However, on various online forums and blogs, I found warnings about how Autoconf and pkg-config
have a fundamentally different approach. Therefore, I decided against using pkg-config
in my setup.
As I understand it, one problem is that merely the success or failure of running pkg-config
does not adequately indicate if linking the package works. For example, if pkg-config
is not installed, the test may fail, even though the dependency we are looking for exists and is linkable. Likewise, there can be a discrepancy between how pkg-config
and our compiler/linker of choice works. Lastly, the information in the .pc
file can be wrong. The last point is probably relevant when chrooting or similar.
All of configure.ac
In the previous sections, I've explained all of my configure.ac
, with the exception of AC_OUTPUT
. There's not much to it: we just have to call it in the end of configure.ac
, as it will generate the files needed for Automake.
My whole configure.ac
is as follows (comments included):
$ cat configure.ac
AC_INIT([algorithm], [1.0])
AM_INIT_AUTOMAKE([-Wall -Werror foreign])
: ${CXXFLAGS="-g -O2 -Wall -Werror"}
AC_PROG_CXX
# check that compiler supports c++17 and set -std=c++17
AX_CXX_COMPILE_STDCXX_17([noext], [mandatory])
AC_CONFIG_FILES([
Makefile
src/Makefile
])
AC_CHECK_LIB([pthread], [pthread_create], [], [AC_MSG_ERROR([pthread is not installed.])])
# AC_CHECK_LIB doesnt work well with C++ linking. Instead, we use a test program to see if linking of gtest works.
AC_LANG(C++)
LIBS="-lgtest $LIBS"
AC_LINK_IFELSE(
[AC_LANG_PROGRAM([#include <gtest/gtest.h>], [EXPECT_EQ(1,1)])],
[HAVE_GTEST=1],
[AC_MSG_ERROR([gtest is not installed.])])
# nlohmann-json is a header only library.
AC_CHECK_HEADER([nlohmann/json.hpp], [], [AC_MSG_ERROR([nlohmann-json is not installed.])])
# we need gmock headers even though we don't link it
AC_CHECK_HEADER([gmock/gmock.h], [], [AC_MSG_ERROR([gmock is not installed.])])
AC_OUTPUT
Recap on Usage
Alright, so you've created all the needed source files and now you wonder what commands to use? First, we do
autoreconf
to generate configure
, Makefile.in
and many more files:
$ autoreconf -vif
Note: This step is only done by package maintainers.
To the end-user, we ship all of configure
, Makefile.in
's etc. The first command the end-user does is to run the
configure
script:
$ ./configure
If we, as the package maintainers, did everything correctly, running configure
at the end-user will either fail, if
the program (or library) would never be able to compile or run properly. Likewise, assuming we did everything correctly,
if configure
succeeds, then also compiling and running the program (or library) will work.
To build the source:
$ make
Note that if we did everything correctly, overriding compiler flags with CXXFLAGS
(or CFLAGS
if C program), linkable
libraries with LIBS
etc will work properly.
The Autoconf Archives
The Autoconf Archives is a separate software distribution from Autoconf itself. It contains a multitude of macros that can be used when generating the configure
. Note that it's not an issue to depend on the Autoconf Archives, as we don't distribute the configure.ac
, but instead we distribute configure
. Hence, the end-user needs neither Autoconf nor the Autoconf Archives.
Conclusion
In this post I went through what I learned as I setup a C++ project which is built with Autoconf (and Automake). Autoconf is quite different from CMake and other build systems. For this reason it's probably off-putting for newcomers as many things seem unnecessarily complicated. However, in my experience, well-written configure
files tend to work far better on various systems compared to many other solutions. I doubt I've reached that level of understanding, but everyone needs to start somewhere :-)
While it was definitely frustrating to figure out many things about Autoconf, I'm happy I did it. I feel I understand the world of package management and software distribution a tad better.
Top comments (0)