So you want to have a non recursive build

Well maybe you don’t want but for some reasons you need one.

I’ve started to spend some time in the Dibbler codebase and lately I also wanted to try ALE with C++. I’ll write more about my journey with Vim and ALE in a later post.

The very first time I opened a file it was not great a lot of lines were red, it’s not a great feeling to see that maybe 200+ lines out of 300 have something that the language server deemed not great, starting with the include.

After a bit of search and also being mislead by the signals from the cc linter from ALE, I realized that I needed to tell the language server where to find the headers. Dibbler is using a configure/make build system so it’s not easy to generate a compile_commands.json file so instead I create a fairly simple .ccls file:

clang
%cpp -std=c++14
%h -x
%h c++-header
-I.
.ccls

The issue was that dibbler was using relative paths like:

#include "Logger.h"

And then the Makefile for the given sub-folder would make sure that the right -I <directory> was added to CFLAGS, it didn’t seems obvious how I could tell ccls to have a per directory config so I went another route: have complete path in the headers that is to say have #include <Misc/Logger.h> instead of #include "Logger.h".

I initially started to change the C++ files manually but quickly realized that because of headers including headers including headers it will be very tedious. But the main issue was not this one, by changing path in include headers I had to also fix the way compiler was called, there was multiple ways to do it but I was already fustrated in the past by the fact that I had to rm some libraries after fixing one source file.

And so I took the route of making the build non recursive.

Fixing the headers

I already mentioned that but doing it manually turned out to be a bad idea, there is around 200 headers and 250 source files in dibbler. So a bit of scripting came to the rescue to do most of the heavy lifting, I don’t have the script anymore but the gist of it was associating the name of the file with the relative path, something like:

  Logger.h: Misc/Logger.h

And then for every file, replace the names with the relative path, this most probably covered 90% of the work and for the remaining 10% well it’s a manual work.

Changing the build to be non-recursive

If you made it so far it’s most probably because you are interested with this part, so initially dibbler was using a recursive make. In an over-simplified version it reads like:

COMMON_SUBDIRS = Port-Linux nettle
common-libs:
       for dir in $(COMMON_SUBDIRS) ; do \
           $(MAKE) -C $$dir ; \
       done

client: common-libs client-libs
      $(MAKE) dibbler-client

So for each folder we cd into it and run make, then we express some dependencies for the client and when the dependencies changes (which is rarely the case because it’s a phony dependency) we then rebuild the real client picking up.

How we move from this situation to a non recursive ?

1 Take the Makefile.am in subfolders and concatenate them in the top level file

So for instance the following is moving to the top level Makefile.am:

libLowLevel_a_SOURCES += Port-linux/daemon.cpp
libLowLevel_a_SOURCES += Port-linux/daemon.h
libLowLevel_a_SOURCES += Port-linux/ethtool-kernel.h
libLowLevel_a_SOURCES += Port-linux/ethtool-local.h
libLowLevel_a_SOURCES += Port-linux/interface.c
libLowLevel_a_SOURCES += Port-linux/interface.h

Repeat and rinse for all the subfolders and you should have a bunch of SOURCES, CPPFLAGS, LDADD variables.

2 Update the main target linking argument

If we take the example of the client one of the line for linking was like this:

dibbler_client_LDADD += -L$(top_builddir)/@PORT_SUBDIR@ -lLowLevel

If we don’t do anything automake/autoconf won’t generate the right dependencies and so libLowLevel won’t be built before the client

We need to tell the new build that we depend on it by adding the .a file for this library to the LDADD variable for this target:

dibbler_client_LDADD += libLowLevel.a

The SOURCES or CPPFLAGS shouldn’t need an adjustment.

Rinse and repeat and now for most of the main target you should have a proper dependency between the target and its libraries.

3 Tell automake/autoconf to generate objects in subfolders

Add subdir-objects to AM_INIT_AUTOMAKE

4 Adjust Makefile.am to deal with issues

The first 3 steps won’t get 100% things right but it should be pretty close, at that moment you should run automake and autoconf and see what is missing.

Twitter  facebook  hackernews