Pre-pre-build commands with qmake
With most non-trivial Qt projects that I create, I like to include a pre-pre-build command in the qmake project file. I'll explain why as we go, but first off, let's look at what I mean by "pre-pre-build" (it is not at all a standard term).
The typical build process (as performed by make) looks something like this:
- For the given target, check if any of the target's dependencies have been updated since it was last (re)built.
- If no dependencies have changed, do nothing - we're done ;)
- If one or more dependencies have changed, then:
- Build each dependent target.
- Build this target.
- If appropriate, link this target with its dependencies.
That is, of course, a gross oversimplification... but it will do the purpose of this post.
Now, like most Qt developers, I use qmake to generate the Makefiles that drive the build process. Qmake, being an excellent tool, allows you to customise the generated Makefiles in many, many ways. For example, you can add custom commands to be executed just before or after the link step (1.b.3 above) via the QMAKE_PRE_LINK and QMAKE_POST_LINK variables.
However, qmake does not provide a QMAKE_PRE_BUILD variable for adding pre-build commands, and certainly no QMAKE_PRE_PRE_BUILD variable either. The former, it turns out, is pretty easy to achieve anyway, but the latter (the topic of this post) is a little bit trickier.
So, to explain what I see as a "pre-pre-build" step, let's repeat the steps show above, but with my imaginary pre-build and pre-pre-build steps included:
- Execute pre-pre-build commands.
- For the given target, check if any of the target's dependencies have been updated since it was last (re)built.
- If no dependencies have changed, do nothing - we're done ;)
- If one or more dependencies have changed, then:
- Build each dependent target.
- Execute pre-build commands.
- Build this target.
- If appropriate, link this target with its dependencies.
So you can see we now have two new steps: 1 and 1.b.2, which are pre-pre-build and pre-build respectively.
As mentioned above, it's pretty easy to create pre-build steps - you can, for example, create a new build target, and add it to the existing target's dependency list... but that's not the same as my pre-pre-build concept. The key difference being this: pre-pre-build commands will always be executed at the start of the make call, whereas pre-build commands will only be executed if the target needs to be rebuilt (ie if at least one of the target's dependencies has been updated).
So, why does that distinction matter? Well, to put it simply - it matters if/when the pre-pre-build commands might result in modification / updates to another target's dependencies! That being the case, clearly they need to be executed before target dependencies are evaluated.
Ok, so let's get to an example, which should make things a little bit clearer: Auto-updating Subversion build numbers. It's not uncommon (just search the Qt-interest mailing list) for people to want to automatically update part of a project's version number based on a revision control system's revision number. Some examples here, and here.
The idea is simple - if a project's version number automatically reflects the Subversion (in my case) revision number, then not only do I have an increasing and unique version number, but I can also easily look back a the repository's commit log to see exactly what changed between releases.
Now many people implement such auto-revision version numbers like this suggestion, which results in a pre-build step, rather than a pre-pre-build step. Which means this:
- When the target's dependencies have changed, and thus a (re)build will occur, both pre-build and pre-pre-build commands would result in the same behavior - the target is built, and receives the current build number automatically.
- But, when the target's dependencies have not changed, but the project's revision count has, then the pre-pre-build approach will result in the target being rebuilt (just to update the build number), whereas the pre-build approach will result in the target being left as is (ie with some earlier version number).
Of course, the "correct" behavior is mostly up to personal preference... on the one hand, if the target itself has not changed, then why update it's version number? but the on the other hand, if you have multiple qmake projects in a given application (eg libraries, etc) then you may (as I usually do) want them to all build to the same release version number.
And, of course, this is just one example... there are many other reasons that you may want a "pre-pre-build" command.
Ok, so that's it for the intro / justification... so how do we do it?! Well, since it's not a standard qmake feature, it is slightly "hackish", but it works by modifying the qmake project file as follows:
- Include a custom qmake target using QMAKE_EXTRA_TARGETS.
- Hook that custom target into the very beginning of the build process (this is the tricky / hacky bit).
So, first off, we create a custom qmake target. I'll call the target "svnbuild", since it updates my project's build numbers to match the Subversion revision count, but obviously you can call it something more appropriate for your particular application.
win32:svnbuild.commands = ..\tools\build\updateBuildNumber.bat ..\svnbuild.h
else:svnbuild.commands = ../tools/build/updateBuildNumber.sh ../svnbuild.h
QMAKE_EXTRA_TARGETS += svnbuild
Here you'll see I've got external updateBuildNumber.bat/sh scripts that do the Subversion checking etc - the *.bat file for win32 builds, and the *.sh script for all other (non-win32) platforms.
Now, the (slightly) tricky bit - hooking our custom "svnbuild" target in as the first step in the generated Makefile. Here goes:
svnbuildhook.depends = svnbuild
CONFIG(debug,debug|release):svnbuildhook.target = Makefile.Debug
CONFIG(release,debug|release):svnbuildhook.target = Makefile.Release
QMAKE_EXTRA_TARGETS += svnbuildhook
So how does that work? Well, to understand, (and here comes the hacky bit) let's look at the Makefiles that qmake generates... assuming that we have not added the above hook yet, the first build target for the generated Makefile.Debug and Makefile.Release files is "all" (no surprises there). Now here's the convenient part - the "all" target actually builds "Makefile.Debug" (or "Makefile.Release", depending on the build mode) before building the actual project target. This is done (presumably) so that the generated "Makefile.*" files are automatically updated if/when the primary "Makefile" file is updated. But since the "Makefile.*" targets are always evaluated first (thanks to the default generated "all" rule), we can simply add a custom build hook target (svnbuildhook) as the build rule for the "Makefile.*" targets... this works because we have not defined any commands for that hook target (ie we don't set "svnbuildhook.commands"), and so make sees an entry like this:
Make interprets this line as simply declaring an additional dependency for "Makefile.Debug" (or "Makefile.Release"), since it does not define any build commands. That is to say, if we skipped the svnbuildhook intermediate target, and instead set our svnbuild target directly as the build rule for "Makefile.Debug", then qmake would generate (and thus make would see) something like the following:
..\tools\build\updateBuildNumber.bat ..\svnbuild.h
Which make would treat as the build rule for "Makefile.Debug" instead of simply being an additional dependency declaration.
Also, note that we did not set any dependencies for the svnbuild target - by leaving svnbuild's dependency list empty, make treats svnbuild as a phony target, which is highly desirably for a pre-pre-build target (it ensures that the associated commands will be executed every time, not just when some dependency has changed... see phony targets for more information).
Okay, well that's it. I know my explanation is a bit convoluted, but as long as you know what qmake is, and you know a bit about the Makefile format, then you should be able to copy and paste the code into your own qmake project files, inspect the resulting Makefiles, and then customise to suit your own desired pre-pre-build commands.
Just to recap, the code to add to qmake project files looks like:
win32:svnbuild.commands = ..\tools\build\updateBuildNumber.bat ..\svnbuild.hxx
else:svnbuild.commands = ../tools/build/updateBuildNumber.sh ../svnbuild.hx
QMAKE_EXTRA_TARGETS += svnbuild
# Hook our svnbuild target in between qmake's Makefile update and the actual project target.
svnbuildhook.depends = svnbuild
CONFIG(debug,debug|release):svnbuildhook.target = Makefile.Debug
CONFIG(release,debug|release):svnbuildhook.target = Makefile.Release
QMAKE_EXTRA_TARGETS += svnbuildhook
Good luck!! :)
Technorati Tags:
almost work
Hi, thanks for the hints, this is what i'm looking for. Though, here it loops (using qt-4.6 on linux):
python updatelocalinfo.py > localinformation.cpp
/usr/bin/qmake -unix -o Makefile kernel.pro
python updatelocalinfo.py > localinformation.cpp
/usr/bin/qmake -unix -o Makefile kernel.pro
python updatelocalinfo.py > localinformation.cpp
/usr/bin/qmake -unix -o Makefile kernel.pro
.....
More info?
Can you post the contents of your kernel.pro and updatelocalinfo.py files?
pc.
Repeating...?
Using QT 4.4.3 under WindowsXP.
I've implemented a custom target & hook under both debug & release modes. When building debug, the target runs fine, as it should.
However, when running release, the custom target runs twice.
When looking at the Makefiles, the custom targets are added, once, as they should.
Any ideas?
Not sure...
Not sure why that might be... but then, I've seen lots of weird things like that with qmake and Makefiles in general - all of which end up being my own misunderstandings in the end. They're complex beasts.
Anyway, post your qmake project files and/or generated Makefiles, and I'll have a look :)
pc.
Better test...
I've made a test project for this, and can actually confirm that the extra build target gets performed twice.
Here come the .pro & makefiles:
test.pro:
# Update the resource file
# Create our custom RCStamp target
win32:rcstampD.commands = "cmd /c C:\Docume~1\hall\mydocu~1\dawn\Tools\rcstamp test.rc *.*.*.+ -v"
QMAKE_EXTRA_TARGETS += rcstampD
# Hook our RCStamp target in between qmake's Makefile update and the actual project target
rcstampDhook.depends = rcstampD
rcstampDhook.target = Makefile.Debug
QMAKE_EXTRA_TARGETS += rcstampDhook
TEMPLATE = app
TARGET =
DEPENDPATH += .
INCLUDEPATH += .
DESTDIR = ./
win32:RC_FILE = test.rc
# Input
SOURCES += main.cpp
#############################################################################
# Makefile for building: test
# Generated by qmake (2.01a) (Qt 4.4.3) on: Tue Feb 9 14:43:32 2010
# Project: test.pro
# Template: app
# Command: c:\Qt\4.4.3\bin\qmake.exe -win32 -o Makefile test.pro
#############################################################################
first: debug
install: debug-install
uninstall: debug-uninstall
MAKEFILE = Makefile
QMAKE = c:\Qt\4.4.3\bin\qmake.exe
DEL_FILE = del
CHK_DIR_EXISTS= if not exist
MKDIR = mkdir
COPY = copy /y
COPY_FILE = $(COPY)
COPY_DIR = xcopy /s /q /y /i
INSTALL_FILE = $(COPY_FILE)
INSTALL_PROGRAM = $(COPY_FILE)
INSTALL_DIR = $(COPY_DIR)
DEL_FILE = del
SYMLINK =
DEL_DIR = rmdir
MOVE = move
CHK_DIR_EXISTS= if not exist
MKDIR = mkdir
SUBTARGETS = \
debug \
release
debug: $(MAKEFILE).Debug FORCE
$(MAKE) -f $(MAKEFILE).Debug
debug-make_default: $(MAKEFILE).Debug FORCE
$(MAKE) -f $(MAKEFILE).Debug
debug-make_first: $(MAKEFILE).Debug FORCE
$(MAKE) -f $(MAKEFILE).Debug first
debug-all: $(MAKEFILE).Debug FORCE
$(MAKE) -f $(MAKEFILE).Debug all
debug-clean: $(MAKEFILE).Debug FORCE
$(MAKE) -f $(MAKEFILE).Debug clean
debug-distclean: $(MAKEFILE).Debug FORCE
$(MAKE) -f $(MAKEFILE).Debug distclean
debug-install: $(MAKEFILE).Debug FORCE
$(MAKE) -f $(MAKEFILE).Debug install
debug-uninstall: $(MAKEFILE).Debug FORCE
$(MAKE) -f $(MAKEFILE).Debug uninstall
release: $(MAKEFILE).Release FORCE
$(MAKE) -f $(MAKEFILE).Release
release-make_default: $(MAKEFILE).Release FORCE
$(MAKE) -f $(MAKEFILE).Release
release-make_first: $(MAKEFILE).Release FORCE
$(MAKE) -f $(MAKEFILE).Release first
release-all: $(MAKEFILE).Release FORCE
$(MAKE) -f $(MAKEFILE).Release all
release-clean: $(MAKEFILE).Release FORCE
$(MAKE) -f $(MAKEFILE).Release clean
release-distclean: $(MAKEFILE).Release FORCE
$(MAKE) -f $(MAKEFILE).Release distclean
release-install: $(MAKEFILE).Release FORCE
$(MAKE) -f $(MAKEFILE).Release install
release-uninstall: $(MAKEFILE).Release FORCE
$(MAKE) -f $(MAKEFILE).Release uninstall
Makefile: test.pro ../../../../Qt/4.4.3/mkspecs/default/qmake.conf ../../../../Qt/4.4.3/mkspecs/qconfig.pri \
../../../../Qt/4.4.3/mkspecs/features/qt_functions.prf \
../../../../Qt/4.4.3/mkspecs/features/qt_config.prf \
../../../../Qt/4.4.3/mkspecs/features/exclusive_builds.prf \
../../../../Qt/4.4.3/mkspecs/features/default_pre.prf \
../../../../Qt/4.4.3/mkspecs/features/win32/default_pre.prf \
../../../../Qt/4.4.3/mkspecs/features/debug.prf \
../../../../Qt/4.4.3/mkspecs/features/debug_and_release.prf \
../../../../Qt/4.4.3/mkspecs/features/default_post.prf \
../../../../Qt/4.4.3/mkspecs/features/win32/rtti.prf \
../../../../Qt/4.4.3/mkspecs/features/win32/exceptions.prf \
../../../../Qt/4.4.3/mkspecs/features/win32/stl.prf \
../../../../Qt/4.4.3/mkspecs/features/shared.prf \
../../../../Qt/4.4.3/mkspecs/features/warn_on.prf \
../../../../Qt/4.4.3/mkspecs/features/qt.prf \
../../../../Qt/4.4.3/mkspecs/features/win32/thread.prf \
../../../../Qt/4.4.3/mkspecs/features/moc.prf \
../../../../Qt/4.4.3/mkspecs/features/win32/windows.prf \
../../../../Qt/4.4.3/mkspecs/features/resources.prf \
../../../../Qt/4.4.3/mkspecs/features/uic.prf \
../../../../Qt/4.4.3/mkspecs/features/yacc.prf \
../../../../Qt/4.4.3/mkspecs/features/lex.prf \
c:/Qt/4.4.3/lib/qtmaind.prl
$(QMAKE) -win32 -o Makefile test.pro
..\..\..\..\Qt\4.4.3\mkspecs\qconfig.pri:
..\..\..\..\Qt\4.4.3\mkspecs\features\qt_functions.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\qt_config.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\exclusive_builds.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\default_pre.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\win32\default_pre.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\debug.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\debug_and_release.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\default_post.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\win32\rtti.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\win32\exceptions.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\win32\stl.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\shared.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\warn_on.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\qt.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\win32\thread.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\moc.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\win32\windows.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\resources.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\uic.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\yacc.prf:
..\..\..\..\Qt\4.4.3\mkspecs\features\lex.prf:
c:\Qt\4.4.3\lib\qtmaind.prl:
qmake: qmake_all FORCE
@$(QMAKE) -win32 -o Makefile test.pro
qmake_all: FORCE
make_default: debug-make_default release-make_default FORCE
make_first: debug-make_first release-make_first FORCE
all: debug-all release-all FORCE
clean: debug-clean release-clean FORCE
distclean: debug-distclean release-distclean FORCE
-$(DEL_FILE) Makefile
rcstampD:
cmd /c C:\Docume~1\hall\mydocu~1\dawn\Tools\rcstamp test.rc *.*.*.+ -v
Makefile.Debug: rcstampD
debug-mocclean: $(MAKEFILE).Debug
$(MAKE) -f $(MAKEFILE).Debug mocclean
release-mocclean: $(MAKEFILE).Release
$(MAKE) -f $(MAKEFILE).Release mocclean
mocclean: debug-mocclean release-mocclean
debug-mocables: $(MAKEFILE).Debug
$(MAKE) -f $(MAKEFILE).Debug mocables
release-mocables: $(MAKEFILE).Release
$(MAKE) -f $(MAKEFILE).Release mocables
mocables: debug-mocables release-mocables
FORCE:
$(MAKEFILE).Debug: Makefile
$(MAKEFILE).Release: Makefile
#############################################################################
# Makefile for building: test
# Generated by qmake (2.01a) (Qt 4.4.3) on: Tue Feb 9 14:43:32 2010
# Project: test.pro
# Template: app
#############################################################################
####### Compiler, tools and options
CC = gcc
CXX = g++
DEFINES = -DUNICODE -DQT_LARGEFILE_SUPPORT -DQT_DLL -DQT_GUI_LIB -DQT_CORE_LIB -DQT_THREAD_SUPPORT -DQT_NEEDS_QMAIN
CFLAGS = -g -Wall $(DEFINES)
CXXFLAGS = -g -frtti -fexceptions -mthreads -Wall $(DEFINES)
INCPATH = -I"..\..\..\..\Qt\4.4.3\include\QtCore" -I"..\..\..\..\Qt\4.4.3\include\QtCore" -I"..\..\..\..\Qt\4.4.3\include\QtGui" -I"..\..\..\..\Qt\4.4.3\include\QtGui" -I"..\..\..\..\Qt\4.4.3\include" -I"." -I"c:\Qt\4.4.3\include\ActiveQt" -I"debug" -I"." -I"..\..\..\..\Qt\4.4.3\mkspecs\default"
LINK = g++
LFLAGS = -enable-stdcall-fixup -Wl,-enable-auto-import -Wl,-enable-runtime-pseudo-reloc -mthreads -Wl -Wl,-subsystem,windows
LIBS = -L"c:\Qt\4.4.3\lib" -lmingw32 -lqtmaind debug\test_res.o -lQtGuid4 -lQtCored4
QMAKE = c:\Qt\4.4.3\bin\qmake.exe
IDC = c:\Qt\4.4.3\bin\idc.exe
IDL = midl
ZIP = zip -r -9
DEF_FILE =
RES_FILE = debug\test_res.o
COPY = copy /y
COPY_FILE = $(COPY)
COPY_DIR = xcopy /s /q /y /i
DEL_FILE = del
DEL_DIR = rmdir
MOVE = move
CHK_DIR_EXISTS= if not exist
MKDIR = mkdir
INSTALL_FILE = $(COPY_FILE)
INSTALL_PROGRAM = $(COPY_FILE)
INSTALL_DIR = $(COPY_DIR)
####### Output directory
OBJECTS_DIR = debug
####### Files
SOURCES = main.cpp
OBJECTS = debug/main.o
DIST =
QMAKE_TARGET = test
DESTDIR = #avoid trailing-slash linebreak
TARGET = test.exe
DESTDIR_TARGET = test.exe
####### Implicit rules
.SUFFIXES: .cpp .cc .cxx .c
.cpp.o:
$(CXX) -c $(CXXFLAGS) $(INCPATH) -o $@ $<
.cc.o:
$(CXX) -c $(CXXFLAGS) $(INCPATH) -o $@ $<
.cxx.o:
$(CXX) -c $(CXXFLAGS) $(INCPATH) -o $@ $<
.c.o:
$(CC) -c $(CFLAGS) $(INCPATH) -o $@ $<
####### Build rules
first: all
all: Makefile.Debug $(DESTDIR_TARGET)
$(DESTDIR_TARGET): $(OBJECTS) debug/test_res.o
$(LINK) $(LFLAGS) -o $(DESTDIR_TARGET) $(OBJECTS) $(LIBS)
debug/test_res.o: test.rc
windres -i test.rc -o debug\test_res.o --include-dir=.
qmake: FORCE
@$(QMAKE) -win32 -o Makefile.Debug test.pro
dist:
$(ZIP) test.zip $(SOURCES) $(DIST) test.pro ..\..\..\..\Qt\4.4.3\mkspecs\qconfig.pri ..\..\..\..\Qt\4.4.3\mkspecs\features\qt_functions.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\qt_config.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\exclusive_builds.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\default_pre.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\win32\default_pre.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\debug.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\debug_and_release.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\default_post.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\build_pass.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\win32\rtti.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\win32\exceptions.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\win32\stl.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\shared.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\warn_on.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\qt.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\win32\thread.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\moc.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\win32\windows.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\resources.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\uic.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\yacc.prf ..\..\..\..\Qt\4.4.3\mkspecs\features\lex.prf c:\Qt\4.4.3\lib\qtmaind.prl HEADERS RESOURCES IMAGES SOURCES OBJECTIVE_SOURCES FORMS YACCSOURCES YACCSOURCES LEXSOURCES
clean: compiler_clean
-$(DEL_FILE) debug\main.o
-$(DEL_FILE) debug\test_res.o
distclean: clean
-$(DEL_FILE) $(DESTDIR_TARGET)
-$(DEL_FILE) Makefile.Debug
rcstampD:
cmd /c C:\Docume~1\hall\mydocu~1\dawn\Tools\rcstamp test.rc *.*.*.+ -v
Makefile.Debug: rcstampD
mocclean: compiler_moc_header_clean compiler_moc_source_clean
mocables: compiler_moc_header_make_all compiler_moc_source_make_all
compiler_moc_header_make_all:
compiler_moc_header_clean:
compiler_rcc_make_all:
compiler_rcc_clean:
compiler_image_collection_make_all: qmake_image_collection.cpp
compiler_image_collection_clean:
-$(DEL_FILE) qmake_image_collection.cpp
compiler_moc_source_make_all:
compiler_moc_source_clean:
compiler_uic_make_all:
compiler_uic_clean:
compiler_yacc_decl_make_all:
compiler_yacc_decl_clean:
compiler_yacc_impl_make_all:
compiler_yacc_impl_clean:
compiler_lex_make_all:
compiler_lex_clean:
compiler_clean:
####### Compile
debug/main.o: main.cpp
$(CXX) -c $(CXXFLAGS) $(INCPATH) -o debug\main.o main.cpp
####### Install
install: FORCE
uninstall: FORCE
FORCE:
Another test...
I have also tested this with the latest QT 4.6.1, and it also happens there too. So it doesn't appear to be a bug with 4.4.3; I must have done something wrong!
Fixed!
I reviewed the documentation, and can now reveal how I fixed it. This is the solution which works for me:
# Create our custom target
rcstamp.target = test.rc #this should be the filename you want to modify
win32:rcstamp.commands = "cmd /c C:\Docume~1\hall\mydocu~1\dawn\Tools\rcstamp test.rc *.*.*.+ -v"
rcstamp.depends = rcstampOut
rcstampOut.commands = @echo *************************
rcstampOut.commands += Updating resource file...
rcstampOut.commands += *************************
QMAKE_EXTRA_TARGETS += rcstamp rcstampOut
I hope this helps :)
Post new comment