Add built-in ccache build support via WITH_CCACHE_BUILD option.

ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not
used to achieve a safe pseudo-incremental build.  This is explained in
more detail upstream [1] [2].  It incurs about a 20%-28% hit to populate the
cache, but with a full cache saves 30-50% in build times.  When combined with
the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the
resulting dependency file, which it does not do when using mkdep(1)/'CC
-E'.  Stats are provided at the end of this message.

This removes the need to modify /etc/make.conf with the CC:= and CXX:=
lines which conflicted with external compiler support [3] (causing the
bootstrap compiler to not be built which lead to obscure failures [4]),
incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid
Clang errors with parenthesis, and did not work with META_MODE.

The option name was picked to match the existing option in ports.  This
feature is available for both in-src and out-of-src builds that use
/usr/share/mk.

Linking, assembly compiles, and pre-processing avoid using ccache since it is
only overhead.  ccache does nothing special in these modes, although there is
no harm in calling it for them.

CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap
compiler to hash the content of the compiler binary to determine if it
should be a cache miss.  For external compilers the 'mtime' option is used
as it is more efficient and likely to be correct.  Future work may optimize the
'content' check using the same checks as whether a bootstrap compiler is needed
to be built.

The CCACHE_CPP2 pessimization is currently default in our devel/ccache
port due to Clang requiring it.  Clang's -Wparentheses-equality,
-Wtautological-compare, and -Wself-assign warnings do not mix well with
compiling already-pre-processed code that may have expanded macros that
trigger the warnings.  GCC has so far not had this issue so it is allowed to
disable the CCACHE_CPP2 default in our port.

Sharing a cache between multiple checkouts, or systems, is explained in
the ccache manual.  Sharing a cache over NFS would likely not be worth
it, but syncing cache directories between systems may be useful for an
organization.  There is also a memcached backend available [5].  Due to using
an object directory outside of the source directory though you will need to
ensure that both are in the same prefix and all users use the same layout.  A
possible working layout is as follows:
  Source: /some/prefix/src1
  Source: /some/prefix/src2
  Source: /some/prefix/src3
  Objdir: /some/prefix/obj
  Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj'
This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace
all absolute paths to be relative.  Using something like this is required due
to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache
adds into its hash for the object without CCACHE_BASEDIR.

distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc.
I have not personally tested this and assume it will not mix well with
using the bootstrap compiler.

The cache from buildworld can be reused in a subdir by first running
'make buildenv' (from r290424).

Note that the cache is currently different depending on whether -j is
used or not due to ccache enabling -fdiagnostics-color automatically if
stderr is a TTY, which bmake only does if not using -j.

The system I used for testing was:
  WITNESS
  Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes
  DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log.
        The arc was fully populated with src tree files and ccache objects.
  RAM: 76GiB
  CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz
       2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16

The WITH_FAST_DEPEND feature was used for comparison here as well to show
the dramatic time savings with a full cache.

buildworld:
  x buildworld-before
  + buildworld-ccache-empty
  * buildworld-ccache-full
  % buildworld-ccache-full-fastdep
  # buildworld-fastdep
  +-------------------------------------------------------------------------------+
  |%            *                               #                                +|
  |%            *                               #                                +|
  |%            *                               #             xxx                +|
  |                                                           |A                  |
  |                                                                              A|
  |             A                                                                 |
  |A                                                                              |
  |                                             A                                 |
  +-------------------------------------------------------------------------------+
      N           Min           Max        Median           Avg        Stddev
  x   3       3744.13       3794.31       3752.25     3763.5633     26.935139
  +   3          4519       4525.04       4520.73       4521.59     3.1104823
  Difference at 95.0% confidence
          758.027 +/- 43.4565
          20.1412% +/- 1.15466%
          (Student's t, pooled s = 19.1726)
  *   3       1823.08        1827.2       1825.62        1825.3     2.0785572
  Difference at 95.0% confidence
          -1938.26 +/- 43.298
          -51.5007% +/- 1.15045%
          (Student's t, pooled s = 19.1026)
  %   3       1266.96       1279.37       1270.47     1272.2667     6.3971113
  Difference at 95.0% confidence
          -2491.3 +/- 44.3704
          -66.1952% +/- 1.17895%
          (Student's t, pooled s = 19.5758)
  #   3       3153.34       3155.16        3154.2     3154.2333    0.91045776
  Difference at 95.0% confidence
          -609.33 +/- 43.1943
          -16.1902% +/- 1.1477%
          (Student's t, pooled s = 19.0569)

buildkernel:
  x buildkernel-before
  + buildkernel-ccache-empty
  * buildkernel-ccache-empty-fastdep
  % buildkernel-ccache-full
  # buildkernel-ccache-full-fastdep
  @ buildkernel-fastdep
  +-------------------------------------------------------------------------------+
  |#                        @   %                  *                              |
  |#                        @   %                  *     x                      + |
  |#                        @   %                  *     xx                     ++|
  |                                                      MA                       |
  |                                                                             MA|
  |                                                A                              |
  |                             A                                                 |
  |A                                                                              |
  |                         A                                                     |
  +-------------------------------------------------------------------------------+
      N           Min           Max        Median           Avg        Stddev
  x   3        571.57        573.94        571.79     572.43333     1.3094401
  +   3        727.97        731.91        728.06     729.31333     2.2492295
  Difference at 95.0% confidence
          156.88 +/- 4.17129
          27.4058% +/- 0.728695%
          (Student's t, pooled s = 1.84034)
  *   3         527.1        528.29        528.08     527.82333    0.63516402
  Difference at 95.0% confidence
          -44.61 +/- 2.33254
          -7.79305% +/- 0.407478%
          (Student's t, pooled s = 1.02909)
  %   3         400.4        401.05        400.62        400.69     0.3306055
  Difference at 95.0% confidence
          -171.743 +/- 2.16453
          -30.0023% +/- 0.378128%
          (Student's t, pooled s = 0.954969)
  #   3        201.94        203.34        202.28        202.52    0.73020545
  Difference at 95.0% confidence
          -369.913 +/- 2.40293
          -64.6212% +/- 0.419774%
          (Student's t, pooled s = 1.06015)
  @   3        369.12        370.57         369.3     369.66333    0.79033748
  Difference at 95.0% confidence
          -202.77 +/- 2.45131
          -35.4225% +/- 0.428227%
          (Student's t, pooled s = 1.0815)

[1] https://ccache.samba.org/performance.html
[2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html
[3] https://reviews.freebsd.org/D3484
[5] https://github.com/jrosdahl/ccache/pull/30

PR:		182944 [4]
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
Relnotes:	yes

diff --git a/Makefile b/Makefile
index 130fdbf6720051b5ce0c8f009ca8f46e989722f6..ce4c0f6963e8c40e1fafa837f7bb479ccc5cf648 100644 (file)
--- a/Makefile
+++ b/Makefile
@@ -100,6 +100,10 @@
 # For more information, see the build(7) manual page.
 #
 
+# This is included so CC is set to ccache for -V, and COMPILER_TYPE/VERSION
+# can be cached for sub-makes.
+.include <bsd.compiler.mk>
+
 # Note: we use this awkward construct to be compatible with FreeBSD's
 # old make used in 10.0 and 9.2 and earlier.
 .if defined(MK_META_MODE) && ${MK_META_MODE} == "yes" && !make(showconfig)

diff --git a/Makefile.inc1 b/Makefile.inc1
index 3f37401b3823c84cb365042a83c32e70122dc4b1..c49bc4ce89caaa6cae2a4fab9702d373a5864a61 100644 (file)
--- a/Makefile.inc1
+++ b/Makefile.inc1
@@ -379,7 +379,7 @@ CROSSENV+=  CC="${XCC} ${XCFLAGS}" CXX="${XCXX} ${XCFLAGS} ${XCXXFLAGS}" \
                RANLIB=${XRANLIB} STRINGS=${XSTRINGS} \
                SIZE="${XSIZE}"
 
-.if ${XCC:M/*}
+.if ${XCC:N${CCACHE_BIN}:M/*}
 .if defined(CROSS_BINUTILS_PREFIX)
 # In the case of xdev-build tools, CROSS_BINUTILS_PREFIX won't be a
 # directory, but the compiler will look in the right place for it's
@@ -456,7 +456,7 @@ LIB32FLAGS= -m32 ${LIB32CPUFLAGS} -DCOMPAT_32BIT \
                -isystem ${LIB32TMP}/usr/include/ \
                -L${LIB32TMP}/usr/lib32 \
                -B${LIB32TMP}/usr/lib32
-.if ${XCC:M/*}
+.if ${XCC:N${CCACHE_BIN}:M/*}
 LIB32FLAGS+=           --sysroot=${WORLDTMP}
 .endif
 
@@ -1599,7 +1599,7 @@ _elftctools=      lib/libelftc \
 
 # If an full path to an external cross compiler is given, don't build
 # a cross compiler.
-.if ${XCC:M/*} == "" && ${MK_CROSS_COMPILER} != "no"
+.if ${XCC:N${CCACHE_BIN}:M/*} == "" && ${MK_CROSS_COMPILER} != "no"
 .if ${MK_CLANG_BOOTSTRAP} != "no"
 _clang=                usr.bin/clang
 _clang_libs=   lib/clang

diff --git a/share/mk/bsd.compiler.mk b/share/mk/bsd.compiler.mk
index 9ad4baee13b781a17d2927eebe614a6e03a7f26d..30da2aff88578877f40fcdcae0ecf8e18bea563e 100644 (file)
--- a/share/mk/bsd.compiler.mk
+++ b/share/mk/bsd.compiler.mk
@@ -25,6 +25,71 @@
 .if !target(__<bsd.compiler.mk>__)
 __<bsd.compiler.mk>__:
 
+.include <bsd.opts.mk>
+
+# Handle ccache after CC is determined, but not if CC/CXX are already
+# overridden with a manual setup.
+.if ${MK_CCACHE_BUILD:Uno} == "yes" && \
+    (${CC:M*ccache/world/*} == "" || ${CXX:M*ccache/world/*} == "")
+# CC is always prepended with the ccache wrapper rather than modifying
+# PATH since it is more clear that ccache is used and avoids wasting time
+# for mkdep/linking/asm builds.
+LOCALBASE?=            /usr/local
+CCACHE_WRAPPER_PATH?=  ${LOCALBASE}/libexec/ccache
+CCACHE_BIN?=           ${LOCALBASE}/bin/ccache
+.if exists(${CCACHE_BIN})
+# Export to ensure sub-makes can filter it out for mkdep/linking and
+# to chain down into kernel build which won't include this file.
+.export CCACHE_BIN
+# Expand and export some variables so they may be based on make vars.
+# This allows doing something like the following in the environment:
+# CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj'
+.for var in CCACHE_LOGFILE CCACHE_BASEDIR
+.if defined(${var})
+${var}:=       ${${var}}
+.export                ${var}
+.endif
+.endfor
+# Handle bootstrapped compiler changes properly by hashing their content
+# rather than checking mtime.  For external compilers it should be safe
+# to use the more optimal mtime check.
+# XXX: CCACHE_COMPILERCHECK= string:<compiler_version, compiler_build_rev, compiler_patch_rev, compiler_default_target, compiler_default_sysroot>
+.if ${CC:N${CCACHE_BIN}:[1]:M/*} == ""
+CCACHE_COMPILERCHECK?= content
+.else
+CCACHE_COMPILERCHECK?= mtime
+.endif
+.export CCACHE_COMPILERCHECK
+# Remove ccache from the PATH to prevent double calls and wasted CPP/LD time.
+PATH:= ${PATH:C,:?${CCACHE_WRAPPER_PATH}(/world)?(:$)?,,g}
+# Ensure no bogus CCACHE_PATH leaks in which might avoid the in-tree compiler.
+CCACHE_PATH=
+.export CCACHE_PATH
+# Override various toolchain vars.
+.for var in CC CXX HOST_CC HOST_CXX
+.if defined(${var}) && ${${var}:M${CCACHE_BIN}} == ""
+${var}:=       ${CCACHE_BIN} ${${var}}
+.endif
+.endfor
+# GCC does not need the CCACHE_CPP2 hack enabled by default in devel/ccache.
+# The port enables it due to ccache passing preprocessed C to clang
+# which fails with -Wparentheses-equality, -Wtautological-compare, and
+# -Wself-assign on macro-expanded lines.
+.if defined(COMPILER_TYPE) && ${COMPILER_TYPE} == "gcc"
+CCACHE_NOCPP2= 1
+.export CCACHE_NOCPP2
+.endif
+# Canonicalize CCACHE_DIR for meta mode usage.
+.if defined(CCACHE_DIR) && empty(.MAKE.META.IGNORE_PATHS:M${CCACHE_DIR})
+CCACHE_DIR:=   ${CCACHE_DIR:tA}
+.MAKE.META.IGNORE_PATHS+= ${CCACHE_DIR}
+.export CCACHE_DIR
+.endif
+ccache-print-options: .PHONY
+       @${CCACHE_BIN} -p
+.endif # exists(${CCACHE_BIN})
+.endif # ${MK_CCACHE_BUILD} == "yes"
+
 # Try to import COMPILER_TYPE and COMPILER_VERSION from parent make.
 # The value is only used/exported for the same environment that impacts
 # CC and COMPILER_* settings here.

diff --git a/share/mk/bsd.dep.mk b/share/mk/bsd.dep.mk
index 4eb4880da3bed4f4d0946e035d6516f2f698ac6a..a0e989254d3224afd72fae336fb8416659243402 100644 (file)
--- a/share/mk/bsd.dep.mk
+++ b/share/mk/bsd.dep.mk
@@ -48,11 +48,13 @@ CTAGSFLAGS?=
 GTAGSFLAGS?=   -o
 HTAGSFLAGS?=
 
-.if ${CC} != "cc"
-MKDEPCMD?=     CC='${CC} ${DEPFLAGS}' mkdep
-.else
-MKDEPCMD?=     mkdep
+_MKDEPCC:=     ${CC:N${CCACHE_BIN}}
+# XXX: DEPFLAGS can come out once Makefile.inc1 properly passes down
+# CXXFLAGS.
+.if !empty(DEPFLAGS)
+_MKDEPCC+=     ${DEPFLAGS}
 .endif
+MKDEPCMD?=     CC='${_MKDEPCC}' mkdep
 DEPENDFILE?=   .depend
 DEPENDFILES=   ${DEPENDFILE}
 .if ${MK_FAST_DEPEND} == "yes" && ${.MAKE.MODE:Unormal:Mmeta*} == ""

diff --git a/share/mk/bsd.lib.mk b/share/mk/bsd.lib.mk
index 579e36cd9240e9d1656e3eef33688ce2572e905b..7226f3f95d2817d24f0cbbf67bed25f64a6445ab 100644 (file)
--- a/share/mk/bsd.lib.mk
+++ b/share/mk/bsd.lib.mk
@@ -109,21 +109,23 @@ PO_FLAG=-pg
        ${CTFCONVERT_CMD}
 
 .asm.po:
-       ${CC} -x assembler-with-cpp -DPROF ${PO_CFLAGS} ${ACFLAGS} \
-               -c ${.IMPSRC} -o ${.TARGET}
+       ${CC:N${CCACHE_BIN}} -x assembler-with-cpp -DPROF ${PO_CFLAGS} \
+           ${ACFLAGS} -c ${.IMPSRC} -o ${.TARGET}
        ${CTFCONVERT_CMD}
 
 .asm.So:
-       ${CC} -x assembler-with-cpp ${PICFLAG} -DPIC ${CFLAGS} ${ACFLAGS} \
-           -c ${.IMPSRC} -o ${.TARGET}
+       ${CC:N${CCACHE_BIN}} -x assembler-with-cpp ${PICFLAG} -DPIC \
+           ${CFLAGS} ${ACFLAGS} -c ${.IMPSRC} -o ${.TARGET}
        ${CTFCONVERT_CMD}
 
 .S.po:
-       ${CC} -DPROF ${PO_CFLAGS} ${ACFLAGS} -c ${.IMPSRC} -o ${.TARGET}
+       ${CC:N${CCACHE_BIN}} -DPROF ${PO_CFLAGS} ${ACFLAGS} -c ${.IMPSRC} \
+           -o ${.TARGET}
        ${CTFCONVERT_CMD}
 
 .S.So:
-       ${CC} ${PICFLAG} -DPIC ${CFLAGS} ${ACFLAGS} -c ${.IMPSRC} -o ${.TARGET}
+       ${CC:N${CCACHE_BIN}} ${PICFLAG} -DPIC ${CFLAGS} ${ACFLAGS} \
+           -c ${.IMPSRC} -o ${.TARGET}
        ${CTFCONVERT_CMD}
 
 .if !defined(_SKIP_BUILD)
@@ -249,7 +251,7 @@ ${SHLIB_NAME_FULL}: ${SOBJS}
 .if defined(SHLIB_LINK) && !commands(${SHLIB_LINK:R}.ld)
        @${INSTALL_SYMLINK} ${SHLIB_NAME} ${SHLIB_LINK}
 .endif
-       ${_LD} ${LDFLAGS} ${SSP_CFLAGS} ${SOLINKOPTS} \
+       ${_LD:N${CCACHE_BIN}} ${LDFLAGS} ${SSP_CFLAGS} ${SOLINKOPTS} \
            -o ${.TARGET} -Wl,-soname,${SONAME} \
            `NM='${NM}' NMFLAGS='${NMFLAGS}' lorder ${SOBJS} | tsort -q` ${LDADD}
 .if ${MK_CTF} != "no"

diff --git a/share/mk/bsd.opts.mk b/share/mk/bsd.opts.mk
index a1f4ccec2aab73b6da24af4eb739f92370dc9acc..d242da5a59fdae34046391c7efde0f245d1fbfb7 100644 (file)
--- a/share/mk/bsd.opts.mk
+++ b/share/mk/bsd.opts.mk
@@ -66,6 +66,7 @@ __DEFAULT_YES_OPTIONS = \
     WARNS
 
 __DEFAULT_NO_OPTIONS = \
+    CCACHE_BUILD \
     FAST_DEPEND \
     CTF \
     DEBUG_FILES \

diff --git a/share/mk/bsd.prog.mk b/share/mk/bsd.prog.mk
index 37ede4c94c6f71b2708fbd0bbc5a09b4bdab60f4..c4c783fb0084d78ddeb4b0c4c4df0190e3af066b 100644 (file)
--- a/share/mk/bsd.prog.mk
+++ b/share/mk/bsd.prog.mk
@@ -82,9 +82,11 @@ ${PROG_FULL}: beforelinking
 .endif
 ${PROG_FULL}: ${OBJS}
 .if defined(PROG_CXX)
-       ${CXX} ${CXXFLAGS:N-M*} ${LDFLAGS} -o ${.TARGET} ${OBJS} ${LDADD}
+       ${CXX:N${CCACHE_BIN}} ${CXXFLAGS:N-M*} ${LDFLAGS} -o ${.TARGET} \
+           ${OBJS} ${LDADD}
 .else
-       ${CC} ${CFLAGS:N-M*} ${LDFLAGS} -o ${.TARGET} ${OBJS} ${LDADD}
+       ${CC:N${CCACHE_BIN}} ${CFLAGS:N-M*} ${LDFLAGS} -o ${.TARGET} ${OBJS} \
+           ${LDADD}
 .endif
 .if ${MK_CTF} != "no"
        ${CTFMERGE} ${CTFFLAGS} -o ${.TARGET} ${OBJS}
@@ -112,9 +114,11 @@ ${PROG_FULL}: beforelinking
 .endif
 ${PROG_FULL}: ${OBJS}
 .if defined(PROG_CXX)
-       ${CXX} ${CXXFLAGS:N-M*} ${LDFLAGS} -o ${.TARGET} ${OBJS} ${LDADD}
+       ${CXX:N${CCACHE_BIN}} ${CXXFLAGS:N-M*} ${LDFLAGS} -o ${.TARGET} \
+           ${OBJS} ${LDADD}
 .else
-       ${CC} ${CFLAGS:N-M*} ${LDFLAGS} -o ${.TARGET} ${OBJS} ${LDADD}
+       ${CC:N${CCACHE_BIN}} ${CFLAGS:N-M*} ${LDFLAGS} -o ${.TARGET} ${OBJS} \
+           ${LDADD}
 .endif
 .if ${MK_CTF} != "no"
        ${CTFMERGE} ${CTFFLAGS} -o ${.TARGET} ${OBJS}

diff --git a/share/mk/local.meta.sys.mk b/share/mk/local.meta.sys.mk
index adc884bc5d164232611b24a165341c917604a648..9aac03d9d2c156deb9a06ba89cad78b4a8f66ed1 100644 (file)
--- a/share/mk/local.meta.sys.mk
+++ b/share/mk/local.meta.sys.mk
@@ -186,12 +186,6 @@ UPDATE_DEPENDFILE= NO
 # define the list of places that contain files we are responsible for
 .MAKE.META.BAILIWICK = ${SB} ${OBJROOT} ${STAGE_ROOT}
 
-.if defined(CCACHE_DIR)
-CCACHE_DIR := ${CCACHE_DIR:tA}
-.MAKE.META.IGNORE_PATHS += ${CCACHE_DIR}
-.export CCACHE_DIR
-.endif
-
 CSU_DIR.${MACHINE_ARCH} ?= csu/${MACHINE_ARCH}
 CSU_DIR := ${CSU_DIR.${MACHINE_ARCH}}
 

diff --git a/share/mk/sys.mk b/share/mk/sys.mk
index b6c75b6f96a63ef0c333869b25ca0e5ead7de2a2..ee0797e743df5dde3ff17a663a4b3801699ec5c7 100644 (file)
--- a/share/mk/sys.mk
+++ b/share/mk/sys.mk
@@ -317,11 +317,11 @@ YFLAGS            ?=      -d
        ${FC} ${RFLAGS} ${EFLAGS} ${FFLAGS} -c ${.IMPSRC} -o ${.TARGET}
 
 .S.o:
-       ${CC} ${CFLAGS} ${ACFLAGS} -c ${.IMPSRC} -o ${.TARGET}
+       ${CC:N${CCACHE_BIN}} ${CFLAGS} ${ACFLAGS} -c ${.IMPSRC} -o ${.TARGET}
        ${CTFCONVERT_CMD}
 
 .asm.o:
-       ${CC} -x assembler-with-cpp ${CFLAGS} ${ACFLAGS} -c ${.IMPSRC} \
+       ${CC:N${CCACHE_BIN}} -x assembler-with-cpp ${CFLAGS} ${ACFLAGS} -c ${.IMPSRC} \
            -o ${.TARGET}
        ${CTFCONVERT_CMD}
 

diff --git a/sys/conf/kern.post.mk b/sys/conf/kern.post.mk
index 851faaeaf6c0d30798e581ede94db3cd5ed210c8..42f67f644d7e8db9e0a384906dc8aeb52e61cd57 100644 (file)
--- a/sys/conf/kern.post.mk
+++ b/sys/conf/kern.post.mk
@@ -195,6 +195,7 @@ SFILES_CDDL=        ${SFILES:M*/cddl/*}
 kernel-depend: .depend
 # The argument list can be very long, so use make -V and xargs to
 # pass it to mkdep.
+_MKDEPCC:= ${CC:N${CCACHE_BIN}}
 SRCS=  assym.s vnode_if.h ${BEFORE_DEPEND} ${CFILES} \
        ${SYSTEM_CFILES} ${GEN_CFILES} ${SFILES} \
        ${MFILES:T:S/.m$/.h/}
@@ -219,14 +220,14 @@ DEPENDFILES_OBJS+=        .depend.${__obj}
 .if ${MK_FAST_DEPEND} == "no"
        rm -f ${.TARGET}.tmp
        ${MAKE} -V CFILES_NOCDDL -V SYSTEM_CFILES -V GEN_CFILES | \
-           CC="${CC}" xargs mkdep -a -f ${.TARGET}.tmp ${CFLAGS}
+           CC="${_MKDEPCC}" xargs mkdep -a -f ${.TARGET}.tmp ${CFLAGS}
        ${MAKE} -V CFILES_CDDL | \
-           CC="${CC}" xargs mkdep -a -f ${.TARGET}.tmp ${ZFS_CFLAGS} \
+           CC="${_MKDEPCC}" xargs mkdep -a -f ${.TARGET}.tmp ${ZFS_CFLAGS} \
            ${FBT_CFLAGS} ${DTRACE_CFLAGS}
        ${MAKE} -V SFILES_NOCDDL | \
-           CC="${CC}" xargs mkdep -a -f ${.TARGET}.tmp ${ASM_CFLAGS}
+           CC="${_MKDEPCC}" xargs mkdep -a -f ${.TARGET}.tmp ${ASM_CFLAGS}
        ${MAKE} -V SFILES_CDDL | \
-           CC="${CC}" xargs mkdep -a -f ${.TARGET}.tmp ${ZFS_ASM_CFLAGS}
+           CC="${_MKDEPCC}" xargs mkdep -a -f ${.TARGET}.tmp ${ZFS_ASM_CFLAGS}
        mv ${.TARGET}.tmp ${.TARGET}
 .else
        : > ${.TARGET}

diff --git a/sys/conf/kern.pre.mk b/sys/conf/kern.pre.mk
index 4328749447a5a205db4b87dc2df49fb1e3ebbcac..1dbd5a9f3285e8470da981014b87891a25dd5c11 100644 (file)
--- a/sys/conf/kern.pre.mk
+++ b/sys/conf/kern.pre.mk
@@ -128,7 +128,7 @@ CFLAGS+=    ${CONF_CFLAGS}
 LINTFLAGS=     ${LINTOBJKERNFLAGS}
 
 NORMAL_C= ${CC} -c ${CFLAGS} ${WERROR} ${PROF} ${.IMPSRC}
-NORMAL_S= ${CC} -c ${ASM_CFLAGS} ${WERROR} ${.IMPSRC}
+NORMAL_S= ${CC:N${CCACHE_BIN}} -c ${ASM_CFLAGS} ${WERROR} ${.IMPSRC}
 PROFILE_C= ${CC} -c ${CFLAGS} ${WERROR} ${.IMPSRC}
 NORMAL_C_NOWERROR= ${CC} -c ${CFLAGS} ${PROF} ${.IMPSRC}
 

diff --git a/tools/build/options/WITH_CCACHE_BUILD b/tools/build/options/WITH_CCACHE_BUILD
new file mode 100644 (file)
index 0000000..4e29c9a
--- /dev/null
+++ b/tools/build/options/WITH_CCACHE_BUILD
@@ -0,0 +1,42 @@
+.\" $FreeBSD$
+Set to use
+.Xr ccache 1
+for the build.
+No configuration is required except to install the
+.Sy devel/ccache
+package.
+Using with
+.Xr distcc 1
+should set
+.Sy CCACHE_PREFIX=/usr/local/bin/distcc .
+The default cache directory of
+.Pa $HOME/.ccache
+will be used, which can be overridden by setting
+.Sy CCACHE_DIR .
+The
+.Sy CCACHE_COMPILERCHECK
+option defaults to
+.Sy content
+when using the in-tree bootstrap compiler,
+and
+.Sy mtime
+when using an external compiler.
+The
+.Sy CCACHE_CPP2
+option is used for Clang but not GCC.
+ccache works best when combined with the
+.Sy WITH_FAST_DEPEND
+option.
+.Pp
+Sharing a cache between multiple work directories requires using a layout
+similar to
+.Pa /some/prefix/src
+.Pa /some/prefix/obj
+and an environment such as:
+.Bd -literal -offset indent
+CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj'
+.Ed
+.Pp
+See
+.Xr ccache 1
+for more configuration options.