docs/hypervisor-guide: Code Coverage

During a discussion in person, it was identified that Coverage doesn't
currently work for ARM yet.  Also, there are a number of errors with the
existing coverage document.

Take the opportunity to rewrite it in RST, making it easier to follow for a
non-expert user.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>

diff --git a/docs/hypervisor-guide/code-coverage.rst b/docs/hypervisor-guide/code-coverage.rst
new file mode 100644 (file)
index 0000000..3cebf2d
--- /dev/null
+++ b/docs/hypervisor-guide/code-coverage.rst
@@ -0,0 +1,102 @@
+Code Coverage
+=============
+
+Xen can be compiled with coverage support.  When configured, Xen will record
+the coverage of its own basic blocks.  Being a piece of system software rather
+than a userspace, it can't automatically write coverage out to the filesystem,
+so some extra steps are required to collect and process the data.
+
+.. warning::
+
+  Coverage has only been tested on x86 at the moment.  ARM doesn't currently
+  boot when the final binary exceeds 2MB in size, and the coverage build tends
+  to exceeds this limit.
+
+
+Compiling Xen
+-------------
+
+Coverage support is dependent on the compiler and toolchain used.  As Xen
+isn't a userspace application, it can't use the compiler supplied library, and
+instead has to provide some parts of the implementation itself.
+
+For x86, coverage support was introduced with GCC 3.4 or later, and Clang 3.9
+or later, and Xen is compatible with these.  However, the compiler internal
+formats do change occasionally, and this may involve adjustments to Xen.
+While we do our best to keep up with these changes, Xen may not be compatible
+with bleeding edge compilers.
+
+To build with coverage support, enable ``CONFIG_COVERAGE`` in Kconfig.  The
+build system will automatically select the appropriate format based on the
+compiler in use.
+
+The resulting binary will record its own coverage while running.
+
+
+Accessing the raw coverage data
+-------------------------------
+
+The ``SYSCTL_coverage_op`` hypercall is used to interact with the coverage
+data.  A dom0 userspace helper, ``xenconv`` is provided as well, which thinly
+wraps this hypercall.
+
+The ``read`` subcommand can be used to obtain the raw coverage data::
+
+  [root@host ~]# xencov read > coverage.dat
+
+This is toolchain-specific data and needs to be fed back to the appropriate
+programs to post-process.
+
+Alternatively, the ``reset`` subcommand can be used reset all counters back to
+0::
+
+  [root@host ~]# xencov reset
+
+
+GCC coverage
+------------
+
+A build using GCC's coverage will result in ``*.gcno`` artefact for every
+object file.  The raw coverage data needs splitting to form the matching
+``*.gcda`` files.
+
+An example of how to view the data is as follows.  It uses ``lcov`` which is a
+graphical frontend to ``gcov``.
+
+* Obtain the raw coverage data from the test host, and pull it back to the
+  build working tree.
+* Use ``xencov_split`` to extract the ``*.gcda`` files.  Note that full build
+  paths are used by the tools, so splitting needs to output relative to ``/``.
+* Use ``geninfo`` to post-process the raw data.
+* Use ``genhtml`` to render the results as HTML.
+* View the results in a browser.
+
+::
+
+  xen.git/xen$ ssh root@host xencov read > coverage.dat
+  xen.git/xen$ ../tools/xencov_split coverage.dat --output-dir=/
+  xen.git/xen$ geninfo . -o cov.info
+  xen.git/xen$ genhtml cov.info -o cov/
+  xen.git/xen$ $BROWSER cov/index.html
+
+Clang coverage
+--------------
+
+An example of how to view the data is as follows.
+
+* Obtain the raw coverage data from the test host, and pull it back to the
+  build working tree.
+* Use ``llvm-profdata`` to post-process the raw data.
+* Use ``llvm-cov show`` in combination with ``xen-syms`` from the build to
+  render the results as HTML.
+* View the results in a browser.
+
+::
+
+  xen.git/xen$ ssh root@host xencov read > xen.profraw
+  xen.git/xen$ llvm-profdata merge xen.profraw -o xen.profdata
+  xen.git/xen$ llvm-cov show -format=html -output-dir=cov/ xen-syms -instr-profile=xen.profdata
+  xen.git/xen$ $BROWSER cov/index.html
+
+Full documentation on Clang's coverage capabilities can be found at:
+https://clang.llvm.org/docs/SourceBasedCodeCoverage.html

diff --git a/docs/hypervisor-guide/index.rst b/docs/hypervisor-guide/index.rst
index 3e0cada37dc5e364de7c7b6a81134ef6573d6687..cbcae398a2fc38ab4f83232c79e92b56582f752e 100644 (file)
--- a/docs/hypervisor-guide/index.rst
+++ b/docs/hypervisor-guide/index.rst
@@ -3,3 +3,5 @@ Hypervisor documentation
 
 .. toctree::
   :maxdepth: 2
+
+  code-coverage

diff --git a/docs/misc/coverage.pandoc b/docs/misc/coverage.pandoc
deleted file mode 100644 (file)
index 3554659..0000000
--- a/docs/misc/coverage.pandoc
+++ /dev/null
@@ -1,124 +0,0 @@
-# Coverage support for Xen
-
-Coverage support allows you to get coverage information from Xen execution.
-You can see how many times a line is executed.
-
-Some compilers have specific options that enable the collection of this
-information. Every basic block in the code will be instrumented by the compiler
-to compute these statistics. It should not be used in production as it slows
-down your hypervisor.
-
-# GCOV (GCC coverage)
-
-## Enable coverage
-
-Test coverage support can be turned on compiling Xen with the `CONFIG_COVERAGE`
-option set to `y`.
-
-Change your `.config` or run `make -C xen menuconfig`.
-
-## Extract coverage data
-
-To extract data you use a simple utility called `xencov`.
-It allows you to do 2 operations:
-
-* `xencov read` extract data
-* `xencov reset` reset all coverage counters
-
-Another utility (`xencov_split`) is used to split extracted data file into
-files needed by userspace tools.
-
-## Split coverage data
-
-Once you extracted data from Xen, it is time to create files which the coverage
-tools can understand. To do it you need to run `xencov_split` utility.
-
-The utility just takes an input file and splits the blob into gcc .gcda files
-in the same directory that you execute the script. As file names are generated
-relative to the current directory, it could be a good idea to run the script
-from `/` on your build machine.
-
-Code for splitting the blob is put in another utility for some reason:
-* It is simpler to maintain a high level script than a C program;
-* You don't need to execute on the Xen host so you just need to copy the file to
-  your development box (you usually need development files anyway).
-
-## Possible use
-
-**This section is just an example on how to use these tools!**
-
-This example assumes you compiled Xen from `~/xen-unstable` and installed into
-the host. **Consider that if you even recompile Xen you are not able to use
-blob extracted from xencov!**
-
-* Ensure the `lcov` package is installed
-* From the Xen host machine extract the coverage blob
-
-        cd /root
-        xencov read coverage.dat
-
-* Copy the extracted blob to your dev machine
-
-        cd ~
-        scp root@myhost:coverage.dat
-
-* Extract the coverage information
-
-        (cd / && xencov_split ~/coverage.dat)
-
-* Produce coverage html output
-
-        cd ~/xen-unstable
-        rm -rf cov.info cov
-        geninfo -o cov.info xen
-        mkdir cov
-        genhtml -o cov cov.info
-
-* See output in a browser
-
-        firefox cov/index.html
-
-# LLVM coverage
-
-## Enable coverage
-
-Coverage can be enabled using a Kconfig option, from the top-level directory
-use the following command to display the Kconfig menu:
-
-    make -C xen menuconfig clang=y
-
-The code coverage option can be found inside of the "Debugging Options"
-section. After enabling it just compile Xen as you would normally do:
-
-    make xen clang=y
-
-## Extract coverage data
-
-LLVM coverage can be extracted from the hypervisor using the `xencov` tool.
-The following actions are available:
-
-* `xencov read` extract data
-* `xencov reset` reset all coverage counters
-* `xencov read-reset` extract data and reset counters at the same time.
-
-## Possible use
-
-**This section is just an example on how to use these tools!**
-
-This example assumes you compiled Xen and copied the xen-syms file from
-xen/xen-syms into your current directory.
-
-* Extract the coverage data from Xen:
-
-    xencov read xen.profraw
-
-* Convert the data into a profile. Note that you can merge more than one
-  profraw file into a single profdata file.
-
-    llvm-profdata merge xen.profraw -o xen.profdata
-
-* Generate a HTML report of the code coverage:
-
-    llvm-cov show -format=html -output-dir=cov/ xen-syms -instr-profile=xen.profdata
-
-* Open cov/index.html with your browser in order to display the profile.