From: Lars Kurth Date: Thu, 27 Jul 2017 21:47:01 +0000 (+0100) Subject: Make README comply with 80char line length and Pandoc compliant + other format improv... X-Git-Url: http://xenbits.xensource.com/gitweb?a=commitdiff_plain;h=c9def66ef692762a5a338face42bff1fc34f08bd;p=people%2Flarsk%2Fxen-release-scripts.git Make README comply with 80char line length and Pandoc compliant + other format improvements --- diff --git a/README b/README index 8e9d126..d2cf3f2 100644 --- a/README +++ b/README @@ -4,23 +4,23 @@ OVERVIEW Directory structure ------------------- -To keep things clean, I use the following directory structure +To keep things clean, use the following directory structure your root directory - xen-release-logs [output/input directory] - xen-release-scripts [git repo] - - xsa-lists [output/input directory, see steps 1-5 in match-xsa] + - xsa-lists [output/input directory, + see steps 1-5 in match-xsa] - xsa [git repo, optional] + Prerequisites ------------- match-xsa and xen-release-logs, require perl and the following perl libraries libfile-slurp-perl - libfile-spec-perl + libfile-spec-perl -See https://wiki.koha-community.org/wiki/How_to_install_missing_Perl_modules_on_Debian on how -to install these Tested on --------- @@ -29,15 +29,19 @@ The scripts were tested on * MacOS Sierra, but do require an installation of pypy and wget * Debian GNU/Linux 8 (jessie) + Tools overview -------------- * xen-release-logs: produces logs for various xen-releases -* match-xsa: checks whether XSAs have been applied to a specific Xen, -QEMU traditional and QEMU upstream tree -* make-webpage: creates webpages for https://xenproject.org/downloads/xen-archives.html +* match-xsa: checks whether XSAs have been applied to a specific Xen, + QEMU traditional and QEMU upstream tree +* make-webpage: creates webpages for + https://xenproject.org/downloads/xen-archives.html Helper tools: can be executed by security team members only -xsa-list-send: sends an XSA file to an e-mail address (not properly tested, needs editing) +xsa-list-send: sends an XSA file to an e-mail address (not properly tested, +needs editing) + Users ===== @@ -46,39 +50,43 @@ This document is written for specific users * Security Team Members * Release Managers and Release Maintainers + Logfiles, --version, --major, --since, --until and --logroot ------------------------------------------------------------ -The tools xen-release-logs, match-xsa and make-webpage use the following common options: ---version, --major, --since, --until and --logroot +The tools xen-release-logs, match-xsa and make-webpage use the following +common options: + + --version, --major, --since, --until and --logroot -Depending on --version, --major, --since and --until a log directory, which contains checked -out git trees, will be created in the current directory or a directory specified with ---logroot +Depending on --version, --major, --since and --until a log directory, which +contains checked out git trees, will be created in the current directory or +a directory specified with --logroot These options are used in the following way --version 4 --major 7 -Gets the logs for the entire 4.7 release up to the specific stable heads on the correct branch. -In this example stable-4.7. The tools use the 4.6.0 release tags as common root. The log -directory is called logs-47none-stable. Use for new major releases, when not yet tagged. +Gets the logs for the entire 4.7 release up to the specific stable heads on +the correct branch. In this example stable-4.7. The tools use the 4.6.0 +release tags as common root. The log directory is called logs-47none-stable. +Use for new major releases, when not yet tagged. --version 4 --major 7 --until 0 -This is a variant of the previous command, which is used when the tree has already been tagged -(in this case with 4.7.0 tags). The log directory is called logs-47none-0. Use for new major -releases, when the tree is tagged. +This is a variant of the previous command, which is used when the tree has +already been tagged (in this case with 4.7.0 tags). The log directory is +called logs-47none-0. Use for new major releases, when the tree is tagged. --version 4 --major 7 --since 0 -Gets the logs from 4.7.0 to 4.7.1 (with 4.7.1 not yet tagged). The log directory is called -logs-470-stable. +Gets the logs from 4.7.0 to 4.7.1 (with 4.7.1 not yet tagged). The log +directory is called logs-470-stable. --version 4 --major 7 --since 0 --until 1 -Gets the logs from 4.7.0 to 4.7.1 (with 4.7.1 already tagged). The log directory is called -logs-470-1. +Gets the logs from 4.7.0 to 4.7.1 (with 4.7.1 already tagged). The log +directory is called logs-470-1. Note: you can also use the options for several releases, e.g. @@ -86,10 +94,12 @@ Note: you can also use the options for several releases, e.g. The log directory is called logs-470-4. + Developers and Users -------------------- -For developers and users only the match-xsa tool is useful. However, the tool requires some -input files, which currently can only be generated by security team members. +For developers and users only the match-xsa tool is useful. However, the tool +requires some input files, which currently can only be generated by security +team members. Create the following directories - xen-release-logs [output/input directory] @@ -97,55 +107,60 @@ Create the following directories - xsa-lists [save XSA list files in this directory] *STEP 1:* go to download page of previous release (in this case: -https://xenproject.org/downloads/xen-archives/xen-project-47-series/xen-472.html) +https://xenproject.org/downloads/xen-archives/xen-project-47-series/ +xen-472.html) *STEP 2:* write down last applied and verified XSA+1 (in this case XSA 210) -*STEP 3:* ask a security team member to create an XSA file using ./Infra/xsa-list-patches -and copy to file into xsa-lists with an appropriate name (e.g. xsa-210-222 or -xsa-213-225) +*STEP 3:* ask a security team member to create an XSA file using ./Infra/xsa- +list-patches and copy to file into xsa-lists with an appropriate name (e.g. +xsa-210-222 or xsa-213-225) *STEP 4:* run the tool in dumb mode, which checks name signatures only. - ./match-xsa --version 4 --major 8 --since 1 --xsa ../xsa-lists/xsa-213-225 --getlogs - --html > ../xen-release-logs/481-stable-xsamatch.html + ./match-xsa --version 4 --major 8 --since 1 --getlogs --html + --xsa ../xsa-lists/xsa-213-225 + > ../xen-release-logs/481-stable-xsamatch.html -The above command line will create a report that shows which XSA patches have matching -commit messages to those in the XSA list file. You will have to manually check whether all -relevant patches have been applied. Typically where there are no matches, the XSA does not -apply (e.g. the XSA may apply to Linux or older Xen releases only): you will have to go and -double check the XSA, which can be easily accessed from within the +The above command line will create a report that shows which XSA patches have +matching commit messages to those in the XSA list file. You will have to +manually check whether all relevant patches have been applied. Typically +where there are no matches, the XSA does not apply (e.g. the XSA may apply to +Linux or older Xen releases only): you will have to go and double check the +XSA, which can be easily accessed from within the -*STEP 5:* run the tool in smart mode, which checks against real patches rather then commit -message titles. Note that this check is more fragile and can throw up false positives in the -following situations: +*STEP 5:* run the tool in smart mode, which checks against real patches +rather then commit message titles. Note that this check is more fragile and +can throw up errors in the following situations: * A wrong patch has been applied -* The committer has made changes to the patch when committing (this may be a real issue) -* The *.patch file in the XSA has been created by a different diff version and file thunks -are ordered differently (this is a tool issue that needs to be fixed). +* The committer has made changes to the patch when committing (this may be a + real issue) +* A bug in the match-xsa tool (none currently known) - ./match-xsa --version 4 --major 8 --since 1 --xsa ../xsa-lists/xsa-213-225 - --html --smart > ../xen-release-logs/481-stable-xsamatch-smart.html + ./match-xsa --version 4 --major 8 --since 1 --html --smart + --xsa ../xsa-lists/xsa-213-225 + > ../xen-release-logs/481-stable-xsamatch-smart.html -Note that you can omit the --getlogs option, as you have the logs already. If there are -unexplained. If there are unexplained discrepancies between +Note that you can omit the --getlogs option, as you have the logs already. -Also note that the tool will fetch XSA patches from xenbits.xenproject.org/xsa and store -these in a ../xsaweb directory. The tool uses git show to fetch commits from tree and will -compare these against patches. +Also note that the tool will fetch XSA patches from xenbits.xenproject.org/ +xsa and store these in a ../xsaweb directory. The tool uses git show to fetch +commits from tree and will compare these against patches. -If at this stage, there no unexplained discrepancies between XSAs as published on -xenbits.xenproject.org/xsa, you are done. If there are you can do the following... +If at this stage, there no unexplained discrepancies between XSAs as +published on xenbits.xenproject.org/xsa, you are done. If there are you can +do the following... *STEP 6:* run the tool in smart and debug mode - ./match-xsa --version 4 --major 8 --since 1 --xsa ../xsa-lists/xsa-213-225 - --html --smart --debug > ../xen-release-logs/481-stable-xsamatch-smartd.html + ./match-xsa --version 4 --major 8 --since 1 --html --smart --debug + --xsa ../xsa-lists/xsa-213-225 + > ../xen-release-logs/481-stable-xsamatch-smartd.html -In this mode, git diffs, patches and logs are saved in a debug directory in the log directory -and are accessible from the generated html (via a DEBUG link). You can look at these to -investigate issues. Let's say a xen patch has been matched by commit message, the following -files will be generated: +In this mode, git diffs, patches and logs are saved in a debug directory in +the log directory and are accessible from the generated html (via a DEBUG +link). You can look at these to investigate issues. Let's say a xen patch has +been matched by commit message, the following files will be generated: * xen-git.txt : this is the original patch from git show * xen-patch.txt : this is the matching patch from xenbits.xenproject.org/xsa @@ -155,21 +170,25 @@ files will be generated: * xen-diff-n.txt: a diff between xen-git-n.txt and xen-patch-n.txt * xen---log.txt : a log on the tool run for xen.git -Similar files will be available for qemu traditional (qemut*) and qemu upstream (qemuu*). +Similar files will be available for qemu traditional (qemut*) and qemu +upstream (qemuu*). + Security Team Members --------------------- -In addition to the regular workflow as outlined in the previous section, security team members -have the capability to use match-xsa on xsa.git (only accessible for security team members). +In addition to the regular workflow as outlined in the previous section, +security team members have the capability to use match-xsa on xsa.git (only +accessible for security team members). To do this use ./match-xsa ... --xsadir In addition, security team members can also create XSA list files using -./Infra/xsa-list-patches or xsa-list-send (needs to be modified), which will call -xsa-list-patches and send an email to either yourself or an another person. +./Infra/xsa-list-patches or xsa-list-send (needs to be modified), which will +call xsa-list-patches and send an email to either yourself or an another +person. Release Managers and Release Maintainers @@ -177,26 +196,37 @@ Release Managers and Release Maintainers Release Managers can use the following two tools: -* xen-release-logs: creates change logs for xen, qemuu and qemut in various formats that can be copied into wikis and webpages +* xen-release-logs: creates change logs for xen, qemuu and qemut in various + formats that can be copied into wikis and webpages -* make-webpage (using the same options as the other tools with the addition of "--xsastart 210 --xsaend 225"). This tool generates text for the download page, with a table that needs to be manually populated. +* make-webpage (using the same options as the other tools with the addition + of "--xsastart 210 --xsaend 225"). This tool generates text for the + download page, with a table that needs to be manually populated. -TODOS AND OPEN ISSUES -===================== +To Dos and Open Issues +====================== -*match-xsa* -* The reporting can probably be improved: aka, highlight blocks with potential issues better. For example, embed relevant portions of an XSA into the output in cases where there is no match -* --xsadir currently only works when providing an absolute path (the issue was introduced in commit 6d371d6adccc36eeeb5d2acd412ba0f3424a4a15) - -*xsa.git:Infra/xsa-list-patches* +match-xsa +--------- +* The reporting can probably be improved: aka, highlight blocks with + potential issues better. For example, embed relevant portions of an XSA + into the output in cases where there is no match +* --xsadir currently only works when providing an absolute path (the issue + was introduced in commit 6d371d6adccc36eeeb5d2acd412ba0f3424a4a15) + +xsa.git:Infra/xsa-list-patches +------------------------------ * Provide an option that allows to -*make-xsa-list (new tool)* -* Create a new tool that fetches the XSA webpage and produces output compatible to Infra/xsa-list-patches +make-xsa-list (new tool) +------------------------ +* Create a new tool that fetches the XSA webpage and produces output + compatible to Infra/xsa-list-patches The file format is -*make-webpage* +make-webpage +------------ * Make generation of XSA table fully automatic