--- /dev/null
+# Purpose
+
+The primary goal of xsatool is to help verify that the patches we're
+going to send out apply and build properly.
+
+A secondary goal is to help in the generation of those patches.
+
+In all cases it is meant to do a lot of the "busy-work" which is
+boring, repetitive, and error-prone when humans do it, but quick,
+efficient, and predictable when computers do it.
+
+See the **Workflow** section for a quick usage guide.
+
+# Design
+
+Our current standard practice is to provide a patch or series (or
+multiple patches or series), such `master`, as well as all currently
+supported versions under security support, will have fixes which apply
+via `git am`.
+
+However, these patches:
+
+1. Apply to a specific tree (`xen.git`, `qemu-xen.git`,
+ `qemu-xen-traditional.git`)
+
+2. Apply to a specific point on some branch (`master` for
+ `xen-unstable`, `stable-${v}` for released versions).
+
+3. May rely on previous unreleased XSAs.
+
+This tool introduces the concept of a *recipe*. Each supported
+release will have its own recipe. The recipe will include the
+specific tree it's targetet at, the *baseline* in `stable-${v}` on
+which it's applied, as well as any XSA prerequisites. The recipe
+would ideally be both machine- and human-readiable, and live in
+`xsa.git`.
+
+Which in turns leads us to the next thing: rather than having to
+specify each time what XSAs need to be applied, this should happen
+automatically. There should be global metadata that contains:
+
+- All the currently supported Xen versions
+
+- All XSAs not yet in the various stable branches
+
+When a new XSA metadata is created with xsatool, "empty" recipes
+should be created based for each currently-supported version,
+including all the previous XSAs on that tree for that version. The
+new XSA should also be added to the list of "Current XSAs". (This in
+particular will need humans to do adjustment when circumstances don't
+allow the release in a nice clean order.)
+
+It should probably be obvious that creating this XSA 'recipe' (as
+opposed to allocating the XSA number) shouldn't be done until a patch
+is reasonably close to being ready; XSA-115 provides a good reason
+why.
+
+The actual creation and manipulation of the patches will be highly
+context-dependent and developer-driven; to that end the standard will
+be to create branches with a specific naming scheme in the various
+trees, which developers can then manipuate. xsatool will have
+commands to 'sync' between the trees.
+
+# Data files
+
+## .xsatool
+
+xsatool expects a top-level directory that contains a file named
+`.xsatool`. This configuration file contains paths for xen,
+qemu-upstream, qemu-traditional, and xsa.git. `xsatool` will
+manipulate these working trees freely, so it is highly advised that
+these trees be separate from normal development trees.
+
+## xsa.git/global.meta
+
+Lives in `xsa.git`. Contains metadata about the XSA tree as a whole.
+At the moment this includes what versions of Xen are currently
+supported, as well as any outstanding (uncommitted) XSAs.
+
+## xsa.git/xsa${N}.meta
+
+Lives in `xsa.git`. Contains metadata about XSA ${N}. Includes which
+versions of Xen and which trees this particular XSA applies to, as
+well as recipes for applying the patch. The recipe includes a
+reference to the point in the stable tree to which the patch will
+apply, other prerequisite XSAs which must be applied first, and a
+"glob" of files to apply for this patch.
+
+## Data format
+
+The current version of the tool uses JSON to store data. It is very
+easy in Go to marshall and unmarshall structures to and from JSON, but
+it is still somewhat human-readable and editable (although it's not
+particularly great).
+
+How human-readable it is depends somewhat on the way the Go structure
+is formatted; I've tried to format the structures such that they have
+reasonable-looking JSON output which can be edited by hand without too
+much effort if need be.
+
+# Trees
+
+## Branch names
+
+When an "empty" XSA is created, two three will be created in each tree
+for which patches are required:
+
+ xsa/${n}/${v}-stable
+ xsa/${n}/${v}-baseline
+ xsa/${n}/${v}
+
+`${v}` is the major release version. The `-stable` branch points to
+the commit in the `stable-${v}` tree at the point at which the xsa was
+created. `baseline` is `stable` with all of the prerequisite XSAs
+applied. The last branch is the branch that contains the actual fix.
+
+## Generating patches from a branch
+
+To create a set of patches for a particular xen version for a
+particular xsa, the tool will run `git format-patch
+xsa/${n}/${v}-baseline..xsa/${n}/${v}` into a temporary directory.
+
+If the result is only a single patch, then the patch will be copied to
+xsa.git/xsa${n}-${VERSION}.patch. If the result is more than one patch,
+the patches will be copied into xsa.git/xsa${n}-${version}/.
+
+# Initial setup
+
+## Creating initial setup
+
+`xsatool repo init`
+
+This command will, in the current directory:
+
+- Create a new `.xsatool` if one doesn't exist. If it does, it will
+be consulted.
+
+- For each of `xen.git`, `qemu-xen.git`, `qemu-xen-traditional.git`,
+and `xsa.git`, it will clone a new copy if one doesn't exist already.
+Existing repositories will be left alone.
+
+- It will also disable the default push on all of those repositories,
+even if they exist already. (Command `git config
+config.origin.pushurl http://localhost/fail`.)
+
+**TODO**: Enable `git rerere` when creating new repositories
+
+## Creating or updating the initial global metadata
+
+`xsatool global update`
+
+If this is run and `xsa.git/global.meta` doesn't exist, it will create
+the file. It firsts asks what the oldest supported Xen version is.
+It then goes through `xen.git` and looks for `RELEASE-M.m.p` tags, and
+marks each version greater than or equal to the specified version as
+supported.
+
+If `global.meta` does exist, it simply does a refresh; this will
+automatically add support for new versions.
+
+# Workflow
+
+## New Xen XSA
+
+Using XSA 213 as an example.
+
+`xsatool 213 init xen`
+
+ - Create `xsa213.meta` with "empty" recipes for master and all
+ currently-supported versions; in this case these versions will
+ probably be `master`, `4.8`, `4.7`, `4.6`, and `4.5`. Tell
+ `xsatool` that `xen` is the only tree that needs patches.
+
+ - Take the current value of `xen.git` `stable-${v}` for the "stable
+ baseline" for each version
+
+ - In `xen.git`:
+
+ - create branches `xsa/213/xsa/${v}-{stable,baseline}`
+
+ - create branch `xsa/213/master`
+
+The developer now generates patch for XSA 213 on `xsa/213/master`.
+
+`xsatool 213 backport`
+
+ - Starting with the latest release and going backwards, looks for
+ versions such that `xsa/213/$v` doesn't exist.
+
+ - It will then create a branch which is a clone of `xsa/213/${v+1}`,
+ and then rebase onto `xsa/213/${v}-baseline`.
+
+If the rebase fails, the developer fixes it up just like she would
+normally.
+
+If the rebase succeeds, the tool still stops, giving the developer a
+chance to inspect and test the backport.
+
+The developer repeats `xsatool 213 backport` until all backports have
+been made.
+
+`xsatool 213 sync-patches`
+
+For each version fro which `xsa/213/${v}` exists, this will:
+
+ - Delete any patches in xsa.git which match its file generation pattern.
+
+ - Generate patches in xsa.git
+
+ - Update the recipe for that version in `xsa213.meta`
+
+## Checking patches before sending them out
+
+`xsatool 213 test-apply`
+
+For each version for which patches exist in `xsa213.meta`, it creates
+`test/213/$v-stable`, `test/213/$v-baseline`, and `test/213/$v`
+
+Individual versions can be specified as well: `xsatool 213 test-apply 4.6 4.5`
+
+`xsatool 213 test-build`
+
+For each version for which patches exist in `xsa213.meta`, it will
+attempt to check out a tree based on `test/213/$v` and do a build. If
+the branch does not exist it will throw an error.
+
+As with `test-apply`, you can also specify individual versions:
+`xsatool 213 test-build 4.5`
+
+`xsatool 213 test`
+
+Do both `test-apply` and `test-build` (i.e., a full test) for each
+version.
+
+## Viewing patches that another developer has created
+
+Suppose instead you want to review another developer's series; or
+suppose another developer has made update to your own series, and you
+want to update your copy.
+
+Assuming the developer has checked her changes into `xsa.git`, update
+your copy of `xsa.git`, and then
+
+`xsatool 213 sync-branch`
+
+For each version specified, it will reset all branches (`-stable`,
+`-baseline`, and patch branch) according to the updated recipe in
+`xsa213.meta`.
+
+As with `sync-patches`, you can either specify a specific version, or
+you can leave the version blank and it will make branches for versions
+*for which recipes exist*.
+
+This could also be useful, for example, if prerequisites are manually
+added or removed.
+
+# Potential future functionality
+
+## Automatically de-duplicating patches
+
+At the moment the tool will always create a patch or a patch series
+for each supported version, even if a single patch can apply to
+multiple versions. Detecting this automatically would be one of the
+obvious next steps.
+
+## Removing xsas from the 'outstanding' list
+
+Either something like:
+
+`xsatool global end-support nnn`
+
+Or having `global update` automatically detect that patches have been
+applied to stable branches.
+
+## Generating osstest test runes automatically
+
+## Rebasing
+
+- Rebasing to new versions of `stable` or `master`
+
+- Automatically rebasing when prerequisites are update.
+
+## Resetting backports
+
+Have a command to automatically blow away all patch branches except
+for `master`, so that `backport` has a clean slate to work from.
+
+## Using `fixup` functionality for backports
+
+Developer checks out xsa/213/master, makes a fix, and commits it using git commit --fixup=$ref
+
+`xsatool 213 backport-fixup [version]`
+
+ - Starting with either xsa/213/master or xsa/213/${version}, looks
+ for patches with 'fixup' in the header, and cherry-picks them (or
+ rebases them?) onto previous versions
+
+`xsatool 213 backport-squash [version]`
+
+ - Starting with xsa/213/master (or the version specified), run `git
+ rebase --autosquash`
+
+## Getting information in a more human-friendly way
+
+`xsatool NNN info`
+
+`xsatool repo info`
+
+`xsatool global info`
+
+# Open issues
+
+# Inter-tree dependcies
+
+Do we need to worry about inter-tree dependencies? For example, could
+it ever be the case that xsa 251, for xen.git, depends on xsa250,
+which contains both xen.git and qemu-upstream.git components? If so
+then when testing xsa251 we'd have to make sure that *all* the trees
+were in the appropriate state.
+
projects / people / gdunlap / xsatool / commitdiff