George Dunlap [Wed, 27 Sep 2017 13:48:54 +0000 (14:48 +0100)]
Use staging rather than master as the default base for `master` commits
The 'master' versions are meant to be applied to staging, and aren't
generally expected to be used directly by downstreams. If there's a
difference between master and staging, having it apply to staging is
more useful.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Mon, 7 Aug 2017 15:57:18 +0000 (16:57 +0100)]
recipe: Fix situation where there's no recipe for a prerequisite
It's not uncommon to "open up" several XSAs in a row, and to begin
work on subsequent ones before the backports to previous ones are
done.
If when making the baseline for a given Xen version for an XSA, one of
the prerequisete XSAs doesn't have a recipe for that Xen version, just
skip making the baseline for that version.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Fri, 23 Jun 2017 16:02:34 +0000 (17:02 +0100)]
systemtest: Always re-run 'repo init'
Always run `repo init`. If `tmp` already exists, delete `tmp/.xsatool`.
Set a limit that ToolConfig.Load won't go up higher than, so that when
starting the test from 'scratch' we don't get "real" configurations
from any higher-level directoiries by accident.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Fri, 5 May 2017 14:33:28 +0000 (15:33 +0100)]
git: Delete stackgit data when deleting or resetting branches
If a developer has modified a branch to be a stackgit branch as part
of development, simply deleting or resetting that branch with git
commands will "work", but will leave leftover state lying around which
may cause problems later.
Before deleting or resetting a branch, check to see if the branch is a
stackgit branch. If so, use `stg branch --delete --force` to get rid of all
the stackgit data.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Wed, 14 Jun 2017 10:06:12 +0000 (11:06 +0100)]
Implement backport
Implement Repo.Rebase() (with full head / base / onto)
Use this to implement Recipe.Backport()
Finally, implement xsatool [nnn] backport:
- Look for the latest xenversion that doens't have a "patch" branch
- Rebase next version from that onto it with 'git rebase'.
Always only do one, so that the caller has a chance to test the build
and look it over before continuing the backport process.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Fri, 16 Jun 2017 10:20:27 +0000 (11:20 +0100)]
XSAMeta: Keep SupportedVersions sorted in reverse order
There are many situations where we want to test things on master first
and then go backwards through supported versions (e.g., building,
looking for backports, &c). Keep the SupportedVersions in this order.
To do this:
- Refactor IsGreaterEqualThen to be usable by both it and IsGreaterThan
- Make a XenVersionSlice type which implements sort.Interface, using the above
- Re-sort XSAMeta.SupportedVersions on insert and load.
Also:
- Have XSAMeta.NewRecipe() add the new SupportedVersions, rather
than having the caller add it manuall.
- Keep XenVersion.PrevVersion() around, even though we're not using
it for now, in case it becomes useful in the future.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Fri, 16 Jun 2017 10:13:40 +0000 (11:13 +0100)]
Build: Use mirror paths for qemu unless there are patches
Cloning from the working tree is running into build problems saying it
can't find the right changeset, even though the changeset is present
in the working tree. For now, use the mirror directory.
This needs to be sorted out properly somehow.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Thu, 8 Jun 2017 16:37:45 +0000 (17:37 +0100)]
Use new branching scheme
New branching scheme:
- $v-stable based on the recorded stable branch at the time the xsa
was created
- $v-baseline; the above, but with 'prerequisites' (outstanding XSAs
not published yet)
- $v: Branch with actual patches, only present when patches exist (or
are being created)
To do this:
- Separate Recipe.Apply() into ApplyBaseline (which only does -stable
and -baseline) and ApplyPatches (which makes the main branch)
- Get rid of IsApplied(), which doesn't do what it says on the tin
- Implement HasPatchBranch, which detect whether the "patch"
branch(es) exist for a given recipe
Then:
- During Init, call only ApplyBaseline for all branches; only call
ApplyPatches for master.
- When calling sync-patches with 'all', have VersionsFromArgs filter
out patches based on HasPatchBranch. (sync-branches and test will still
test all supported branches for 'all'.)
- Add a test for the failure path of sync-branches with a
non-existent branch
While we're here:
- Use fmt.Print instead of fmt.Printf when there are no format tokens
- Add missing '\n' to many status output messages
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Fri, 19 May 2017 16:05:44 +0000 (17:05 +0100)]
Change to in-binary test
`go test` apparently has a default 10-minute timeout. This isn't
enough for all the tests in the "system test", and there's no way to
change the default except to manually add a --timeout parameter to "go
test".
Instead pull the test out into a special xsatool command.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Fri, 19 May 2017 14:01:17 +0000 (15:01 +0100)]
main: Implement 'init' command
New method of creating a new xsa: "xsatool NNN init [trees]"
This will:
- For a given set of trees
- And all the currently-supported versions
- Create an XSA that contains all the currently-outstanding XSAs for that tree and set of versions
- And create branches for all of those versions in the correct tree
- And add it to the list of currently-outstanding XSAs
At that point, the user can go into the appropriate tree and create patches.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Wed, 17 May 2017 16:49:02 +0000 (17:49 +0100)]
Recipe: Allow MakePatches to detect and tolerate "empty" trees
In order to move to a world where all supported branches are created
automatically, with prerequisites applied, as "empty" when the XSA is
created, allow Recipe.MakePatches() to detect and tolerate "empty"
versions:
- Have MakePatches() return the number of trees which contained
patches
- Instead of verifying the reference, get the hashes for base and
branch and compare them; simply return 0 if they are the same
- Don't bother copying patches if none are to be created.
Modify xsa.go:sync() to handle this case as well.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Wed, 10 May 2017 16:09:48 +0000 (17:09 +0100)]
Testing: Introduce 'system' test
To begin with, just clone repos and begin to set them up in a specific
state, so that we can test operations.
To make testing iteration faster, set FullClone == false. After test
development is done, this should be set to "true" to make the cloning
continues to work.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Wed, 10 May 2017 13:27:05 +0000 (14:27 +0100)]
Refactor main to be more test-friendly
Have a special function, XsaMain(), which does the work and is passed
parameters and a return value. Have a wrapper which interacts with
the operating system under normal conditions. This allows the testing
infrastructure to simulate being called as closely as possible.
Get rid of global references to 'xsa' entirely; instead pass
references to a local copy on the stack.
Collect all remaining global state into a struct called 'G', and reset
its state every time XsaMain() is called.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Fri, 28 Apr 2017 10:27:47 +0000 (11:27 +0100)]
Introduce Tree type to avoid hard-coding tree names
Replace repos.{xen,qemuu,qemut}, config.*Path, and Recipes.{Xen,Qemu*}
with arrays indexed by Tree. Replace Get*Path with GetPath(t).
Replace XRType with Tree. Introduce ForEachCodeTree, ForEachTree, and
ForEachXenRepo helper functions to run a specific function for each
tree / repo; rewrite lots of code to use those functions instead of
doing code duplication.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Fri, 24 Mar 2017 17:44:50 +0000 (17:44 +0000)]
Finish sync-patch, add sync-branch as well
Have MakePatches include a flag deciding whether to sync or not.
If set, after creating all appropriate patches in /tmp, make the
appropriate directory/ies in xsa.git and copy the files there (or just
copy a single patch if the file is a single patch).
Also implement sync-branch which will delete and re-create xsa/NNN/VVV
from xsa.git according to the recipe.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Fri, 24 Mar 2017 11:07:16 +0000 (11:07 +0000)]
main: Add partial implementation of sync-patches command
Partial implementation of sync-patches, which will look for trees
which contain xsa/NNN/VVV and xsa/NNN/VVV-baseline and run
`git format-patch` to generate patches.
To do this:
- Implement Repo.FormatPatch()
- Implement XSAMeta.GetRecipe(version) to get a recipe for a specific version
- Modify IsApplied() to look for both $prefix/NNN/VVV and $prefix/NNN/VVV-baseline
- Implement Recipe.MakePatches()
- Modify IsApplied() to return only an error; but make return a
distinguishable type to say what branch wasn't applied
A full implementation would also copy the patches into xsa.git either
as a single patch or a directory full of patches, as appropriate.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Thu, 23 Mar 2017 13:52:02 +0000 (13:52 +0000)]
Main: Fake up config and xsa info
"Fake up" config and XSA info in Main(), so that we can start writing
functionality as though the config loading already existed, rather
than have to store it all in test.go.
This should make it easier to extend functionality as needed in a way
which will actually be useful once the config loading stuff is ready.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Thu, 23 Mar 2017 11:38:50 +0000 (11:38 +0000)]
Recipe: Break down per-tree recipe into prerequisites and patches
Create a new struct, TreeRecipe, to be the recipe for a particular
tree (Xen, QemuU or QemuT). Have this contain separate prereq and
patches sections.
When building, create a branch $branch-baseline to apply the Prereq
patches to, and then base $branch based on the baseline.
The idea here is to make an easy way to programmatically sync changes
to the tree back to xsa.git.
Also unconditionally use local qemu trees, even if we don't have local
patches, because we know they'll be up to date with upstream changes
(since we pull at start-of-day).
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Thu, 23 Mar 2017 10:46:43 +0000 (10:46 +0000)]
Recipe: Make more flexible for both backporting and testing
Make the interface a bit more flexible, so it can be used both for
backporting (which will create branches named xsa/NNN/VVV), and for
testing (which will create branches named test/NNN/VVV).
Add branchName() helper to calculate branch name, rather than storing
a single tempBranch.
Add IsApplied() method, which will (at the moment) only check for the
existence of the target branches in the appropriate trees.
Also:
- Use AmClean() rather than Am() to make it easier to follow up a
failure by cherry-picking
- Fix a bug in Build() which would build the wrong version
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Thu, 23 Mar 2017 10:39:27 +0000 (10:39 +0000)]
Recipe: Don't clean up temporary branches automatically (for now)
When doing testing, we probably want to remove temporary branches; but
when doing backporting we most certainly do not. This is something
which still needs to be figured out.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Thu, 23 Mar 2017 10:30:33 +0000 (10:30 +0000)]
git: Implement VerifyRef and AmClean
VerifyRef returns true if the specified ref exists.
AmClean() is like Am(), but will reset the tree to a useable state if
the Am fails. This is useful for situations like backporting where
you typically want to cherry-pick on a failed `git am` to take
advantage of the three-way merge functionality.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Thu, 16 Mar 2017 17:15:15 +0000 (17:15 +0000)]
Allow Recipes to have multiple globs, implement Recipe.Build()
Allow each recipe to have multiple "globs" associated with each tree.
Implement Recipe.Build(), which will construct the appropriate config
file to make the build use the appropriate branches, and then
configure and run make.
While we're here fix some mistaken status messages in Recipe.Cleanup(), and
fix XenRepo.CheckoutVersion when XenVersion == Master.
And update the "test" to XSA211.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap [Fri, 24 Feb 2017 16:36:48 +0000 (16:36 +0000)]
Implement basic repoinit
Clone xen.git, qemu-xen.git, qemu-xen-traditional.git, and xsa.git,
and set up a default .xsatool config file. Disable pushes on the xen
and qemu branches.
Modify gitCmd() to not add the -C prefix if r.Path isn't set.
Implement GitClone and XenRepoClone, and Repo.DisablePush.
Move global repo s into a global structure called 'repo'; and also
include a plain Repo for xsa.
Implement ToolConfig type, with Load and Save.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>