]> xenbits.xensource.com Git - people/gdunlap/xsatool.git/log
people/gdunlap/xsatool.git
7 years agoUse staging rather than master as the default base for `master` commits
George Dunlap [Wed, 27 Sep 2017 13:48:54 +0000 (14:48 +0100)]
Use staging rather than master as the default base for `master` commits

The 'master' versions are meant to be applied to staging, and aren't
generally expected to be used directly by downstreams.  If there's a
difference between master and staging, having it apply to staging is
more useful.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoxsa: Add set-patches command to set patches for a particular xsa + version
George Dunlap [Wed, 27 Sep 2017 13:40:45 +0000 (14:40 +0100)]
xsa: Add set-patches command to set patches for a particular xsa + version

...rather than editing the metadata file manually.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoxsatool: Update README.md with more complete usage information
George Dunlap [Wed, 27 Sep 2017 11:34:40 +0000 (12:34 +0100)]
xsatool: Update README.md with more complete usage information

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoxsa: Return success when backport succeeds
George Dunlap [Fri, 15 Sep 2017 14:56:57 +0000 (15:56 +0100)]
xsa: Return success when backport succeeds

Forgot to change the "unimplemented" message and return value after
implementing the 'backport' command.

While we're here, make backport a bit more chatty about what it's
doing.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agorecipe: Bring patch name for master in line with standard practice
George Dunlap [Fri, 15 Sep 2017 14:52:09 +0000 (15:52 +0100)]
recipe: Bring patch name for master in line with standard practice

Standard practice is to have the patch that applies to master be
simply "xsaNNN.patch" (or xsaNNN/*.patch in case of a series).

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agorecipe: Fix situation where there's no recipe for a prerequisite
George Dunlap [Mon, 7 Aug 2017 15:57:18 +0000 (16:57 +0100)]
recipe: Fix situation where there's no recipe for a prerequisite

It's not uncommon to "open up" several XSAs in a row, and to begin
work on subsequent ones before the backports to previous ones are
done.

If when making the baseline for a given Xen version for an XSA, one of
the prerequisete XSAs doesn't have a recipe for that Xen version, just
skip making the baseline for that version.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoImplement reset-base
George Dunlap [Mon, 26 Jun 2017 11:17:32 +0000 (12:17 +0100)]
Implement reset-base

Implement reset-base, to reset the 'base' version to the upstream
stable trees without rebasing.

While we're here, document 'repo update' and add some comments.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agosystemtest: Include fakeups for all 206 backports
George Dunlap [Tue, 27 Jun 2017 10:06:02 +0000 (11:06 +0100)]
systemtest: Include fakeups for all 206 backports

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agorepo init: Enable git rerere on the repos by default
George Dunlap [Mon, 26 Jun 2017 15:02:54 +0000 (16:02 +0100)]
repo init: Enable git rerere on the repos by default

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoREADME: Include a brief description of how to build.
George Dunlap [Mon, 26 Jun 2017 14:02:29 +0000 (15:02 +0100)]
README: Include a brief description of how to build.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoREADME: Mention mirrordir and gitproxy
George Dunlap [Mon, 26 Jun 2017 13:59:35 +0000 (14:59 +0100)]
README: Mention mirrordir and gitproxy

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agometa: Get rid of VersionMeta.Latest
George Dunlap [Mon, 26 Jun 2017 11:44:36 +0000 (12:44 +0100)]
meta: Get rid of VersionMeta.Latest

NB that the action of resetting outstanding XSAs will be manual as of
this commit.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agorepo init: Add --mirrordir and --gitproxy options
George Dunlap [Fri, 23 Jun 2017 16:56:36 +0000 (17:56 +0100)]
repo init: Add --mirrordir and --gitproxy options

...to set the appropriate proxy location.  Also add options to systemtest
to allow testing with those options.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoToolConfig: Add MirrorDir and ProxyURL options
George Dunlap [Fri, 23 Jun 2017 16:56:29 +0000 (17:56 +0100)]
ToolConfig: Add MirrorDir and ProxyURL options

If available, use in Repo.Build when building Xen to create .config.

To do this:

- Make a XenBuildConfig type that contains methods for adding configs
for URLs, as well as both the repo path and revision (for qemu-based patches)

- Write unit tests for these methods

Use these in Repo.Build.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agosystemtest: Delete all existing branches when not making from a clean state
George Dunlap [Fri, 23 Jun 2017 16:38:05 +0000 (17:38 +0100)]
systemtest: Delete all existing branches when not making from a clean state

Except `master`, of course.

Make sure to open all repos before cleaning branches.  (xsa.git will typically not be open
after `repo init` if it wasn't cloned.)

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoRepo: Refuse to execute without a command except for 'git clone'
George Dunlap [Mon, 26 Jun 2017 11:12:55 +0000 (12:12 +0100)]
Repo: Refuse to execute without a command except for 'git clone'

Running with a non-open repo could destroy the wrong data.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agosystemtest: Implement --build
George Dunlap [Fri, 23 Jun 2017 16:34:02 +0000 (17:34 +0100)]
systemtest: Implement --build

Always run 'test-apply'; run 'test-build' afterwards if --build is specified.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agosystemtest: Implement --clean option to start from scratch
George Dunlap [Fri, 23 Jun 2017 16:20:54 +0000 (17:20 +0100)]
systemtest: Implement --clean option to start from scratch

Pass all arguments to systemtest.  Use the `flag` package to read `-clean`.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agosystemtest: Always re-run 'repo init'
George Dunlap [Fri, 23 Jun 2017 16:02:34 +0000 (17:02 +0100)]
systemtest: Always re-run 'repo init'

Always run `repo init`.  If `tmp` already exists, delete `tmp/.xsatool`.

Set a limit that ToolConfig.Load won't go up higher than, so that when
starting the test from 'scratch' we don't get "real" configurations
from any higher-level directoiries by accident.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agosystemtest: Abort the failed rebase
George Dunlap [Fri, 23 Jun 2017 16:00:29 +0000 (17:00 +0100)]
systemtest: Abort the failed rebase

So that subsequent git commands (from another test) work

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agogit: Implement Repo.RebaseAbort
George Dunlap [Fri, 23 Jun 2017 15:59:08 +0000 (16:59 +0100)]
git: Implement Repo.RebaseAbort

Wish there were a better interface, but here we are.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoMiscelaneous fixes
George Dunlap [Fri, 23 Jun 2017 15:56:49 +0000 (16:56 +0100)]
Miscelaneous fixes

Add missing arguments, remove a duplicated stanza

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoRun go fmt
George Dunlap [Fri, 23 Jun 2017 09:36:15 +0000 (10:36 +0100)]
Run go fmt

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoInclude README.md
George Dunlap [Fri, 16 Jun 2017 10:33:03 +0000 (11:33 +0100)]
Include README.md

This is based on the design document posted before, but is updated
with more workflow description, as well as future development plans.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agogit: Delete stackgit data when deleting or resetting branches
George Dunlap [Fri, 5 May 2017 14:33:28 +0000 (15:33 +0100)]
git: Delete stackgit data when deleting or resetting branches

If a developer has modified a branch to be a stackgit branch as part
of development, simply deleting or resetting that branch with git
commands will "work", but will leave leftover state lying around which
may cause problems later.

Before deleting or resetting a branch, check to see if the branch is a
stackgit branch.  If so, use `stg branch --delete --force` to get rid of all
the stackgit data.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoImplement backport
George Dunlap [Wed, 14 Jun 2017 10:06:12 +0000 (11:06 +0100)]
Implement backport

Implement Repo.Rebase() (with full head / base / onto)

Use this to implement Recipe.Backport()

Finally, implement xsatool [nnn] backport:
 - Look for the latest xenversion that doens't have a "patch" branch
 - Rebase next version from that onto it with 'git rebase'.

Always only do one, so that the caller has a chance to test the build
and look it over before continuing the backport process.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoXSAMeta: Keep SupportedVersions sorted in reverse order
George Dunlap [Fri, 16 Jun 2017 10:20:27 +0000 (11:20 +0100)]
XSAMeta: Keep SupportedVersions sorted in reverse order

There are many situations where we want to test things on master first
and then go backwards through supported versions (e.g., building,
looking for backports, &c).  Keep the SupportedVersions in this order.

To do this:
 - Refactor IsGreaterEqualThen to be usable by both it and IsGreaterThan
 - Make a XenVersionSlice type which implements sort.Interface, using the above
 - Re-sort XSAMeta.SupportedVersions on insert and load.

Also:

 - Have XSAMeta.NewRecipe() add the new SupportedVersions, rather
than having the caller add it manuall.

 - Keep XenVersion.PrevVersion() around, even though we're not using
 it for now, in case it becomes useful in the future.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agorecipe: Handle prerequisites
George Dunlap [Fri, 16 Jun 2017 10:14:52 +0000 (11:14 +0100)]
recipe: Handle prerequisites

When making baselines, iterate through prerequisite list and apply all
patches.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoBuild: Use mirror paths for qemu unless there are patches
George Dunlap [Fri, 16 Jun 2017 10:13:40 +0000 (11:13 +0100)]
Build: Use mirror paths for qemu unless there are patches

Cloning from the working tree is running into build problems saying it
can't find the right changeset, even though the changeset is present
in the working tree.  For now, use the mirror directory.

This needs to be sorted out properly somehow.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoUse new branching scheme
George Dunlap [Thu, 8 Jun 2017 16:37:45 +0000 (17:37 +0100)]
Use new branching scheme

New branching scheme:

 - $v-stable based on the recorded stable branch at the time the xsa
   was created

 - $v-baseline; the above, but with 'prerequisites' (outstanding XSAs
   not published yet)

 - $v: Branch with actual patches, only present when patches exist (or
   are being created)

To do this:

 - Separate Recipe.Apply() into ApplyBaseline (which only does -stable
   and -baseline) and ApplyPatches (which makes the main branch)

 - Get rid of IsApplied(), which doesn't do what it says on the tin

 - Implement HasPatchBranch, which detect whether the "patch"
   branch(es) exist for a given recipe

Then:

 - During Init, call only ApplyBaseline for all branches; only call
   ApplyPatches for master.

 - When calling sync-patches with 'all', have VersionsFromArgs filter
   out patches based on HasPatchBranch.  (sync-branches and test will still
   test all supported branches for 'all'.)

 - Add a test for the failure path of sync-branches with a
   non-existent branch

While we're here:

 - Use fmt.Print instead of fmt.Printf when there are no format tokens

 - Add missing '\n' to many status output messages

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agorecipe: Use StableBranch to create VVV-baseline branch
George Dunlap [Thu, 8 Jun 2017 16:23:56 +0000 (17:23 +0100)]
recipe: Use StableBranch to create VVV-baseline branch

Get rid of XenVersionFull from the recipe and replace it with
XenVersion for convenience.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoAdd stable branch reference to TreeRecipe
George Dunlap [Wed, 7 Jun 2017 16:25:46 +0000 (17:25 +0100)]
Add stable branch reference to TreeRecipe

Add XenRepo.MakeStableBranch() and XenRepo.GetRefVersion(), which will
make a "stable branch" and get the ref for the appropriate version.

Check out the 'stable' branch for each version when creating a new
repo, and take a reference.

Also update the testing infrastructure to set up the tree such that
the stable branches reflect the appropriate branches at the time.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoImplement 'repo update'
George Dunlap [Wed, 7 Jun 2017 14:07:49 +0000 (15:07 +0100)]
Implement 'repo update'

Implement Repo.Branches() with a pattern.  Add code to handle empty
output and 'stray' values.

Implement MainRepoUpdate, which will call 'git fetch' and 'git merge'
on master and stable-* branches.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agotesting: Test master build
George Dunlap [Fri, 19 May 2017 17:04:51 +0000 (18:04 +0100)]
testing: Test master build

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agotest: Check for patches in recipe rather than relying on the existence of a branch
George Dunlap [Fri, 19 May 2017 17:03:07 +0000 (18:03 +0100)]
test: Check for patches in recipe rather than relying on the existence of a branch

Implement Recipe.HasPatches(), which returns true if any of the trees
has patches.

Call it in test.go:test() before applying and/or building.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoChange to in-binary test
George Dunlap [Fri, 19 May 2017 16:05:44 +0000 (17:05 +0100)]
Change to in-binary test

`go test` apparently has a default 10-minute timeout.  This isn't
enough for all the tests in the "system test", and there's no way to
change the default except to manually add a --timeout parameter to "go
test".

Instead pull the test out into a special xsatool command.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agotesting: Implement 206 'implement and sync' tests
George Dunlap [Fri, 19 May 2017 14:37:47 +0000 (15:37 +0100)]
testing: Implement 206 'implement and sync' tests

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agomain: Accept an empty set of versions as 'all'
George Dunlap [Fri, 19 May 2017 14:37:06 +0000 (15:37 +0100)]
main: Accept an empty set of versions as 'all'

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoApply: Always check out local master rather than origin/master
George Dunlap [Fri, 19 May 2017 14:35:58 +0000 (15:35 +0100)]
Apply: Always check out local master rather than origin/master

To make testing work

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoTesting: Test xsa 206 init
George Dunlap [Fri, 19 May 2017 14:24:37 +0000 (15:24 +0100)]
Testing: Test xsa 206 init

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agomain: Implement 'init' command
George Dunlap [Fri, 19 May 2017 14:01:17 +0000 (15:01 +0100)]
main: Implement 'init' command

New method of creating a new xsa:  "xsatool NNN init [trees]"

This will:
- For a given set of trees
- And all the currently-supported versions
- Create an XSA that contains all the currently-outstanding XSAs for that tree and set of versions
- And create branches for all of those versions in the correct tree
- And add it to the list of currently-outstanding XSAs

At that point, the user can go into the appropriate tree and create patches.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoRecipe: Allow MakePatches to detect and tolerate "empty" trees
George Dunlap [Wed, 17 May 2017 16:49:02 +0000 (17:49 +0100)]
Recipe: Allow MakePatches to detect and tolerate "empty" trees

In order to move to a world where all supported branches are created
automatically, with prerequisites applied, as "empty" when the XSA is
created, allow Recipe.MakePatches() to detect and tolerate "empty"
versions:

- Have MakePatches() return the number of trees which contained
  patches

- Instead of verifying the reference, get the hashes for base and
  branch and compare them; simply return 0 if they are the same

- Don't bother copying patches if none are to be created.

Modify xsa.go:sync() to handle this case as well.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agogit: Implement ParseRef to change a ref into a changeset
George Dunlap [Wed, 17 May 2017 16:39:55 +0000 (17:39 +0100)]
git: Implement ParseRef to change a ref into a changeset

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agomain: add loadGlobal option
George Dunlap [Wed, 17 May 2017 16:43:39 +0000 (17:43 +0100)]
main: add loadGlobal option

To indicate whether a command should try to load the global metadata.

Signed-of-by: George Dunlap <george.dunlap@citrix.com>
7 years agoglobal init: Always create a 'master' version as well
George Dunlap [Wed, 17 May 2017 16:41:30 +0000 (17:41 +0100)]
global init: Always create a 'master' version as well

This will help is keep track of un-released XSAs.

TODO: Still need a good way to reset this when we've published the
XSAs.

While we're here, fix a typo.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agosystem_test: Check for specific versions, not just presence
George Dunlap [Wed, 17 May 2017 16:51:23 +0000 (17:51 +0100)]
system_test: Check for specific versions, not just presence

Check for the specific versions, not just the presence of the
branches.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoVersionMeta: Make XSA list per-tree
George Dunlap [Wed, 17 May 2017 16:32:07 +0000 (17:32 +0100)]
VersionMeta: Make XSA list per-tree

Not all XSAs apply to each tree; therefore to which XSAs need to be
applied as prerequisites for a given XSA, we need to store them
per-tree.

Change this to a map[Tree][]int.  Create SecurityMeta.InitVersion() to
initialize an empty version properly.

Also:
- explicitly list in an XSA which trees are supported (unused)
- Add SecurityMeta.AddXSA(), which will update VersionMeta based on xsa.Trees

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoXSAMeta: Store recipes in a map[XenVersion] rather than an array
George Dunlap [Wed, 17 May 2017 16:22:22 +0000 (17:22 +0100)]
XSAMeta: Store recipes in a map[XenVersion] rather than an array

This makes accessing it easier, and also makes the json easier for
humans to parse.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoRecipe: Add ForEachTree method to do something for each tree
George Dunlap [Wed, 17 May 2017 15:27:18 +0000 (16:27 +0100)]
Recipe: Add ForEachTree method to do something for each tree

Have MakePatches, IsApplied, and Apply methods use it rather than
ForEachXenRepo().

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agogit: Use full path for AM operations
George Dunlap [Wed, 17 May 2017 15:17:17 +0000 (16:17 +0100)]
git: Use full path for AM operations

...so that Repo.Am() works regardless of your cwd.

Factor "normalization" into normalizePath.

Implement globToFullPath, which will take a glob and return a slice of
normalized filenames.

Use globToFullPath() for Repo.Am() and Repo.AmClean().

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoTesting: Test "xsatool global update" initialization
George Dunlap [Thu, 11 May 2017 12:42:03 +0000 (13:42 +0100)]
Testing: Test "xsatool global update" initialization

Re-constructing the history of XSA 206, go back to version 4.4.

Check to make sure that after the initialization we have exactly the
versions we expect.

Also sketch out a plan for our 'system testing'.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoTesting: Add a testing "input indirection" layer
George Dunlap [Thu, 11 May 2017 12:39:14 +0000 (13:39 +0100)]
Testing: Add a testing "input indirection" layer

If called from main(), print the string and query the user.

Otherwise, look up the response in the index.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoTesting: Add a wrapper for XsaMain so that calling it is less ugly
George Dunlap [Thu, 11 May 2017 12:37:44 +0000 (13:37 +0100)]
Testing: Add a wrapper for XsaMain so that calling it is less ugly

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoglobal: Refer to global SecurityMeta structure, not a local one
George Dunlap [Thu, 11 May 2017 12:34:24 +0000 (13:34 +0100)]
global: Refer to global SecurityMeta structure, not a local one

In part so that the testing harness can see what's going on afterwards.

While we're here add a newline to the "manually add XSAs" message.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoXenRepo: Don't automatically fetch on open to make testing more predictable
George Dunlap [Thu, 11 May 2017 11:36:10 +0000 (12:36 +0100)]
XenRepo: Don't automatically fetch on open to make testing more predictable

This may need to be revisited at some point.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agosystem_test: Add versions and lack of metadata to 'state'
George Dunlap [Wed, 10 May 2017 17:30:16 +0000 (18:30 +0100)]
system_test: Add versions and lack of metadata to 'state'

Delete all versions >= 4.9, and any tags after 4.8.0 or 4.7.2.

Also delete xsa.git/*.meta

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agogit: Add Repo.DeleteTag
George Dunlap [Wed, 10 May 2017 17:28:39 +0000 (18:28 +0100)]
git: Add Repo.DeleteTag

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoTesting: Introduce 'system' test
George Dunlap [Wed, 10 May 2017 16:09:48 +0000 (17:09 +0100)]
Testing: Introduce 'system' test

To begin with, just clone repos and begin to set them up in a specific
state, so that we can test operations.

To make testing iteration faster, set FullClone == false.  After test
development is done, this should be set to "true" to make the cloning
continues to work.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agomain: Call OpenRepos when loading config
George Dunlap [Wed, 10 May 2017 16:03:50 +0000 (17:03 +0100)]
main: Call OpenRepos when loading config

Rather than having to open them each place they're used.

Also do some minor reorganization of variables, just code motion.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agomain: Open xsa repo in OpenRepos
George Dunlap [Wed, 10 May 2017 16:02:20 +0000 (17:02 +0100)]
main: Open xsa repo in OpenRepos

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agogit: Add Repo.ResetBranch
George Dunlap [Wed, 10 May 2017 16:01:14 +0000 (17:01 +0100)]
git: Add Repo.ResetBranch

To check out a particular branch.  This will be used in testing to set
up a particular view of the tree.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoRefactor main to be more test-friendly
George Dunlap [Wed, 10 May 2017 13:27:05 +0000 (14:27 +0100)]
Refactor main to be more test-friendly

Have a special function, XsaMain(), which does the work and is passed
parameters and a return value.  Have a wrapper which interacts with
the operating system under normal conditions.  This allows the testing
infrastructure to simulate being called as closely as possible.

Get rid of global references to 'xsa' entirely; instead pass
references to a local copy on the stack.

Collect all remaining global state into a struct called 'G', and reset
its state every time XsaMain() is called.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoXenRepo: Quieten tagsToVersions
George Dunlap [Wed, 10 May 2017 13:32:42 +0000 (14:32 +0100)]
XenRepo: Quieten tagsToVersions

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agorecipe: Replace Prerequisite glob with xsa list
George Dunlap [Wed, 10 May 2017 13:32:03 +0000 (14:32 +0100)]
recipe: Replace Prerequisite glob with xsa list

Punt on actually implementing applying the XSA until later.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoRun go fmt
George Dunlap [Tue, 9 May 2017 17:05:49 +0000 (18:05 +0100)]
Run go fmt

No functional change.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoAdd global config
George Dunlap [Tue, 9 May 2017 14:51:55 +0000 (15:51 +0100)]
Add global config

Modify global metadata to be a map (for easier readability).

Modify per-version metadata to include most recent full version and
XSAs since that point.

Add 'global update' command which will create a metadata file if not
there, and update it if there.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoFix global.Load() to return IsNotExist() error
George Dunlap [Tue, 9 May 2017 14:48:15 +0000 (15:48 +0100)]
Fix global.Load() to return IsNotExist() error

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoImplement unit tests for various parsing / comparsions
George Dunlap [Tue, 9 May 2017 14:46:00 +0000 (15:46 +0100)]
Implement unit tests for various parsing / comparsions

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agoxen: Implement XenRepo.GetVersions()
George Dunlap [Tue, 9 May 2017 14:45:06 +0000 (15:45 +0100)]
xen: Implement XenRepo.GetVersions()

Have GetVersions() fetch the tags from the repo, parse the releases,
and return a list of versions present in the tree.

Also implement XenVersion.Check() and XenVersion.IsGreaterEqualThan().

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
7 years agogit: Add Tags() method to get tags
George Dunlap [Tue, 9 May 2017 11:29:35 +0000 (12:29 +0100)]
git: Add Tags() method to get tags

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoMove XenVersion and XenVersionFull into xen.go
George Dunlap [Fri, 5 May 2017 16:17:25 +0000 (17:17 +0100)]
Move XenVersion and XenVersionFull into xen.go

Code-motion only (apart from dependency changes)

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoExtend XenVersion parsing: fv.Check() and fv.Point()
George Dunlap [Fri, 5 May 2017 16:14:18 +0000 (17:14 +0100)]
Extend XenVersion parsing: fv.Check() and fv.Point()

Add an internal function to parse XenVersionFull into XenVersion and
Point.  Have .Check(), .Point(), and .XenVersion() be wrappers of
this.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoAdd NNN new-recipe command
George Dunlap [Fri, 28 Apr 2017 10:18:33 +0000 (11:18 +0100)]
Add NNN new-recipe command

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agorepo init: Actually save config file
George Dunlap [Fri, 5 May 2017 14:31:42 +0000 (15:31 +0100)]
repo init: Actually save config file

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agosync-patches: Actually update and write the recipe file
George Dunlap [Fri, 5 May 2017 14:30:40 +0000 (15:30 +0100)]
sync-patches: Actually update and write the recipe file

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoUse a string for Tree rather than an int; make maps instead of arrays
George Dunlap [Fri, 5 May 2017 14:20:30 +0000 (15:20 +0100)]
Use a string for Tree rather than an int; make maps instead of arrays

This is primarily to get better config files, where the indexes are
strings rather than integers or lists.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoA number of miscellaneous fixes
George Dunlap [Fri, 5 May 2017 14:04:58 +0000 (15:04 +0100)]
A number of miscellaneous fixes

* In main(), actually include the xsa number in the printf when the load fails

* Load and save the actual xsa file (xsa.git/xsaNNN.meta) rather than
  the global meta file.  "Normalize" (set the xsa number for recipes) on load.

* Use the correct path for QEMU_TRADITIONAL_URL

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoFix config load
George Dunlap [Fri, 5 May 2017 13:52:26 +0000 (14:52 +0100)]
Fix config load

Have ToolConfig.Load return an error which returns true for
os.IsNotExist().

When the config isn't present, initialize it.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoIntroduce Tree type to avoid hard-coding tree names
George Dunlap [Fri, 28 Apr 2017 10:27:47 +0000 (11:27 +0100)]
Introduce Tree type to avoid hard-coding tree names

Replace repos.{xen,qemuu,qemut}, config.*Path, and Recipes.{Xen,Qemu*}
with arrays indexed by Tree.  Replace Get*Path with GetPath(t).
Replace XRType with Tree.  Introduce ForEachCodeTree, ForEachTree, and
ForEachXenRepo helper functions to run a specific function for each
tree / repo; rewrite lots of code to use those functions instead of
doing code duplication.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoAdd XSA.Load() and Save(), and use it rather than hardcoding
George Dunlap [Fri, 28 Apr 2017 10:30:03 +0000 (11:30 +0100)]
Add XSA.Load() and Save(), and use it rather than hardcoding

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoDo a bit more towards global security metadata
George Dunlap [Fri, 31 Mar 2017 16:02:57 +0000 (17:02 +0100)]
Do a bit more towards global security metadata

Still need to sort out the Time serialization issue; leave a comment
to this effect.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoLoad xsatool config from a file
George Dunlap [Fri, 31 Mar 2017 14:28:42 +0000 (15:28 +0100)]
Load xsatool config from a file

Recursively look for ".xsatool" in all directories going up until
there are no more.

Add 'repo info' command to read config and print info

Also:

 - Use config.GetXXXPath() to get normalized path rather than
   accessing config.XXXPath directly

 - Change config.GetXXXPath() to return NULL if either config.rootDir
   or config.XXXPath are undefined

 - Separate out XSA fake loading into a separate flag

 - use xenproject.org rather than xen.org for default configuration

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agohack: Include hack recipes for both 206 and 212
George Dunlap [Fri, 31 Mar 2017 11:10:07 +0000 (12:10 +0100)]
hack: Include hack recipes for both 206 and 212

NB that 212 requires you to make an e-mail-able version.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoImplement test[-apply|-build]
George Dunlap [Fri, 24 Mar 2017 17:50:01 +0000 (17:50 +0000)]
Implement test[-apply|-build]

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoFinish sync-patch, add sync-branch as well
George Dunlap [Fri, 24 Mar 2017 17:44:50 +0000 (17:44 +0000)]
Finish sync-patch, add sync-branch as well

Have MakePatches include a flag deciding whether to sync or not.

If set, after creating all appropriate patches in /tmp, make the
appropriate directory/ies in xsa.git and copy the files there (or just
copy a single patch if the file is a single patch).

Also implement sync-branch which will delete and re-create xsa/NNN/VVV
from xsa.git according to the recipe.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agomain: Add partial implementation of sync-patches command
George Dunlap [Fri, 24 Mar 2017 11:07:16 +0000 (11:07 +0000)]
main: Add partial implementation of sync-patches command

Partial implementation of sync-patches, which will look for trees
which contain xsa/NNN/VVV and xsa/NNN/VVV-baseline and run
`git format-patch` to generate patches.

To do this:
 - Implement Repo.FormatPatch()
 - Implement XSAMeta.GetRecipe(version) to get a recipe for a specific version
 - Modify IsApplied() to look for both $prefix/NNN/VVV and $prefix/NNN/VVV-baseline
 - Implement Recipe.MakePatches()
 - Modify IsApplied() to return only an error; but make return a
   distinguishable type to say what branch wasn't applied

A full implementation would also copy the patches into xsa.git either
as a single patch or a directory full of patches, as appropriate.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agorecipe: Have branchName return both branch and baseline branch names
George Dunlap [Fri, 24 Mar 2017 10:54:12 +0000 (10:54 +0000)]
recipe: Have branchName return both branch and baseline branch names

...to make sure we have consistent naming.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoMain: Fake up config and xsa info
George Dunlap [Thu, 23 Mar 2017 13:52:02 +0000 (13:52 +0000)]
Main: Fake up config and xsa info

"Fake up" config and XSA info in Main(), so that we can start writing
functionality as though the config loading already existed, rather
than have to store it all in test.go.

This should make it easier to extend functionality as needed in a way
which will actually be useful once the config loading stuff is ready.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoRecipe: Break down per-tree recipe into prerequisites and patches
George Dunlap [Thu, 23 Mar 2017 11:38:50 +0000 (11:38 +0000)]
Recipe: Break down per-tree recipe into prerequisites and patches

Create a new struct, TreeRecipe, to be the recipe for a particular
tree (Xen, QemuU or QemuT).  Have this contain separate prereq and
patches sections.

When building, create a branch $branch-baseline to apply the Prereq
patches to, and then base $branch based on the baseline.

The idea here is to make an easy way to programmatically sync changes
to the tree back to xsa.git.

Also unconditionally use local qemu trees, even if we don't have local
patches, because we know they'll be up to date with upstream changes
(since we pull at start-of-day).

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agotest: Make separate backport and test frameworks
George Dunlap [Thu, 23 Mar 2017 10:47:53 +0000 (10:47 +0000)]
test: Make separate backport and test frameworks

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoRecipe: Make more flexible for both backporting and testing
George Dunlap [Thu, 23 Mar 2017 10:46:43 +0000 (10:46 +0000)]
Recipe: Make more flexible for both backporting and testing

Make the interface a bit more flexible, so it can be used both for
backporting (which will create branches named xsa/NNN/VVV), and for
testing (which will create branches named test/NNN/VVV).

Add branchName() helper to calculate branch name, rather than storing
a single tempBranch.

Add IsApplied() method, which will (at the moment) only check for the
existence of the target branches in the appropriate trees.

Also:

- Use AmClean() rather than Am() to make it easier to follow up a
  failure by cherry-picking

- Fix a bug in Build() which would build the wrong version

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoRecipe: Don't clean up temporary branches automatically (for now)
George Dunlap [Thu, 23 Mar 2017 10:39:27 +0000 (10:39 +0000)]
Recipe: Don't clean up temporary branches automatically (for now)

When doing testing, we probably want to remove temporary branches; but
when doing backporting we most certainly do not.  This is something
which still needs to be figured out.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agogit: Implement VerifyRef and AmClean
George Dunlap [Thu, 23 Mar 2017 10:30:33 +0000 (10:30 +0000)]
git: Implement VerifyRef and AmClean

VerifyRef returns true if the specified ref exists.

AmClean() is like Am(), but will reset the tree to a useable state if
the Am fails.  This is useful for situations like backporting where
you typically want to cherry-pick on a failed `git am` to take
advantage of the three-way merge functionality.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoMake XenVersion string-only
George Dunlap [Fri, 17 Mar 2017 10:11:18 +0000 (10:11 +0000)]
Make XenVersion string-only

Hopefully this will make it easier to store in a human-readable
format.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoAllow Recipes to have multiple globs, implement Recipe.Build()
George Dunlap [Thu, 16 Mar 2017 17:15:15 +0000 (17:15 +0000)]
Allow Recipes to have multiple globs, implement Recipe.Build()

Allow each recipe to have multiple "globs" associated with each tree.

Implement Recipe.Build(), which will construct the appropriate config
file to make the build use the appropriate branches, and then
configure and run make.

While we're here fix some mistaken status messages in Recipe.Cleanup(), and
fix XenRepo.CheckoutVersion when XenVersion == Master.

And update the "test" to XSA211.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoGit: Misc functionality and fixes
George Dunlap [Thu, 16 Mar 2017 12:39:31 +0000 (12:39 +0000)]
Git: Misc functionality and fixes

Have Repo.Am() check to make sure that the glob matched at least one
file; if not, throw an error.

Do "git clean -ffdx" after checkout in MakeBranch and Checkout.

Add GetPath to return the full path of a repository.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoRefactor arguments to make "repo" target
George Dunlap [Wed, 1 Mar 2017 17:36:40 +0000 (17:36 +0000)]
Refactor arguments to make "repo" target

Rather than "repoinit" as a single argument, make the first argument
the target (either an xsa or the repo), and separate commands based on
that.

Also fix a bug that causes a range check if there aren't enough
arguments.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoImplement basic repoinit
George Dunlap [Fri, 24 Feb 2017 16:36:48 +0000 (16:36 +0000)]
Implement basic repoinit

Clone xen.git, qemu-xen.git, qemu-xen-traditional.git, and xsa.git,
and set up a default .xsatool config file.  Disable pushes on the xen
and qemu branches.

Modify gitCmd() to not add the -C prefix if r.Path isn't set.

Implement GitClone and XenRepoClone, and Repo.DisablePush.

Move global repo s into a global structure called 'repo'; and also
include a plain Repo for xsa.

Implement ToolConfig type, with Load and Save.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoAdd draft infrastructure for different commands
George Dunlap [Fri, 24 Feb 2017 14:32:58 +0000 (14:32 +0000)]
Add draft infrastructure for different commands

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
8 years agoAdd support for all repos, and test different versions.
George Dunlap [Fri, 24 Feb 2017 13:45:44 +0000 (13:45 +0000)]
Add support for all repos, and test different versions.

Also rename the structure to XSAMeta, in preparation for adding other
metadata.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>