po: minimize & canonicalize translations stored in git
Similar to the libvirt.pot, .po files contain line numbers and file
names identifying where in the source a translatable string comes from.
The source locations in the .po files are thrown away and replaced with
content from the libvirt.pot whenever msgmerge is run, so this is not
precious information that needs to be stored in git.
When msgmerge processes a .po file, it will add in any msgids from the
libvirt.pot that were not already present. Thus, if a particular msgid
currently has no translation, it can be considered redundant and again
does not need storing in git.
When msgmerge processes a .po file and can't find an exact existing
translation match, it will try todo fuzzy matching instead, marking such
entries with a "# fuzzy" comment to alert the translator to take a
look and either discard, edit or accept the match. Looking at the
existing fuzzy matches in .po files shows that the quality is awful,
with many having a completely different set of printf format specifiers
between the msgid and fuzzy msgstr entry. Fortunately when msgfmt
generates the .gmo, the fuzzy entries are all ignored anyway. The fuzzy
entries could be useful to translators if they were working on the .po
files directly from git, but Libvirt outsourced translation to the
Fedora Zanata system, so keeping fuzzy matches in git is not much help.
Finally, by default msgids are sorted based on source location. Thus, if
a bit of code with translatable text is moved from one file to another,
it may shift around in the .po file, despite the msgid not itself changing.
If the msgids were sorted alphabetically, the .po files would have
stable ordering when code is refactored.
This patch takes advantage of the above observations to canonicalize
and minimize the content stored for .po files in git. Instead of storing
the real .po files, we now store .mini.po files.
The .mini.po files are the same file format as .po files, but have no
source location comments, are sorted alphabetically, and all fuzzy
msgstrs and msgids with no translation are discarded. This cuts the size
of content in the po directory from 109MB to 19MB.
Users working from a libvirt git checkout who need the full .po files
can run "make update-po", which merges the libvirt.pot and .mini.po
file to create a .po file containing all the content previously stored
in git.
Conversely if a full .po file has been modified, for example, by
downloading new content from Zanata, the .mini.po files can be updated
by running "make update-mini-po". The resulting diffs of the .mini.po
file will clearly show the changed translations without any of the noise
that previously obscured content. Being able to see content changes
clearly actually identified a bug in the zanata python client where it
was adding bogus "fuzzy" annotations to many messages:
Users working from libvirt releases should not see any difference in
behaviour, since the tarballs only contain the full .po files, not the
.mini.po files.
As an added benefit, generating tarballs with "make dist", will no
longer cause creation of dirty files in git, since it won't touch the
.mini.po files, only the .po files which are no longer kept in git.
To avoid creating a single commit 100+MB in size, each language is
minimized separately in a following commit.
Reviewed-by: Ján Tomko <jtomko@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Storing the libvirt.pot file is a bad idea because it is an
automatically generated file. Most patches will invalidate the stored
libvirt.pot file by changing line numbers or introducing/removing files
with translatable content.
Anyone working with a libvirt GIT checkout who needs the libvirt.pot is
better served creating a fresh copy with "make libvirt.pot".
libvirt.pot is still included in the release dists, so those building
from tarballs see no change in behaviour.
Reviewed-by: Ján Tomko <jtomko@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
po: provide custom make rules for po file management
Historically we have relied on autopoint/gettextize to install a
standard po/Makefile.in.in. There is very limited scope for customizing
this and it also causes a bunch of extra stuff to be pulled into
configure.ac which potentially clashes with gnulib. Writing make rules
for po file management is no more difficult than any other rules libvirt
has, so stop using autopoint/gettextize.
Reviewed-by: Ján Tomko <jtomko@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Peter Krempa [Wed, 4 Apr 2018 07:44:34 +0000 (09:44 +0200)]
qemu: Format 'write-cache' parameter for disk frontends
The disk cache mode translates to various frontend and backend
attributes for the qemu block layer. For the frontend device the
'writeback' parameter is used and provided as 'write-cache'. Implement
this so that we can later switch to using -blockdev where we will not
pass the cachemode directly any more.
Peter Krempa [Tue, 3 Apr 2018 17:11:22 +0000 (19:11 +0200)]
qemu: caps: Add capability for 'write-cache' parameter of disk frontends
QEMU translates the cache mode of a disk internally into 3 flags.
'write-cache' is a flag of the frontend while others are flag of the
backing storage. Add capability which will allow expressing it via the
frontend attribute.
Peter Krempa [Wed, 4 Apr 2018 07:46:50 +0000 (09:46 +0200)]
tests: qemuxml2argv: Test formatting of 'write-cache' parameter
Prepare the tests for adding the new parameter. The parameter was
introduced in qemu-2.7.0, so add a forked version of the test case to
see that it is formatted properly.
This test is also an example how the new testing macros should be used.
Peter Krempa [Wed, 4 Apr 2018 07:42:36 +0000 (09:42 +0200)]
tests: qemuxml2argv: Add infrastructure for testing with real qemuCaps
Allow testing of XML->argv conversion with using a real capability map
as used in the qemucapabilitiestest. This allows specifying the required
qemu version with the test rather than having to enumerate all the
required capabilities or allows to use the newest capabilities present.
Peter Krempa [Tue, 17 Apr 2018 14:17:53 +0000 (16:17 +0200)]
tests: qemuxml2argv: Add infrastructure to pass output file suffix
To allow having more than one output file in the qemuxml2argvtest add a
suffix member to the testInfo struct which will allow testing the same
XML file with multiple capabilities files.
Peter Krempa [Wed, 18 Apr 2018 08:47:31 +0000 (10:47 +0200)]
tests: qemu: Add helper code to lookup latest capability file
The helper iterates the directory with files for the capability test and
looks up the most recent one for the given architecture. This will allow
testing against the newest qemu capabilities so that we can catch
regressions in behaviour more easily.
Broken by [1] commit - trailing comma instead of semicolon. Fortunately
the issue did not get sneak in released 4.2 version. Note that uriSchemes
for parallelsConnectDriver should not be allocated on stack.
[1] 8e4f9a27: "driver: declare supported URI schemes in virConnectDriver struct"
Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Peter Krempa [Tue, 10 Apr 2018 15:05:20 +0000 (17:05 +0200)]
conf: Disk 'shared' state is not guest ABI
Drop the checking of 'shared' from the ABI stability check. This
property controls whether the hypervisor allows concurrent access to the
same file, but this fact does not influence guest ABI.
Peter Krempa [Tue, 10 Apr 2018 14:57:22 +0000 (16:57 +0200)]
conf: Move 'driverName' back to disk definition structure
Currently it is not used in backing chains and does not seem that we
will need to use it so return it back to the disk definition. Thankfully
most accesses are done via the accessors.
Peter Krempa [Tue, 10 Apr 2018 14:51:47 +0000 (16:51 +0200)]
conf, qemu: Replace access to disk driver name with accessors
Replace direct usage of disk->src->driverName with the existing
accessors. The parser code where we assign the driver from XML is
intentionally not fixed to save an allocation.
Peter Krempa [Tue, 10 Apr 2018 14:38:22 +0000 (16:38 +0200)]
qemu: Move disk driver name validation into disk validation callback
There were two places where we'd check this independently. Move it to
the disk definition validation callback. This also fixes possible use of
NULL in a printf for network storage.
The problem is in full CPU calculation in virQEMUCapsInitHostCPUModel.
It is supposed to include features emulated by qemu and missed on host. Some of
such features may be not included however.
For Core i5 650 host CPU is detected as Westmere and reported CPU as
SandyBridge. x2apic is missed on host and provided by installed qemu. The
feature is not mentioned in reported CPU features explicitly because SandyBridge
model include it. As a result full CPU does not include x2apic too.
Solution is to expand guest cpu features before updating fullCPU features.
[1] error: the CPU is incompatible with host CPU: \
Host CPU does not provide required features: x2apic
Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com> Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
Michal Privoznik [Fri, 13 Apr 2018 14:41:17 +0000 (16:41 +0200)]
cfg.mk: Introduce syntax-check rule to prefer VIR_CLASS_NEW
Now that we have macro that does some checks lets forbid raw
usage of virClassNew() in favor of VIR_CLASS_NEW().
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Michal Privoznik [Tue, 17 Apr 2018 15:52:03 +0000 (17:52 +0200)]
virobject: Check if @parent is the first member in class
Our virObject code relies heavily on the fact that the first
member of the class struct is type of virObject (or some
derivation of if). Let's check for that.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Michal Privoznik [Tue, 17 Apr 2018 15:42:33 +0000 (17:42 +0200)]
virobject: Introduce VIR_CLASS_NEW() macro
So far we are repeating the following lines over and over:
if (!(virSomeObjectClass = virClassNew(virClassForObject(),
"virSomeObject",
sizeof(virSomeObject),
virSomeObjectDispose)))
return -1;
While this works, it is impossible to do some checking. Firstly,
the class name (the 2nd argument) doesn't match the name in the
code in all cases (the 3rd argument). Secondly, the current style
is needlessly verbose. This commit turns example into following:
if (!(VIR_CLASS_NEW(virSomeObject,
virClassForObject)))
return -1;
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Michal Privoznik [Fri, 13 Apr 2018 11:51:23 +0000 (13:51 +0200)]
src: Unify virObject member name
Whenever we declare a new object the first member of the struct
has to be virObject (or any other member of that family). Now, up
until now we did not care about the name of the struct member.
But lets unify it so that we can do some checks at compile time
later.
The unified name is 'parent'.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Michal Privoznik [Fri, 13 Apr 2018 11:42:16 +0000 (13:42 +0200)]
datatypes: Rename @parent to @parentName in virNodeDevice
In next patches this name will be needed for a different memeber.
Also, it makes sense to rename the variable because it does not
contain reference to parent device, just its name.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Michal Privoznik [Wed, 18 Apr 2018 07:37:09 +0000 (09:37 +0200)]
virmocklibxl.c: Fix include of virfile.h
Firstly, this isn't supposed to be in angle brackets because it's
not a system header file (not that gcc distinguishes these two,
it's just guide for us developers).
Secondly, no need to use util/ path prefix because CFLAGS already
contain -I$(srcdir)/util.
The include is needed because virmocklibxl is mocking
virFileMakePath() and even when VIR_MOCK_IMPL_RET_ARGS() does
forward declaration to shut up compiler we need real
virFileMakePath() declaration so that when it changes compiler
catches mismatching declarations.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
tests: add test case for CPUID in xenconfig driver
Check conversion of "cpuid" setting, check all supported policy settings
("1", "0", "x"). Also, check interaction with "nestedhvm" - should not
be included as "vmx=1" in "cpuid" setting.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> Reviewed-by: Jim Fehlig <jfehlig@suse.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
tests: check CPU features handling in libxl driver
Test enabling/disabling individual CPU features and also setting
nested HVM support, which is also controlled by CPU features node.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> Reviewed-by: Jim Fehlig <jfehlig@suse.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Convert CPU features policy into libxl cpuid policy settings. Use new
("libxl") syntax, which allow to enable/disable specific bits, using
host CPU as a base. For this reason, only "host-passthrough" mode is
accepted.
Libxl do not have distinction between "force" and "required" policy
(there is only "force") and also between "forbid" and "disable" (there
is only "disable"). So, merge them appropriately. If anything, "require"
and "forbid" should be enforced outside of specific driver.
Nested HVM (vmx and svm features) is handled separately, so exclude it
from translation.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
libxl: do not enable nested HVM unless global nested_hvm option enabled
Introduce global libxl option for enabling nested HVM feature, similar
to kvm module parameter. This will prevent enabling experimental feature
by mere presence of <cpu mode='host-passthrough'> element in domain
config, unless explicitly enabled. <cpu mode='host-passthrough'> element
may be used to configure other features, like NUMA, or CPUID.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Jim Fehlig <jfehlig@suse.com>
libxl: pass driver config to libxlMakeDomBuildInfo
Preparation for global nestedhvm configuration - libxlMakeDomBuildInfo
needs access to libxlDriverConfig.
No functional change.
Adjusting tests require slightly more mockup functions, because of
libxlDriverConfigNew() call.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Jim Fehlig <jfehlig@suse.com>
libxl: fix libxlDriverConfigDispose for partially constructed object
libxlDriverConfigNew() use libxlDriverConfigDispose() for cleanup in
case of errors. Do not call libxlLoggerFree() on not allocated logger
(NULL).
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> Reviewed-by: Jim Fehlig <jfehlig@suse.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The vmware driver wants to execute vmware-vmx from the same directory in
which vmrun was found. However, on VMware Fusion 10 vmrun at
/Applications/VMware Fusion.app/Contents/Public/vmrun is a symlink
pointing to ../Library/vmrun. vmware-vmx cannot be found, as
it is not in PATH, but only in this Library directory.
Therefore, follow the vmrun symlink and use the resulting path. Then the
assumption that vmware-vmx is right next to it will still work.
qemu: refresh vcpu halted state only via query-cpus-fast
In order to not affect running VMs, refreshing the halted state
is only performed if QEMU supports the query-cpus-fast QAPI.
Signed-off-by: Viktor Mihajlovski <mihajlov@linux.vnet.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.vnet.ibm.com> Reviewed-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
Extract architecture specific data from query-cpus[-fast] if
available. A new function qemuMonitorJSONExtractCPUArchInfo()
can then call architecture-specific extraction handlers.
Initially, there's a handler for s390 cpu info to
set the halted property depending on the s390 cpu state
returned by QEMU. With this it's still possible to report
the halted condition even when using query-cpus-fast.
Signed-off-by: Viktor Mihajlovski <mihajlov@linux.vnet.ibm.com> Reviewed-by: John Ferlan <jferlan@redhat.com>
projects / libvirt.git / log