vga allows banked access to video memory using the window at 0xa00000
and it supports a different access modes with different address
calculations.
The VBE bochs extentions support banked access too, using the
VBE_DISPI_INDEX_BANK register. The code tries to take the different
address calculations into account and applies different limits to
VBE_DISPI_INDEX_BANK depending on the current access mode.
Which is probably effective in stopping misprogramming by accident.
But from a security point of view completely useless as an attacker
can easily change access modes after setting the bank register.
Drop the bogus check, add range checks to vga_mem_{readb,writeb}
instead.
Fixes: CVE-2016-3710 Reported-by: Qinghao Tang <luodalongde@gmail.com> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
[Backport to qemu-xen-tradition] Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>