From: Daniel P. Berrangé Date: Mon, 5 Feb 2018 11:49:35 +0000 (+0000) Subject: ui: avoid risk of 32-bit int overflow in VNC buffer check X-Git-Url: http://xenbits.xensource.com/gitweb?a=commitdiff_plain;h=dffa1de071aa956308172170107b7b60d99bf34b;p=people%2Fpauldu%2Fqemu.git ui: avoid risk of 32-bit int overflow in VNC buffer check For very large framebuffers, it is theoretically possible for the result of 'vs->throttle_output_offset * VNC_THROTTLE_OUTPUT_LIMIT_SCALE' to exceed the size of a 32-bit int. For this to happen in practice, the video RAM would have to be set to a large enough value, which is not likely today. None the less we can be paranoid against future growth by using division instead of multiplication when checking the limits. Reported-by: Laszlo Ersek Signed-off-by: Daniel P. Berrangé Reviewed-by: Laszlo Ersek Reviewed-by: Philippe Mathieu-Daudé Message-id: 20180205114938.15784-2-berrange@redhat.com Signed-off-by: Gerd Hoffmann --- diff --git a/ui/vnc.c b/ui/vnc.c index e371710f4f..746293ddfa 100644 --- a/ui/vnc.c +++ b/ui/vnc.c @@ -1579,8 +1579,8 @@ void vnc_write(VncState *vs, const void *data, size_t len) * handshake, or from the job thread's VncState clone */ if (vs->throttle_output_offset != 0 && - vs->output.offset > (vs->throttle_output_offset * - VNC_THROTTLE_OUTPUT_LIMIT_SCALE)) { + (vs->output.offset / VNC_THROTTLE_OUTPUT_LIMIT_SCALE) > + vs->throttle_output_offset) { trace_vnc_client_output_limit(vs, vs->ioc, vs->output.offset, vs->throttle_output_offset); vnc_disconnect_start(vs);