From: Ian Jackson <ian.jackson@eu.citrix.com>
Date: Fri, 26 Sep 2008 14:40:11 +0000 (+0100)
Subject: vnc and xenfb integer overflow and division by zero vuln fixes
X-Git-Tag: t.master-before-merge~80
X-Git-Url: http://xenbits.xensource.com/gitweb?a=commitdiff_plain;h=a83c1174b942d0f0f0e05927eb5b69fe8489b7ab;p=qemu-xen-4.2-testing.git

vnc and xenfb integer overflow and division by zero vuln fixes

row_stride_div0.patch: a malicious frontend can send row_stride==0 and force
qemu-dm to perform division by 0
vnc_resize_doublecheck.patch: there is an unchecked multiplication when
calculating framebuffer size. Cs 17630 sanitizes framebuffer dimensions
passed by the frontend, so most probably no integer overflow can happen, but
there should be a check for overflow close to the actual computation (to
make code review easier and to cope with other codepaths in the future).

(Patches submitted by Rafal Wojtczuk <rafal@invisiblethingslab.com>
against xen-3.2 ioemu; adapted for xen-unstable by Ian Jackson and also
edited to actually compile and do be correct.)

Contributed-by: Rafal Wojtczuk <rafal@invisiblethingslab.com>
Modified-by: Ian Jackson <ian.jackson@eu.citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---

diff --git a/hw/xenfb.c b/hw/xenfb.c
index 41ede97c5..f235658bb 100644
--- a/hw/xenfb.c
+++ b/hw/xenfb.c
@@ -511,7 +511,7 @@ static int xenfb_configure_fb(struct xenfb *xenfb, size_t fb_len_lim,
 			depth);
 		return -1;
 	}
-	if (row_stride < 0 || row_stride > fb_len) {
+	if (row_stride <= 0 || row_stride > fb_len) {
 		fprintf(stderr,
 			"FB: invalid frontend stride %d\n", row_stride);
 		return -1;
diff --git a/vnc.c b/vnc.c
index 2cf3a31af..476d4ce97 100644
--- a/vnc.c
+++ b/vnc.c
@@ -366,6 +366,13 @@ static void vnc_framebuffer_update(VncState *vs, int x, int y, int w, int h,
     vnc_write_s32(vs, encoding);
 }
 
+static int mult_overflows(int x, int y)
+{
+    if (x<=0 || y<=0 || x>=INT_MAX/y)
+        return 1;
+    else return 0;
+}
+
 static void vnc_dpy_resize_shared(DisplayState *ds, int w, int h, int depth, int linesize, void *pixels)
 {
     static int allocated;
@@ -373,6 +380,11 @@ static void vnc_dpy_resize_shared(DisplayState *ds, int w, int h, int depth, int
     VncState *vs = ds->opaque;
     int o;
 
+    if (mult_overflows(w, h) || mult_overflows(w*h, vs->depth) ||
+        mult_overflows(h, sizeof(vs->dirty_row[0]))) {
+        fprintf(stderr, "vnc: suspicious vnc_dpy_resize arguments, exiting\n");
+        exit(1);
+    }
     vnc_colourdepth(ds, depth);
     if (!ds->shared_buf) {
         ds->linesize = w * vs->depth;