From: Ian Jackson <ian.jackson@eu.citrix.com>
Date: Wed, 1 Oct 2008 11:12:57 +0000 (+0100)
Subject: vnc integer overflow check fix overzealous zero checking
X-Git-Tag: t.master-before-merge~79
X-Git-Url: http://xenbits.xensource.com/gitweb?a=commitdiff_plain;h=81b31c9f37ac4e3584bdfe8e7b04bedcb8940b88;p=qemu-xen-4.3-testing.git

vnc integer overflow check fix overzealous zero checking

In a83c1174b942d0f0f0e05927eb5b69fe8489b7ab, we arranged to avoid
integer overflow and calls to realloc(nonzero,0).  However
vs->depth==0 is legitimate on entry to vnc_dpy_resize_shared.

We need to move the check for vs->depth until after vnc_colourdepth.
---

diff --git a/vnc.c b/vnc.c
index 476d4ce97..01e22e544 100644
--- a/vnc.c
+++ b/vnc.c
@@ -380,12 +380,14 @@ static void vnc_dpy_resize_shared(DisplayState *ds, int w, int h, int depth, int
     VncState *vs = ds->opaque;
     int o;
 
+    vnc_colourdepth(ds, depth);
     if (mult_overflows(w, h) || mult_overflows(w*h, vs->depth) ||
         mult_overflows(h, sizeof(vs->dirty_row[0]))) {
-        fprintf(stderr, "vnc: suspicious vnc_dpy_resize arguments, exiting\n");
+        fprintf(stderr, "vnc: suspicious vnc_dpy_resize arguments"
+		" (w=%d h=%d depth=%d linesize=%d vs->depth=%d), exiting\n",
+		w, h, depth, linesize, vs->depth);
         exit(1);
     }
-    vnc_colourdepth(ds, depth);
     if (!ds->shared_buf) {
         ds->linesize = w * vs->depth;
         if (allocated)