ia64/xen-unstable

view docs/HOWTOs/XenDebugger-HOWTO @ 2006:bd310c8b4b5c

bitkeeper revision 1.1108.43.1 (410a5973b_ww-XNociMt5BotV87vBQ)

Tweaks.
author mwilli2@equilibrium.research.intel-research.net
date Fri Jul 30 14:21:39 2004 +0000 (2004-07-30)
parents a825d179881d
children 30bc4c5fe838
line source
1 Pervasive Debugging
2 ===================
4 Alex Ho (alex.ho at cl.cam.ac.uk)
6 Introduction
7 ------------
9 The pervasive debugging project is leveraging Xen to
10 debug distributed systems. We have added a gdb stub
11 to Xen to allow for remote debugging of both Xen and
12 guest operating systems. More information about the
13 pervasive debugger is available at: http://www.cl.cam.ac.uk/netos/pdb
16 Implementation
17 --------------
19 The gdb stub communicates with gdb running over a serial line.
20 The main entry point is pdb_handle_exception() which is invoked
21 from: pdb_key_pressed() ('D' on the console)
22 do_int3_exception() (interrupt 3: breakpoint exception)
23 do_debug() (interrupt 1: debug exception)
25 This accepts characters from the serial port and passes gdb
26 commands to pdb_process_command() which implements the gdb stub
27 interface. This file draws heavily from the kgdb project and
28 sample gdbstub provided with gdb.
30 The stub can examine registers, single step and continue, and
31 read and write memory (in Xen, a domain, or a Linux process'
32 address space). The debugger does not currently trace the
33 current process, so all bets are off if context switch occurs
34 in the domain.
37 Setup
38 -----
40 +-------+ telnet +-----------+ serial +-------+
41 | GDB |--------| nsplitd |--------| Xen |
42 +-------+ +-----------+ +-------+
44 To run pdb, Xen must be appropriately configured and
45 a suitable serial interface attached to the target machine.
46 GDB and nsplitd can run on the same machine.
48 Xen Configuration
50 Add the "pdb=xxx" option to your Xen boot command line
51 where xxx is one of the following values:
52 com1 gdb stub should communicate on com1
53 com1H gdb stub should communicate on com1 (with high bit set)
54 com2 gdb stub should communicate on com2
55 com2H gdb stub should communicate on com2 (with high bit set)
57 Symbolic debugging infomration is quite helpful too:
58 xeno.bk/xen/arch/i386/Rules.mk
59 add -g to CFLAGS to compile Xen with symbols
60 xeno.bk/xenolinux-2.4.24-sparse/arch/xen/Makefile
61 add -g to CFLAGS to compile Linux with symbols
63 You may also want to consider dedicating a register to the
64 frame pointer (disable the -fomit-frame-pointer compile flag).
66 When booting Xen and domain 0, look for the console text
67 "Initializing pervasive debugger (PDB)" just before DOM0 starts up.
69 Serial Port Configuration
71 pdb expects to communicate with gdb using the serial port. Since
72 this port is often shared with the machine's console output, pdb can
73 discriminate its communication by setting the high bit of each byte.
75 A new tool has been added to the source tree which splits
76 the serial output from a remote machine into two streams:
77 one stream (without the high bit) is the console and
78 one stream (with the high bit stripped) is the pdb communication.
80 See: xeno.bk/tools/misc/nsplitd
82 nsplitd configuration
83 ---------------------
84 hostname$ more /etc/xinetd.d/nsplit
85 service nsplit1
86 {
87 socket_type = stream
88 protocol = tcp
89 wait = no
90 user = wanda
91 server = /usr/sbin/in.nsplitd
92 server_args = serial.cl.cam.ac.uk:wcons00
93 disable = no
94 only_from = 128.232.0.0/17 127.0.0.1
95 }
97 hostname$ egrep 'wcons00|nsplit1' /etc/services
98 wcons00 9600/tcp # Wanda remote console
99 nsplit1 12010/tcp # Nemesis console splitter ports.
101 Note: nsplitd was originally written for the Nemesis project
102 at Cambridge.
104 After nsplitd accepts a connection on <port> (12010 in the above
105 example), it starts listening on port <port + 1>. Characters sent
106 to the <port + 1> will have the high bit set and vice versa for
107 characters received.
109 You can connect to the nsplitd using
110 'tools/misc/xencons <host> <port>'
112 GDB 6.0
113 pdb has been tested with gdb 6.0. It should also work with
114 earlier versions.
117 Usage
118 -----
120 1. Boot Xen and Linux
121 2. Interrupt Xen by pressing 'D' at the console
122 You should see the console message:
123 (XEN) pdb_handle_exception [0x88][0x101000:0xfc5e72ac]
124 At this point Xen is frozen and the pdb stub is waiting for gdb commands
125 on the serial line.
126 3. Attach with gdb
127 (gdb) file xeno.bk/xen/xen
128 Reading symbols from xeno.bk/xen/xen...done.
129 (gdb) target remote <hostname>:<port + 1> /* contact nsplitd */
130 Remote debugging using serial.srg:12131
131 continue_cpu_idle_loop () at current.h:10
132 warning: shared library handler failed to enable breakpoint
133 (gdb) break __enter_scheduler
134 Breakpoint 1 at 0xfc510a94: file schedule.c, line 330.
135 (gdb) cont
136 Continuing.
138 Program received signal SIGTRAP, Trace/breakpoint trap.
139 __enter_scheduler () at schedule.c:330
140 (gdb) step
141 (gdb) step
142 (gdb) print next /* the variable prev has been optimized away! */
143 $1 = (struct task_struct *) 0x0
144 (gdb) delete
145 Delete all breakpoints? (y or n) y
146 4. You can add additional symbols to gdb
147 (gdb) add-sym xenolinux-2.4.24/vmlinux
148 add symbol table from file "xenolinux-2.4.24/vmlinux" at
149 (y or n) y
150 Reading symbols from xenolinux-2.4.24/vmlinux...done.
151 (gdb) x/s cpu_vendor_names[0]
152 0xc01530d2 <cpdext+62898>: "Intel"
153 (gdb) break free_uid
154 Breakpoint 2 at 0xc0012250
155 (gdb) cont
156 Continuing. /* run a command in domain 0 */
158 Program received signal SIGTRAP, Trace/breakpoint trap.
159 free_uid (up=0xbffff738) at user.c:77
161 (gdb) print *up
162 $2 = {__count = {counter = 0}, processes = {counter = 135190120}, files = {
163 counter = 0}, next = 0x395, pprev = 0xbffff878, uid = 134701041}
164 (gdb) finish
165 Run till exit from #0 free_uid (up=0xbffff738) at user.c:77
167 Program received signal SIGTRAP, Trace/breakpoint trap.
168 release_task (p=0xc2da0000) at exit.c:51
169 (gdb) print *p
170 $3 = {state = 4, flags = 4, sigpending = 0, addr_limit = {seg = 3221225472},
171 exec_domain = 0xc016a040, need_resched = 0, ptrace = 0, lock_depth = -1,
172 counter = 1, nice = 0, policy = 0, mm = 0x0, processor = 0,
173 cpus_runnable = 1, cpus_allowed = 4294967295, run_list = {next = 0x0,
174 prev = 0x0}, sleep_time = 18995, next_task = 0xc017c000,
175 prev_task = 0xc2f94000, active_mm = 0x0, local_pages = {next = 0xc2da0054,
176 prev = 0xc2da0054}, allocation_order = 0, nr_local_pages = 0,
177 ...
178 5. To resume Xen, enter the "continue" command to gdb.
179 This sends the packet $c#63 along the serial channel.
181 (gdb) cont
182 Continuing.
184 Debugging Multiple Domains & Processes
185 --------------------------------------
187 pdb supports debugging multiple domains & processes. You can switch
188 between different domains and processes within domains and examine
189 variables in each.
191 The pdb context identifies the current debug target. It is stored
192 in the xen variable pdb_ctx and defaults to xen.
194 target pdb_ctx.domain pdb_ctx.process
195 ------ -------------- ---------------
196 xen -1 -1
197 guest os 0,1,2,... -1
198 process 0,1,2,... 0,1,2,...
200 Unfortunately, gdb doesn't understand debugging multiple process
201 simultaneously (we're working on it), so at present you are limited
202 to just one set of symbols for symbolic debugging. When debugging
203 processes, pdb currently supports just Linux 2.4.
205 define setup
206 file xeno-clone/xeno.bk/xen/xen
207 add-sym xeno-clone/xenolinux-2.4.25/vmlinux
208 add-sym ~ach61/a.out
209 end
212 1. Connect with gdb as before. A couple of Linux-specific
213 symbols need to be defined.
215 (gdb) target remote <hostname>:<port + 1> /* contact nsplitd */
216 Remote debugging using serial.srg:12131
217 continue_cpu_idle_loop () at current.h:10
218 warning: shared library handler failed to enable breakpoint
219 (gdb) set pdb_pidhash_addr = &pidhash
220 (gdb) set pdb_init_task_union_addr = &init_task_union
222 2. The pdb context defaults to Xen and we can read Xen's memory.
223 An attempt to access domain 0 memory fails.
225 (gdb) print pdb_ctx
226 $1 = {valid = 0, domain = -1, process = -1, ptbr = 1052672}
227 (gdb) print hexchars
228 $2 = "0123456789abcdef"
229 (gdb) print cpu_vendor_names
230 Cannot access memory at address 0xc0191f80
232 3. Now we change to domain 0. In addition to changing pdb_ctx.domain,
233 we need to change pdb_ctx.valid to signal pdb of the change.
234 It is now possible to examine Xen and Linux memory.
236 (gdb) set pdb_ctx.domain=0
237 (gdb) set pdb_ctx.valid=1
238 (gdb) print hexchars
239 $3 = "0123456789abcdef"
240 (gdb) print cpu_vendor_names
241 $4 = {0xc0158b46 "Intel", 0xc0158c37 "Cyrix", 0xc0158b55 "AMD",
242 0xc0158c3d "UMC", 0xc0158c41 "NexGen", 0xc0158c48 "Centaur",
243 0xc0158c50 "Rise", 0xc0158c55 "Transmeta"}
245 4. Now change to a process within domain 0. Again, we need to
246 change pdb_ctx.valid in addition to pdb_ctx.process.
248 (gdb) set pdb_ctx.process=962
249 (gdb) set pdb_ctx.valid =1
250 (gdb) print pdb_ctx
251 $1 = {valid = 0, domain = 0, process = 962, ptbr = 52998144}
252 (gdb) print aho_a
253 $2 = 20
255 5. Now we can read the same variable from another process running
256 the same executable in another domain.
258 (gdb) set pdb_ctx.domain=1
259 (gdb) set pdb_ctx.process=1210
260 (gdb) set pdb_ctx.valid=1
261 (gdb) print pdb_ctx
262 $3 = {valid = 0, domain = 1, process = 1210, ptbr = 70574080}
263 (gdb) print aho_a
264 $4 = 27
267 Some Helpful .gdbinit Commands
268 ------------------------------
270 define setup
271 file .../install/boot/xen-syms
272 add-sym .../install/boot/vmlinux-syms-2.4.26-xen0
273 add-sym /homes/aho/a.out
274 end
275 document setup
276 load symbols for xen, xenolinux (dom 0), and "a.out"
277 end
279 define setup-linux
280 set pdb_pidhash_addr = &pidhash
281 set pdb_init_task_union_addr = &init_task_union
283 set task_struct_mm_offset = (void *)&(init_task_union.task.mm) - (void *)&(init_task_union.task)
284 set task_struct_next_task_offset = (void *)&(init_task_union.task.next_task) - (void *)&(init_task_union.task)
285 set task_struct_pid_offset = (void *)&(init_task_union.task.pid) - (void *)&(init_task_union.task)
286 set task_struct_pidhash_next_offset = (void *)&(init_task_union.task.pidhash_next) - (void *)&(init_task_union.task)
287 set task_struct_comm_offset = (void *)&(init_task_union.task.comm) - (void *)&(init_task_union.task)
288 set task_struct_comm_length = sizeof (init_task_union.task.comm)
290 set mm_struct_pgd_offset = sizeof (struct vm_area_struct *) * 2 + sizeof (rb_root_t)
291 end
292 document setup-linux
293 define various xenolinux specific offsets and sizes in pdb
294 end
299 Changes
300 -------
302 04.07.15 aho .gdbinit
303 04.02.05 aho creation
304 04.03.31 aho add description on debugging multiple domains