view Documentation/kprobes.txt @ 452:c7ed6fe5dca0

kexec: dont initialise regions in reserve_memory()

There is no need to initialise efi_memmap_res and boot_param_res in
reserve_memory() for the initial xen domain as it is done in
machine_kexec_setup_resources() using values from the kexec hypercall.

Signed-off-by: Simon Horman <horms@verge.net.au>
author Keir Fraser <keir.fraser@citrix.com>
date Thu Feb 28 10:55:18 2008 +0000 (2008-02-28)
parents 831230e53067
line source
1 Title : Kernel Probes (Kprobes)
2 Authors : Jim Keniston <jkenisto@us.ibm.com>
3 : Prasanna S Panchamukhi <prasanna@in.ibm.com>
7 1. Concepts: Kprobes, Jprobes, Return Probes
8 2. Architectures Supported
9 3. Configuring Kprobes
10 4. API Reference
11 5. Kprobes Features and Limitations
12 6. Probe Overhead
13 7. TODO
14 8. Kprobes Example
15 9. Jprobes Example
16 10. Kretprobes Example
18 1. Concepts: Kprobes, Jprobes, Return Probes
20 Kprobes enables you to dynamically break into any kernel routine and
21 collect debugging and performance information non-disruptively. You
22 can trap at almost any kernel code address, specifying a handler
23 routine to be invoked when the breakpoint is hit.
25 There are currently three types of probes: kprobes, jprobes, and
26 kretprobes (also called return probes). A kprobe can be inserted
27 on virtually any instruction in the kernel. A jprobe is inserted at
28 the entry to a kernel function, and provides convenient access to the
29 function's arguments. A return probe fires when a specified function
30 returns.
32 In the typical case, Kprobes-based instrumentation is packaged as
33 a kernel module. The module's init function installs ("registers")
34 one or more probes, and the exit function unregisters them. A
35 registration function such as register_kprobe() specifies where
36 the probe is to be inserted and what handler is to be called when
37 the probe is hit.
39 The next three subsections explain how the different types of
40 probes work. They explain certain things that you'll need to
41 know in order to make the best use of Kprobes -- e.g., the
42 difference between a pre_handler and a post_handler, and how
43 to use the maxactive and nmissed fields of a kretprobe. But
44 if you're in a hurry to start using Kprobes, you can skip ahead
45 to section 2.
47 1.1 How Does a Kprobe Work?
49 When a kprobe is registered, Kprobes makes a copy of the probed
50 instruction and replaces the first byte(s) of the probed instruction
51 with a breakpoint instruction (e.g., int3 on i386 and x86_64).
53 When a CPU hits the breakpoint instruction, a trap occurs, the CPU's
54 registers are saved, and control passes to Kprobes via the
55 notifier_call_chain mechanism. Kprobes executes the "pre_handler"
56 associated with the kprobe, passing the handler the addresses of the
57 kprobe struct and the saved registers.
59 Next, Kprobes single-steps its copy of the probed instruction.
60 (It would be simpler to single-step the actual instruction in place,
61 but then Kprobes would have to temporarily remove the breakpoint
62 instruction. This would open a small time window when another CPU
63 could sail right past the probepoint.)
65 After the instruction is single-stepped, Kprobes executes the
66 "post_handler," if any, that is associated with the kprobe.
67 Execution then continues with the instruction following the probepoint.
69 1.2 How Does a Jprobe Work?
71 A jprobe is implemented using a kprobe that is placed on a function's
72 entry point. It employs a simple mirroring principle to allow
73 seamless access to the probed function's arguments. The jprobe
74 handler routine should have the same signature (arg list and return
75 type) as the function being probed, and must always end by calling
76 the Kprobes function jprobe_return().
78 Here's how it works. When the probe is hit, Kprobes makes a copy of
79 the saved registers and a generous portion of the stack (see below).
80 Kprobes then points the saved instruction pointer at the jprobe's
81 handler routine, and returns from the trap. As a result, control
82 passes to the handler, which is presented with the same register and
83 stack contents as the probed function. When it is done, the handler
84 calls jprobe_return(), which traps again to restore the original stack
85 contents and processor state and switch to the probed function.
87 By convention, the callee owns its arguments, so gcc may produce code
88 that unexpectedly modifies that portion of the stack. This is why
89 Kprobes saves a copy of the stack and restores it after the jprobe
90 handler has run. Up to MAX_STACK_SIZE bytes are copied -- e.g.,
91 64 bytes on i386.
93 Note that the probed function's args may be passed on the stack
94 or in registers (e.g., for x86_64 or for an i386 fastcall function).
95 The jprobe will work in either case, so long as the handler's
96 prototype matches that of the probed function.
98 1.3 How Does a Return Probe Work?
100 When you call register_kretprobe(), Kprobes establishes a kprobe at
101 the entry to the function. When the probed function is called and this
102 probe is hit, Kprobes saves a copy of the return address, and replaces
103 the return address with the address of a "trampoline." The trampoline
104 is an arbitrary piece of code -- typically just a nop instruction.
105 At boot time, Kprobes registers a kprobe at the trampoline.
107 When the probed function executes its return instruction, control
108 passes to the trampoline and that probe is hit. Kprobes' trampoline
109 handler calls the user-specified handler associated with the kretprobe,
110 then sets the saved instruction pointer to the saved return address,
111 and that's where execution resumes upon return from the trap.
113 While the probed function is executing, its return address is
114 stored in an object of type kretprobe_instance. Before calling
115 register_kretprobe(), the user sets the maxactive field of the
116 kretprobe struct to specify how many instances of the specified
117 function can be probed simultaneously. register_kretprobe()
118 pre-allocates the indicated number of kretprobe_instance objects.
120 For example, if the function is non-recursive and is called with a
121 spinlock held, maxactive = 1 should be enough. If the function is
122 non-recursive and can never relinquish the CPU (e.g., via a semaphore
123 or preemption), NR_CPUS should be enough. If maxactive <= 0, it is
124 set to a default value. If CONFIG_PREEMPT is enabled, the default
125 is max(10, 2*NR_CPUS). Otherwise, the default is NR_CPUS.
127 It's not a disaster if you set maxactive too low; you'll just miss
128 some probes. In the kretprobe struct, the nmissed field is set to
129 zero when the return probe is registered, and is incremented every
130 time the probed function is entered but there is no kretprobe_instance
131 object available for establishing the return probe.
133 2. Architectures Supported
135 Kprobes, jprobes, and return probes are implemented on the following
136 architectures:
138 - i386
139 - x86_64 (AMD-64, EM64T)
140 - ppc64
141 - ia64 (Does not support probes on instruction slot1.)
142 - sparc64 (Return probes not yet implemented.)
144 3. Configuring Kprobes
146 When configuring the kernel using make menuconfig/xconfig/oldconfig,
147 ensure that CONFIG_KPROBES is set to "y". Under "Instrumentation
148 Support", look for "Kprobes".
150 So that you can load and unload Kprobes-based instrumentation modules,
151 make sure "Loadable module support" (CONFIG_MODULES) and "Module
152 unloading" (CONFIG_MODULE_UNLOAD) are set to "y".
154 You may also want to ensure that CONFIG_KALLSYMS and perhaps even
155 CONFIG_KALLSYMS_ALL are set to "y", since kallsyms_lookup_name()
156 is a handy, version-independent way to find a function's address.
158 If you need to insert a probe in the middle of a function, you may find
159 it useful to "Compile the kernel with debug info" (CONFIG_DEBUG_INFO),
160 so you can use "objdump -d -l vmlinux" to see the source-to-object
161 code mapping.
163 4. API Reference
165 The Kprobes API includes a "register" function and an "unregister"
166 function for each type of probe. Here are terse, mini-man-page
167 specifications for these functions and the associated probe handlers
168 that you'll write. See the latter half of this document for examples.
170 4.1 register_kprobe
172 #include <linux/kprobes.h>
173 int register_kprobe(struct kprobe *kp);
175 Sets a breakpoint at the address kp->addr. When the breakpoint is
176 hit, Kprobes calls kp->pre_handler. After the probed instruction
177 is single-stepped, Kprobe calls kp->post_handler. If a fault
178 occurs during execution of kp->pre_handler or kp->post_handler,
179 or during single-stepping of the probed instruction, Kprobes calls
180 kp->fault_handler. Any or all handlers can be NULL.
182 register_kprobe() returns 0 on success, or a negative errno otherwise.
184 User's pre-handler (kp->pre_handler):
185 #include <linux/kprobes.h>
186 #include <linux/ptrace.h>
187 int pre_handler(struct kprobe *p, struct pt_regs *regs);
189 Called with p pointing to the kprobe associated with the breakpoint,
190 and regs pointing to the struct containing the registers saved when
191 the breakpoint was hit. Return 0 here unless you're a Kprobes geek.
193 User's post-handler (kp->post_handler):
194 #include <linux/kprobes.h>
195 #include <linux/ptrace.h>
196 void post_handler(struct kprobe *p, struct pt_regs *regs,
197 unsigned long flags);
199 p and regs are as described for the pre_handler. flags always seems
200 to be zero.
202 User's fault-handler (kp->fault_handler):
203 #include <linux/kprobes.h>
204 #include <linux/ptrace.h>
205 int fault_handler(struct kprobe *p, struct pt_regs *regs, int trapnr);
207 p and regs are as described for the pre_handler. trapnr is the
208 architecture-specific trap number associated with the fault (e.g.,
209 on i386, 13 for a general protection fault or 14 for a page fault).
210 Returns 1 if it successfully handled the exception.
212 4.2 register_jprobe
214 #include <linux/kprobes.h>
215 int register_jprobe(struct jprobe *jp)
217 Sets a breakpoint at the address jp->kp.addr, which must be the address
218 of the first instruction of a function. When the breakpoint is hit,
219 Kprobes runs the handler whose address is jp->entry.
221 The handler should have the same arg list and return type as the probed
222 function; and just before it returns, it must call jprobe_return().
223 (The handler never actually returns, since jprobe_return() returns
224 control to Kprobes.) If the probed function is declared asmlinkage,
225 fastcall, or anything else that affects how args are passed, the
226 handler's declaration must match.
228 register_jprobe() returns 0 on success, or a negative errno otherwise.
230 4.3 register_kretprobe
232 #include <linux/kprobes.h>
233 int register_kretprobe(struct kretprobe *rp);
235 Establishes a return probe for the function whose address is
236 rp->kp.addr. When that function returns, Kprobes calls rp->handler.
237 You must set rp->maxactive appropriately before you call
238 register_kretprobe(); see "How Does a Return Probe Work?" for details.
240 register_kretprobe() returns 0 on success, or a negative errno
241 otherwise.
243 User's return-probe handler (rp->handler):
244 #include <linux/kprobes.h>
245 #include <linux/ptrace.h>
246 int kretprobe_handler(struct kretprobe_instance *ri, struct pt_regs *regs);
248 regs is as described for kprobe.pre_handler. ri points to the
249 kretprobe_instance object, of which the following fields may be
250 of interest:
251 - ret_addr: the return address
252 - rp: points to the corresponding kretprobe object
253 - task: points to the corresponding task struct
254 The handler's return value is currently ignored.
256 4.4 unregister_*probe
258 #include <linux/kprobes.h>
259 void unregister_kprobe(struct kprobe *kp);
260 void unregister_jprobe(struct jprobe *jp);
261 void unregister_kretprobe(struct kretprobe *rp);
263 Removes the specified probe. The unregister function can be called
264 at any time after the probe has been registered.
266 5. Kprobes Features and Limitations
268 Kprobes allows multiple probes at the same address. Currently,
269 however, there cannot be multiple jprobes on the same function at
270 the same time.
272 In general, you can install a probe anywhere in the kernel.
273 In particular, you can probe interrupt handlers. Known exceptions
274 are discussed in this section.
276 The register_*probe functions will return -EINVAL if you attempt
277 to install a probe in the code that implements Kprobes (mostly
278 kernel/kprobes.c and arch/*/kernel/kprobes.c, but also functions such
279 as do_page_fault and notifier_call_chain).
281 If you install a probe in an inline-able function, Kprobes makes
282 no attempt to chase down all inline instances of the function and
283 install probes there. gcc may inline a function without being asked,
284 so keep this in mind if you're not seeing the probe hits you expect.
286 A probe handler can modify the environment of the probed function
287 -- e.g., by modifying kernel data structures, or by modifying the
288 contents of the pt_regs struct (which are restored to the registers
289 upon return from the breakpoint). So Kprobes can be used, for example,
290 to install a bug fix or to inject faults for testing. Kprobes, of
291 course, has no way to distinguish the deliberately injected faults
292 from the accidental ones. Don't drink and probe.
294 Kprobes makes no attempt to prevent probe handlers from stepping on
295 each other -- e.g., probing printk() and then calling printk() from a
296 probe handler. If a probe handler hits a probe, that second probe's
297 handlers won't be run in that instance, and the kprobe.nmissed member
298 of the second probe will be incremented.
300 As of Linux v2.6.15-rc1, multiple handlers (or multiple instances of
301 the same handler) may run concurrently on different CPUs.
303 Kprobes does not use mutexes or allocate memory except during
304 registration and unregistration.
306 Probe handlers are run with preemption disabled. Depending on the
307 architecture, handlers may also run with interrupts disabled. In any
308 case, your handler should not yield the CPU (e.g., by attempting to
309 acquire a semaphore).
311 Since a return probe is implemented by replacing the return
312 address with the trampoline's address, stack backtraces and calls
313 to __builtin_return_address() will typically yield the trampoline's
314 address instead of the real return address for kretprobed functions.
315 (As far as we can tell, __builtin_return_address() is used only
316 for instrumentation and error reporting.)
318 If the number of times a function is called does not match the number
319 of times it returns, registering a return probe on that function may
320 produce undesirable results. We have the do_exit() case covered.
321 do_execve() and do_fork() are not an issue. We're unaware of other
322 specific cases where this could be a problem.
324 If, upon entry to or exit from a function, the CPU is running on
325 a stack other than that of the current task, registering a return
326 probe on that function may produce undesirable results. For this
327 reason, Kprobes doesn't support return probes (or kprobes or jprobes)
328 on the x86_64 version of __switch_to(); the registration functions
329 return -EINVAL.
331 6. Probe Overhead
333 On a typical CPU in use in 2005, a kprobe hit takes 0.5 to 1.0
334 microseconds to process. Specifically, a benchmark that hits the same
335 probepoint repeatedly, firing a simple handler each time, reports 1-2
336 million hits per second, depending on the architecture. A jprobe or
337 return-probe hit typically takes 50-75% longer than a kprobe hit.
338 When you have a return probe set on a function, adding a kprobe at
339 the entry to that function adds essentially no overhead.
341 Here are sample overhead figures (in usec) for different architectures.
342 k = kprobe; j = jprobe; r = return probe; kr = kprobe + return probe
343 on same function; jr = jprobe + return probe on same function
345 i386: Intel Pentium M, 1495 MHz, 2957.31 bogomips
346 k = 0.57 usec; j = 1.00; r = 0.92; kr = 0.99; jr = 1.40
348 x86_64: AMD Opteron 246, 1994 MHz, 3971.48 bogomips
349 k = 0.49 usec; j = 0.76; r = 0.80; kr = 0.82; jr = 1.07
351 ppc64: POWER5 (gr), 1656 MHz (SMT disabled, 1 virtual CPU per physical CPU)
352 k = 0.77 usec; j = 1.31; r = 1.26; kr = 1.45; jr = 1.99
354 7. TODO
356 a. SystemTap (http://sourceware.org/systemtap): Provides a simplified
357 programming interface for probe-based instrumentation. Try it out.
358 b. Kernel return probes for sparc64.
359 c. Support for other architectures.
360 d. User-space probes.
361 e. Watchpoint probes (which fire on data references).
363 8. Kprobes Example
365 Here's a sample kernel module showing the use of kprobes to dump a
366 stack trace and selected i386 registers when do_fork() is called.
367 ----- cut here -----
368 /*kprobe_example.c*/
369 #include <linux/kernel.h>
370 #include <linux/module.h>
371 #include <linux/kprobes.h>
372 #include <linux/kallsyms.h>
373 #include <linux/sched.h>
375 /*For each probe you need to allocate a kprobe structure*/
376 static struct kprobe kp;
378 /*kprobe pre_handler: called just before the probed instruction is executed*/
379 int handler_pre(struct kprobe *p, struct pt_regs *regs)
380 {
381 printk("pre_handler: p->addr=0x%p, eip=%lx, eflags=0x%lx\n",
382 p->addr, regs->eip, regs->eflags);
383 dump_stack();
384 return 0;
385 }
387 /*kprobe post_handler: called after the probed instruction is executed*/
388 void handler_post(struct kprobe *p, struct pt_regs *regs, unsigned long flags)
389 {
390 printk("post_handler: p->addr=0x%p, eflags=0x%lx\n",
391 p->addr, regs->eflags);
392 }
394 /* fault_handler: this is called if an exception is generated for any
395 * instruction within the pre- or post-handler, or when Kprobes
396 * single-steps the probed instruction.
397 */
398 int handler_fault(struct kprobe *p, struct pt_regs *regs, int trapnr)
399 {
400 printk("fault_handler: p->addr=0x%p, trap #%dn",
401 p->addr, trapnr);
402 /* Return 0 because we don't handle the fault. */
403 return 0;
404 }
406 int init_module(void)
407 {
408 int ret;
409 kp.pre_handler = handler_pre;
410 kp.post_handler = handler_post;
411 kp.fault_handler = handler_fault;
412 kp.addr = (kprobe_opcode_t*) kallsyms_lookup_name("do_fork");
413 /* register the kprobe now */
414 if (!kp.addr) {
415 printk("Couldn't find %s to plant kprobe\n", "do_fork");
416 return -1;
417 }
418 if ((ret = register_kprobe(&kp) < 0)) {
419 printk("register_kprobe failed, returned %d\n", ret);
420 return -1;
421 }
422 printk("kprobe registered\n");
423 return 0;
424 }
426 void cleanup_module(void)
427 {
428 unregister_kprobe(&kp);
429 printk("kprobe unregistered\n");
430 }
433 ----- cut here -----
435 You can build the kernel module, kprobe-example.ko, using the following
436 Makefile:
437 ----- cut here -----
438 obj-m := kprobe-example.o
439 KDIR := /lib/modules/$(shell uname -r)/build
440 PWD := $(shell pwd)
441 default:
442 $(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
443 clean:
444 rm -f *.mod.c *.ko *.o
445 ----- cut here -----
447 $ make
448 $ su -
449 ...
450 # insmod kprobe-example.ko
452 You will see the trace data in /var/log/messages and on the console
453 whenever do_fork() is invoked to create a new process.
455 9. Jprobes Example
457 Here's a sample kernel module showing the use of jprobes to dump
458 the arguments of do_fork().
459 ----- cut here -----
460 /*jprobe-example.c */
461 #include <linux/kernel.h>
462 #include <linux/module.h>
463 #include <linux/fs.h>
464 #include <linux/uio.h>
465 #include <linux/kprobes.h>
466 #include <linux/kallsyms.h>
468 /*
469 * Jumper probe for do_fork.
470 * Mirror principle enables access to arguments of the probed routine
471 * from the probe handler.
472 */
474 /* Proxy routine having the same arguments as actual do_fork() routine */
475 long jdo_fork(unsigned long clone_flags, unsigned long stack_start,
476 struct pt_regs *regs, unsigned long stack_size,
477 int __user * parent_tidptr, int __user * child_tidptr)
478 {
479 printk("jprobe: clone_flags=0x%lx, stack_size=0x%lx, regs=0x%p\n",
480 clone_flags, stack_size, regs);
481 /* Always end with a call to jprobe_return(). */
482 jprobe_return();
484 return 0;
485 }
487 static struct jprobe my_jprobe = {
488 .entry = (kprobe_opcode_t *) jdo_fork
489 };
491 int init_module(void)
492 {
493 int ret;
494 my_jprobe.kp.addr = (kprobe_opcode_t *) kallsyms_lookup_name("do_fork");
495 if (!my_jprobe.kp.addr) {
496 printk("Couldn't find %s to plant jprobe\n", "do_fork");
497 return -1;
498 }
500 if ((ret = register_jprobe(&my_jprobe)) <0) {
501 printk("register_jprobe failed, returned %d\n", ret);
502 return -1;
503 }
504 printk("Planted jprobe at %p, handler addr %p\n",
505 my_jprobe.kp.addr, my_jprobe.entry);
506 return 0;
507 }
509 void cleanup_module(void)
510 {
511 unregister_jprobe(&my_jprobe);
512 printk("jprobe unregistered\n");
513 }
516 ----- cut here -----
518 Build and insert the kernel module as shown in the above kprobe
519 example. You will see the trace data in /var/log/messages and on
520 the console whenever do_fork() is invoked to create a new process.
521 (Some messages may be suppressed if syslogd is configured to
522 eliminate duplicate messages.)
524 10. Kretprobes Example
526 Here's a sample kernel module showing the use of return probes to
527 report failed calls to sys_open().
528 ----- cut here -----
529 /*kretprobe-example.c*/
530 #include <linux/kernel.h>
531 #include <linux/module.h>
532 #include <linux/kprobes.h>
533 #include <linux/kallsyms.h>
535 static const char *probed_func = "sys_open";
537 /* Return-probe handler: If the probed function fails, log the return value. */
538 static int ret_handler(struct kretprobe_instance *ri, struct pt_regs *regs)
539 {
540 // Substitute the appropriate register name for your architecture --
541 // e.g., regs->rax for x86_64, regs->gpr[3] for ppc64.
542 int retval = (int) regs->eax;
543 if (retval < 0) {
544 printk("%s returns %d\n", probed_func, retval);
545 }
546 return 0;
547 }
549 static struct kretprobe my_kretprobe = {
550 .handler = ret_handler,
551 /* Probe up to 20 instances concurrently. */
552 .maxactive = 20
553 };
555 int init_module(void)
556 {
557 int ret;
558 my_kretprobe.kp.addr =
559 (kprobe_opcode_t *) kallsyms_lookup_name(probed_func);
560 if (!my_kretprobe.kp.addr) {
561 printk("Couldn't find %s to plant return probe\n", probed_func);
562 return -1;
563 }
564 if ((ret = register_kretprobe(&my_kretprobe)) < 0) {
565 printk("register_kretprobe failed, returned %d\n", ret);
566 return -1;
567 }
568 printk("Planted return probe at %p\n", my_kretprobe.kp.addr);
569 return 0;
570 }
572 void cleanup_module(void)
573 {
574 unregister_kretprobe(&my_kretprobe);
575 printk("kretprobe unregistered\n");
576 /* nmissed > 0 suggests that maxactive was set too low. */
577 printk("Missed probing %d instances of %s\n",
578 my_kretprobe.nmissed, probed_func);
579 }
582 ----- cut here -----
584 Build and insert the kernel module as shown in the above kprobe
585 example. You will see the trace data in /var/log/messages and on the
586 console whenever sys_open() returns a negative value. (Some messages
587 may be suppressed if syslogd is configured to eliminate duplicate
588 messages.)
590 For additional information on Kprobes, refer to the following URLs:
591 http://www-106.ibm.com/developerworks/library/l-kprobes.html?ca=dgr-lnxw42Kprobe
592 http://www.redhat.com/magazine/005mar05/features/kprobes/