view Documentation/networking/s2io.txt @ 897:329ea0ccb344

balloon: try harder to balloon up under memory pressure.

Currently if the balloon driver is unable to increase the guest's
reservation it assumes the failure was due to reaching its full
allocation, gives up on the ballooning operation and records the limit
it reached as the "hard limit". The driver will not try again until
the target is set again (even to the same value).

However it is possible that ballooning has in fact failed due to
memory pressure in the host and therefore it is desirable to keep
attempting to reach the target in case memory becomes available. The
most likely scenario is that some guests are ballooning down while
others are ballooning up and therefore there is temporary memory
pressure while things stabilise. You would not expect a well behaved
toolstack to ask a domain to balloon to more than its allocation nor
would you expect it to deliberately over-commit memory by setting
balloon targets which exceed the total host memory.

This patch drops the concept of a hard limit and causes the balloon
driver to retry increasing the reservation on a timer in the same
manner as when decreasing the reservation.

Also if we partially succeed in increasing the reservation
(i.e. receive less pages than we asked for) then we may as well keep
those pages rather than returning them to Xen.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
author Keir Fraser <keir.fraser@citrix.com>
date Fri Jun 05 14:01:20 2009 +0100 (2009-06-05)
parents 831230e53067
line source
1 Release notes for Neterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver.
3 Contents
4 =======
5 - 1. Introduction
6 - 2. Identifying the adapter/interface
7 - 3. Features supported
8 - 4. Command line parameters
9 - 5. Performance suggestions
10 - 6. Available Downloads
13 1. Introduction:
14 This Linux driver supports Neterion's Xframe I PCI-X 1.0 and
15 Xframe II PCI-X 2.0 adapters. It supports several features
16 such as jumbo frames, MSI/MSI-X, checksum offloads, TSO, UFO and so on.
17 See below for complete list of features.
18 All features are supported for both IPv4 and IPv6.
20 2. Identifying the adapter/interface:
21 a. Insert the adapter(s) in your system.
22 b. Build and load driver
23 # insmod s2io.ko
24 c. View log messages
25 # dmesg | tail -40
26 You will see messages similar to:
27 eth3: Neterion Xframe I 10GbE adapter (rev 3), Version, Intr type INTA
28 eth4: Neterion Xframe II 10GbE adapter (rev 2), Version, Intr type INTA
29 eth4: Device is on 64 bit 133MHz PCIX(M1) bus
31 The above messages identify the adapter type(Xframe I/II), adapter revision,
32 driver version, interface name(eth3, eth4), Interrupt type(INTA, MSI, MSI-X).
33 In case of Xframe II, the PCI/PCI-X bus width and frequency are displayed
34 as well.
36 To associate an interface with a physical adapter use "ethtool -p <ethX>".
37 The corresponding adapter's LED will blink multiple times.
39 3. Features supported:
40 a. Jumbo frames. Xframe I/II supports MTU upto 9600 bytes,
41 modifiable using ifconfig command.
43 b. Offloads. Supports checksum offload(TCP/UDP/IP) on transmit
44 and receive, TSO.
46 c. Multi-buffer receive mode. Scattering of packet across multiple
47 buffers. Currently driver supports 2-buffer mode which yields
48 significant performance improvement on certain platforms(SGI Altix,
49 IBM xSeries).
51 d. MSI/MSI-X. Can be enabled on platforms which support this feature
52 (IA64, Xeon) resulting in noticeable performance improvement(upto 7%
53 on certain platforms).
55 e. NAPI. Compile-time option(CONFIG_S2IO_NAPI) for better Rx interrupt
56 moderation.
58 f. Statistics. Comprehensive MAC-level and software statistics displayed
59 using "ethtool -S" option.
61 g. Multi-FIFO/Ring. Supports up to 8 transmit queues and receive rings,
62 with multiple steering options.
64 4. Command line parameters
65 a. tx_fifo_num
66 Number of transmit queues
67 Valid range: 1-8
68 Default: 1
70 b. rx_ring_num
71 Number of receive rings
72 Valid range: 1-8
73 Default: 1
75 c. tx_fifo_len
76 Size of each transmit queue
77 Valid range: Total length of all queues should not exceed 8192
78 Default: 4096
80 d. rx_ring_sz
81 Size of each receive ring(in 4K blocks)
82 Valid range: Limited by memory on system
83 Default: 30
85 e. intr_type
86 Specifies interrupt type. Possible values 1(INTA), 2(MSI), 3(MSI-X)
87 Valid range: 1-3
88 Default: 1
90 5. Performance suggestions
91 General:
92 a. Set MTU to maximum(9000 for switch setup, 9600 in back-to-back configuration)
93 b. Set TCP windows size to optimal value.
94 For instance, for MTU=1500 a value of 210K has been observed to result in
95 good performance.
96 # sysctl -w net.ipv4.tcp_rmem="210000 210000 210000"
97 # sysctl -w net.ipv4.tcp_wmem="210000 210000 210000"
98 For MTU=9000, TCP window size of 10 MB is recommended.
99 # sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000"
100 # sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000"
102 Transmit performance:
103 a. By default, the driver respects BIOS settings for PCI bus parameters.
104 However, you may want to experiment with PCI bus parameters
105 max-split-transactions(MOST) and MMRBC (use setpci command).
106 A MOST value of 2 has been found optimal for Opterons and 3 for Itanium.
107 It could be different for your hardware.
108 Set MMRBC to 4K**.
110 For example you can set
111 For opteron
112 #setpci -d 17d5:* 62=1d
113 For Itanium
114 #setpci -d 17d5:* 62=3d
116 For detailed description of the PCI registers, please see Xframe User Guide.
118 b. Ensure Transmit Checksum offload is enabled. Use ethtool to set/verify this
119 parameter.
120 c. Turn on TSO(using "ethtool -K")
121 # ethtool -K <ethX> tso on
123 Receive performance:
124 a. By default, the driver respects BIOS settings for PCI bus parameters.
125 However, you may want to set PCI latency timer to 248.
126 #setpci -d 17d5:* LATENCY_TIMER=f8
127 For detailed description of the PCI registers, please see Xframe User Guide.
128 b. Use 2-buffer mode. This results in large performance boost on
129 on certain platforms(eg. SGI Altix, IBM xSeries).
130 c. Ensure Receive Checksum offload is enabled. Use "ethtool -K ethX" command to
131 set/verify this option.
132 d. Enable NAPI feature(in kernel configuration Device Drivers ---> Network
133 device support ---> Ethernet (10000 Mbit) ---> S2IO 10Gbe Xframe NIC) to
134 bring down CPU utilization.
136 ** For AMD opteron platforms with 8131 chipset, MMRBC=1 and MOST=1 are
137 recommended as safe parameters.
138 For more information, please review the AMD8131 errata at
139 http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/26310.pdf
141 6. Available Downloads
142 Neterion "s2io" driver in Red Hat and Suse 2.6-based distributions is kept up
143 to date, also the latest "s2io" code (including support for 2.4 kernels) is
144 available via "Support" link on the Neterion site: http://www.neterion.com.
146 For Xframe User Guide (Programming manual), visit ftp site ns1.s2io.com,
147 user: linuxdocs password: HALdocs
149 7. Support
150 For further support please contact either your 10GbE Xframe NIC vendor (IBM,
151 HP, SGI etc.) or click on the "Support" link on the Neterion site:
152 http://www.neterion.com.