1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
|
# ipoibmodemtu is designed to be able to set an InfiniBand port's mode and MTU.
# But, mode datagram and MTU 2044 should be the default for an Infiniband
# port, so unless you're changing to connected mode and MTU 65520, you
# shouldn't need ipoibmodemtu.
# By default, InfiniBand uses UD (Unreliable Datagram) transport, which
# typically forces the maximum MTU to be 2044 bytes. (Technically to the
# InfiniBand L2 MTU - 4 bytes for the IPoIB encapsulation header.)
# Switching to RC (Reliable Connected) transport allows a MTU up to the maximum
# IP packet size of 64K (65520 bytes is the exact maximum.)
# The drawback is that Reliable Connected mode increases TCP latency for short
# messages by about 5% over datagram mode's latency.
# Different setups will see different results. Some people see a gigantic
# speed increase switching to connected mode and a 65520 MTU. A few see about
# the same, or even worse, speeds. Use qperf, perftest, and/or iperf to fine-
# tune your system.
# mode[<network device name>]=<mode>
# network device name:
# Use your InfiniBand port's IPoIB network device name. Ex: ib0
# mode valid values:
# connected - Use Reliable Connected transport.
# datagram - Use Unreliable Datagram transport. This should be the
# port's default without ipoibmodemtu, so if you want this
# option, you probably don't need ipoibmodemtu.
# (you can have a mode line for each InfiniBand IPoIB network device)
# Ex: If you have an InfiniBand card that is assigned network devices ib0 and
# ib1 after enabling IPoIB, and want ib0 to run connected and ib1 to run
# datagram:
# mode[ib0]=connected
# mode[ib1]=datagram
# (In practice, it's usually best to have all devices on an InfiniBand
# subnet running the same transport mode. This is just as an example.)
mode[ib0]=connected
# mtu[<network device name>]=<mtu>
# network device name:
# Use your InfiniBand port's IPoIB network device name. Ex: ib0
# mtu valid values:
# ipoibmodemtu allows any integer value.
# The value is directly passed along to the program 'ip', so the actual
# valid values will depend on your InfiniBand card, kernel drivers, etc.
# Typically, valid values are up to:
# (for connected mode) 65520
# (for datagram mode) 2044
# (you can have a MTU line for each InfiniBand IPoIB network device)
# Ex: If you have an InfiniBand card that is assigned network devices ib0 and
# ib1 after enabling IPoIB, and want ib0 to run a MTU of 65520 and ib0 to
# run 2044:
# mtu[ib0]=65520
# mtu[ib1]=2044
mtu[ib0]=65520
|