Eric Dumazet 46d3ceabd8
tcp: TCP Small Queues ...
This introduce TSQ (TCP Small Queues)
TSQ goal is to reduce number of TCP packets in xmit queues (qdisc &
device queues), to reduce RTT and cwnd bias, part of the bufferbloat
problem.
sk->sk_wmem_alloc not allowed to grow above a given limit,
allowing no more than ~128KB [1] per tcp socket in qdisc/dev layers at a
given time.
TSO packets are sized/capped to half the limit, so that we have two
TSO packets in flight, allowing better bandwidth use.
As a side effect, setting the limit to 40000 automatically reduces the
standard gso max limit (65536) to 40000/2 : It can help to reduce
latencies of high prio packets, having smaller TSO packets.
This means we divert sock_wfree() to a tcp_wfree() handler, to
queue/send following frames when skb_orphan() [2] is called for the
already queued skbs.
Results on my dev machines (tg3/ixgbe nics) are really impressive,
using standard pfifo_fast, and with or without TSO/GSO.
Without reduction of nominal bandwidth, we have reduction of buffering
per bulk sender :
< 1ms on Gbit (instead of 50ms with TSO)
< 8ms on 100Mbit (instead of 132 ms)
I no longer have 4 MBytes backlogged in qdisc by a single netperf
session, and both side socket autotuning no longer use 4 Mbytes.
As skb destructor cannot restart xmit itself ( as qdisc lock might be
taken at this point ), we delegate the work to a tasklet. We use one
tasklest per cpu for performance reasons.
If tasklet finds a socket owned by the user, it sets TSQ_OWNED flag.
This flag is tested in a new protocol method called from release_sock(),
to eventually send new segments.
[1] New /proc/sys/net/ipv4/tcp_limit_output_bytes tunable
[2] skb_orphan() is usually called at TX completion time,
but some drivers call it in their start_xmit() handler.
These drivers should at least use BQL, or else a single TCP
session can still fill the whole NIC TX ring, since TSQ will
have no effect.
Signed-off-by: Eric Dumazet <edumazet@google.com >
Cc: Dave Taht <dave.taht@bufferbloat.net >
Cc: Tom Herbert <therbert@google.com >
Cc: Matt Mathis <mattmathis@google.com >
Cc: Yuchung Cheng <ycheng@google.com >
Cc: Nandita Dukkipati <nanditad@google.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2012-07-11 18:12:59 -07:00
2012-05-25 09:31:59 -07:00
2012-06-27 15:23:16 -04:00
2012-06-19 21:40:14 +02:00
2012-05-30 11:17:19 -07:00
2012-05-07 15:39:35 -07:00
2012-05-20 10:06:08 -03:00
2012-04-20 09:22:02 -07:00
2012-05-12 13:32:17 +08:00
2012-04-23 14:23:32 +03:00
2012-05-24 10:34:29 -07:00
2012-05-14 13:34:31 -07:00
2012-05-10 22:39:43 -07:00
2012-04-11 17:55:06 -04:00
2012-05-29 11:53:11 -07:00
2012-07-07 16:23:06 -07:00
2012-06-20 14:41:57 -07:00
2012-06-01 16:55:42 -07:00
2012-07-04 19:51:50 +02:00
2012-02-21 13:29:06 +01:00
2012-06-07 14:58:38 +02:00
2012-05-09 12:53:47 +02:00
2012-05-15 17:30:30 -04:00
2012-05-31 18:18:11 -07:00
2012-05-12 19:57:34 -07:00
2012-05-23 14:15:09 -07:00
2012-05-05 20:28:11 -07:00
2012-05-22 13:54:18 +10:00
2012-05-14 18:03:04 +01:00
2012-05-29 16:22:33 -07:00
2012-06-28 17:37:00 -07:00
2012-06-06 15:20:22 -04:00
2012-05-31 20:29:58 -04:00
2012-06-28 17:37:00 -07:00
2012-03-16 10:38:24 -04:00
2012-03-30 16:45:39 -07:00
2012-02-20 12:48:48 -08:00
2012-04-20 14:56:36 -07:00
2012-03-01 16:14:48 +01:00
2012-05-29 16:22:29 -07:00
2012-03-24 10:41:37 -07:00
2012-03-01 13:38:49 +01:00
2012-05-15 20:14:35 -04:00
2012-03-07 17:06:08 +11:00
2012-03-16 10:38:24 -04:00
2012-04-14 11:13:20 +10:00
2012-05-19 12:17:35 -06:00
2012-03-20 21:29:46 -04:00
2012-04-01 12:55:00 -07:00
2012-03-04 17:54:34 -05:00
2012-03-23 16:58:34 -07:00
2012-03-06 21:27:24 +01:00
2012-06-15 08:46:23 +02:00
2012-05-29 16:22:22 -07:00
2012-05-29 16:22:20 -07:00
2012-03-16 10:38:24 -04:00
2012-07-04 13:07:05 +02:00
2012-05-23 17:42:39 -07:00
2012-03-24 10:41:37 -07:00
2012-04-11 09:16:48 -07:00
2012-05-08 14:13:25 -07:00
2012-05-15 12:02:39 -07:00
2012-05-11 17:13:00 +02:00
2012-05-02 09:30:32 +01:00
2012-05-25 01:44:51 +02:00
2012-03-28 18:30:03 +01:00
2012-06-03 20:05:57 -07:00
2012-05-31 17:49:32 -07:00
2012-06-18 09:47:00 -04:00
2012-02-28 16:02:54 +01:00
2012-05-31 17:49:29 -07:00
2012-03-24 10:41:37 -07:00
2012-03-30 03:23:30 -04:00
2012-03-29 15:38:31 +10:30
2012-03-29 14:46:05 -07:00
2012-03-16 10:38:24 -04:00
2012-03-23 16:58:38 -07:00
2012-05-31 17:49:28 -07:00
2012-03-24 10:08:39 -07:00
2012-03-29 19:52:48 +08:00
2012-05-07 15:02:26 +02:00
2012-05-10 19:54:35 -07:00
2012-04-05 05:08:04 -04:00
2012-04-14 15:24:26 -04:00
2012-03-28 15:58:21 -07:00
2012-04-17 00:18:36 -04:00
2012-03-17 21:51:34 +01:00
2012-05-25 09:18:59 -07:00
2012-02-20 19:46:36 +11:00
2012-05-02 14:15:27 -05:00
2012-03-28 16:36:44 +02:00
2012-05-25 12:46:23 +05:30
2012-05-21 15:09:37 +02:00
2012-04-04 17:13:43 -07:00
2012-05-31 17:49:31 -07:00
2012-05-07 14:35:00 +02:00
2012-05-09 15:10:45 +02:00
2012-05-09 15:10:45 +02:00
2012-05-10 12:00:56 +02:00
2012-02-22 18:15:39 +05:30
2012-05-01 08:58:06 -04:00
2012-05-28 19:13:50 -03:00
2012-04-30 15:30:18 -07:00
2012-03-06 21:27:24 +01:00
2012-03-04 17:54:34 -05:00
2012-06-01 12:12:01 -04:00
2012-07-10 18:06:44 -07:00
2012-06-27 15:42:24 -07:00
2012-05-31 17:49:32 -07:00
2012-05-05 21:50:41 +02:00
2012-05-29 23:28:33 -04:00
2012-03-31 16:03:16 -04:00
2012-05-04 16:25:29 -07:00
2012-06-01 16:57:51 -07:00
2012-02-19 10:30:57 -08:00
2012-03-20 21:29:32 -04:00
2012-05-21 20:27:36 -07:00
2012-03-18 22:15:39 +01:00
2012-05-24 12:57:47 -07:00
2012-05-14 09:47:23 -03:00
2012-05-15 11:34:08 -04:00
2012-06-03 14:50:19 -07:00
2012-04-18 13:46:42 -07:00
2012-05-30 21:04:53 -04:00
2012-03-04 17:54:34 -05:00
2012-05-10 15:55:43 -04:00
2012-05-16 20:00:27 -04:00
2012-04-25 12:25:05 +02:00
2012-04-04 09:25:44 -07:00
2012-04-09 11:14:50 -07:00
2012-05-29 22:33:56 -04:00
2012-05-15 08:22:04 +02:00
2012-05-21 15:09:33 +02:00
2012-02-28 17:10:21 +00:00
2012-03-19 17:56:18 -07:00
2012-04-27 10:46:45 +08:00
2012-05-18 16:48:35 -06:00
2012-05-22 11:32:31 +02:00
2012-04-30 10:39:17 +02:00
2012-03-24 10:08:39 -07:00
2012-05-29 16:22:21 -07:00
2012-05-25 12:48:26 -04:00
2012-03-16 10:38:24 -04:00
2012-03-16 10:38:24 -04:00
2012-03-16 10:38:24 -04:00
2012-05-22 16:02:13 -07:00
2012-03-26 21:47:19 +02:00
2012-03-26 21:47:19 +02:00
2012-03-26 21:47:19 +02:00
2012-05-12 14:28:18 +02:00
2012-06-04 16:49:43 +02:00
2012-05-12 14:28:15 +02:00
2012-05-12 14:28:14 +02:00
2012-03-26 21:47:19 +02:00
2012-05-30 10:55:34 +02:00
2012-05-17 19:06:13 -04:00
2012-03-04 17:54:34 -05:00
2012-03-28 15:58:21 -07:00
2012-06-21 16:43:28 +02:00
2012-05-22 17:13:11 -04:00
2012-03-27 22:45:26 -04:00
2012-06-19 21:39:50 +02:00
2012-04-15 13:06:05 -04:00
2012-04-15 13:06:05 -04:00
2012-03-04 20:41:38 -05:00
2012-04-28 22:21:51 -04:00
2012-05-01 09:30:55 -04:00
2012-06-26 21:09:39 -07:00
2012-03-24 10:08:39 -07:00
2012-06-30 01:08:00 -07:00
2012-05-17 15:49:51 -04:00
2012-06-12 15:25:46 -07:00
2012-05-30 14:02:25 +02:00
2012-03-09 10:56:35 -08:00
2012-05-23 22:11:12 -04:00
2012-03-04 17:54:34 -05:00
2012-03-20 12:47:48 +01:00
2012-05-22 18:08:08 +02:00
2012-05-07 10:58:57 -06:00
2012-03-20 12:47:47 +01:00
2012-05-08 19:38:31 +02:00
2012-05-31 17:49:31 -07:00
2012-03-16 10:38:24 -04:00
2012-03-16 10:38:24 -04:00
2012-07-11 02:39:24 -07:00
2012-05-23 13:49:23 -04:00
2012-06-30 15:56:40 -07:00
2012-05-19 12:41:56 -06:00
2012-04-09 11:16:55 -07:00
2012-05-27 08:12:42 -04:00
2012-04-29 18:21:10 -04:00
2012-05-15 23:34:37 +02:00
2012-03-13 22:45:25 -04:00
2012-02-28 20:01:08 +01:00
2012-03-16 10:38:24 -04:00
2012-05-29 16:22:32 -07:00
2012-03-08 10:50:35 -08:00
2012-05-31 17:49:30 -07:00
2012-05-31 17:49:32 -07:00
2012-04-12 18:35:58 -07:00
2012-05-29 16:22:21 -07:00
2012-05-31 17:49:26 -07:00
2012-05-31 17:49:30 -07:00
2012-05-23 22:11:31 -04:00
2012-03-08 10:50:35 -08:00
2012-05-11 10:56:56 +01:00
2012-03-29 17:41:25 -05:00
2012-04-19 19:17:30 -07:00
2012-05-31 17:49:28 -07:00
2012-06-18 15:55:21 -07:00
2012-03-04 17:54:34 -05:00
2012-06-04 11:43:00 -04:00
2012-05-16 16:08:28 -03:00
2012-05-17 21:06:01 -03:00
2012-05-01 09:30:55 -04:00
2012-05-29 16:22:29 -07:00
2012-05-29 16:22:32 -07:00
2012-03-23 16:58:34 -07:00
2012-05-29 16:22:31 -07:00
2012-05-29 23:28:41 -04:00
2012-05-07 15:32:23 -04:00
2012-03-28 18:30:03 +01:00
2012-05-15 08:08:31 -07:00
2012-03-23 16:58:33 -07:00
2012-03-26 20:40:59 +04:00
2012-04-09 12:23:04 -04:00
2012-03-23 16:58:31 -07:00
2012-03-16 10:38:24 -04:00
2012-03-15 21:41:34 +01:00
2012-05-07 22:58:09 -04:00
2012-07-01 03:34:50 -07:00
2012-05-09 13:58:06 -07:00
2012-05-29 16:22:28 -07:00
2012-03-04 17:54:34 -05:00
2012-05-29 16:22:20 -07:00
2012-05-22 15:20:28 -04:00
2012-03-21 17:54:57 -07:00
2012-07-01 03:34:50 -07:00
2012-05-16 15:24:36 -04:00
2012-05-16 15:23:46 -04:00
2012-05-29 16:22:28 -07:00
2012-06-20 14:39:35 -07:00
2012-06-01 10:34:35 -07:00
2012-05-29 16:22:20 -07:00
2012-05-29 16:22:26 -07:00
2012-05-01 12:54:54 +02:00
2012-03-26 12:50:52 +10:30
2012-06-08 14:58:13 +09:30
2012-05-31 17:49:27 -07:00
2012-03-27 08:26:34 +01:00
2012-05-08 16:33:56 -07:00
2012-04-15 13:06:04 -04:00
2012-05-29 22:33:55 -04:00
2012-02-24 01:42:07 -08:00
2012-07-05 03:06:44 -07:00
2012-04-24 00:16:24 -04:00
2012-06-16 15:40:02 +02:00
2012-06-16 15:40:02 +02:00
2012-06-22 02:49:52 +02:00
2012-06-29 16:46:06 -07:00
2012-06-04 21:34:30 +02:00
2012-05-26 14:17:30 -04:00
2012-06-18 08:45:16 -04:00
2012-05-24 16:15:48 -04:00
2012-03-05 15:26:47 -05:00
2012-02-16 14:55:27 -05:00
2012-05-09 15:17:43 -04:00
2012-06-05 18:38:47 -04:00
2012-03-04 17:54:34 -05:00
2012-06-26 09:32:28 +02:00
2012-06-26 21:06:11 -07:00
2012-03-23 16:58:31 -07:00
2012-02-16 06:11:23 -07:00
2012-03-24 10:41:37 -07:00
2012-05-18 16:48:36 -06:00
2012-05-12 14:28:17 +02:00
2012-04-17 16:37:22 -06:00
2012-05-07 22:58:09 -04:00
2012-03-15 23:28:19 +08:00
2012-05-21 14:31:48 +01:00
2012-03-05 08:09:09 -07:00
2012-04-18 15:07:53 -07:00
2012-05-02 01:14:40 +02:00
2012-02-23 09:39:23 +02:00
2012-05-29 16:22:24 -07:00
2012-03-16 10:38:24 -04:00
2012-03-14 17:25:56 +08:00
2012-03-21 17:55:01 -07:00
2012-03-24 10:08:39 -07:00
2012-05-21 15:09:33 +02:00
2012-05-29 16:22:18 -07:00
2012-03-28 18:30:03 +01:00
2012-06-20 14:39:36 -07:00
2012-05-09 15:57:01 +02:00
2012-02-23 12:05:21 -08:00
2012-06-13 13:11:39 -07:00
2012-05-14 14:15:32 -07:00
2012-06-06 17:08:00 +02:00
2012-07-09 00:10:56 -07:00
2012-05-15 14:59:28 -07:00
2012-04-29 13:12:42 -07:00
2012-07-04 13:07:05 +02:00
2012-05-12 15:53:42 -04:00
2012-05-18 20:45:26 +02:00
2012-03-28 23:31:24 +02:00
2012-05-01 21:28:38 +02:00
2012-05-01 21:25:49 +02:00
2012-05-01 21:28:38 +02:00
2012-03-23 16:58:38 -07:00
2012-03-04 17:54:34 -05:00
2012-05-04 21:15:52 -07:00
2012-03-04 20:41:38 -05:00
2012-03-04 20:41:38 -05:00
2012-03-04 20:41:38 -05:00
2012-06-07 14:43:55 -07:00
2012-03-01 10:28:04 +01:00
2012-05-07 17:03:27 -07:00
2012-05-15 14:59:28 -07:00
2012-06-13 16:52:40 -07:00
2012-04-04 05:28:45 -04:00
2012-04-14 11:13:21 +10:00
2012-06-11 12:26:15 +08:00
2012-03-20 21:29:38 -04:00
2012-05-03 03:29:34 -07:00
2012-06-05 10:46:40 -07:00
2012-05-17 15:18:37 +02:00
2012-04-24 20:54:49 -07:00
2012-05-11 10:14:21 -07:00
2012-06-06 20:43:28 -07:00
2012-06-06 20:43:28 -07:00
2012-05-14 17:40:05 +01:00
2012-03-24 10:08:39 -07:00
2012-03-20 21:29:43 -04:00
2012-03-20 21:29:42 -04:00
2012-03-04 17:54:34 -05:00
2012-03-06 19:14:37 +02:00
2012-05-29 16:22:27 -07:00
2012-03-16 10:38:24 -04:00
2012-04-23 21:17:51 -04:00
2012-05-31 17:49:31 -07:00
2012-05-31 17:49:31 -07:00
2012-05-29 16:22:21 -07:00
2012-05-12 15:15:20 -04:00
2012-05-29 16:22:33 -07:00
2012-07-10 22:40:13 -07:00
2012-03-28 18:30:03 +01:00
2012-03-07 11:32:21 +00:00
2012-03-04 17:54:34 -05:00
2012-06-08 14:59:29 -07:00
2012-04-18 12:24:50 +10:00
2012-06-01 10:37:01 -04:00
2012-03-24 10:08:39 -07:00
2012-05-04 15:13:54 -07:00
2012-05-23 17:12:06 -07:00
2012-04-18 15:57:31 -07:00
2012-03-16 10:38:24 -04:00
2012-04-09 17:39:34 +09:00
2012-05-17 19:02:14 -04:00
2012-04-04 09:25:42 -07:00
2012-04-11 12:05:50 +09:00
2012-02-16 17:08:09 -05:00
2012-05-22 19:07:55 +09:00
2012-05-15 14:59:29 -07:00
2012-06-01 12:58:52 -04:00
2012-02-24 11:42:50 -08:00
2012-06-15 15:30:15 -07:00
2012-05-31 17:49:26 -07:00
2012-03-28 15:04:26 -07:00
2012-05-08 12:35:06 +02:00
2012-03-19 16:53:08 -04:00
2012-06-04 11:27:40 -04:00
2012-04-15 12:44:40 -04:00
2012-03-23 13:18:57 +01:00
2012-03-28 15:58:21 -07:00
2012-06-13 21:16:42 +02:00
2012-04-30 10:48:25 -07:00
2012-05-15 14:08:35 -07:00
2012-02-24 10:05:59 +01:00
2012-04-14 07:47:49 -07:00
2012-05-14 18:53:19 -04:00
2012-04-20 23:27:08 +02:00
2012-03-28 18:30:03 +01:00
2012-05-01 21:25:49 +02:00
2012-06-04 12:28:45 -07:00
2012-05-15 11:33:58 -04:00
2012-06-15 21:48:14 -07:00
2012-05-31 17:49:32 -07:00
2012-06-22 23:02:22 -07:00
2012-05-14 12:19:56 -07:00
2012-02-20 12:48:47 -08:00
2012-05-23 22:09:21 -04:00
2012-03-13 14:06:33 -04:00
2012-07-11 18:12:59 -07:00
2012-06-01 12:58:50 -04:00
2012-05-09 14:19:13 -07:00
2012-05-21 16:16:58 -07:00
2012-03-22 19:43:43 -07:00
2012-05-22 18:27:32 -07:00
2012-03-20 21:29:40 -04:00
2012-05-23 22:11:31 -04:00
2012-02-24 10:05:59 +01:00
2012-03-04 17:54:34 -05:00
2012-03-08 11:38:51 -08:00
2012-05-10 11:24:03 -07:00
2012-06-02 15:21:43 -07:00
2012-05-30 21:04:50 -04:00
2012-04-26 02:01:39 -07:00
2012-04-14 13:25:48 +02:00
2012-05-21 09:00:03 -07:00
2012-05-03 03:28:39 -07:00
2012-05-09 13:57:30 -07:00
2012-05-20 08:27:44 -03:00
2012-04-10 15:00:41 -03:00
2012-06-21 09:32:16 +01:00
2012-05-03 09:30:55 +01:00
2012-06-11 15:47:21 -03:00
2012-05-22 12:16:16 +09:30
2012-03-27 16:30:09 -07:00
2012-04-15 03:23:31 -04:00
2012-02-19 09:50:20 -06:00
2012-03-31 08:09:50 +05:30
2012-04-25 21:26:33 -07:00
2012-04-26 12:34:58 -07:00
2012-04-24 16:14:14 -07:00
2012-03-28 18:30:03 +01:00
2012-05-30 07:55:31 +02:00
2012-03-02 10:51:00 +01:00
2012-05-06 13:43:41 +08:00
46d3ceabd8