Why one path is rarely enough
The Internet was built with redundancy in mind — multiple routes almost always exist between two endpoints. Yet nearly all end-to-end communication today uses a single path at a time. The reason is practical: even minor differences between paths can cause packet reordering and other degradation, so routers handle path selection and endpoints stay on one route.
Multi-Path TCP (MPTCP) challenges that model by exploiting devices with multiple network interfaces — a smartphone with both Wi-Fi and cellular, for instance — to use several paths for a single connection. The protocol has a long history (see RFC 8684 for the full specification) and remains a major extension to classic TCP. Historically, significant TCP extensions have failed to gain traction, but MPTCP differs: it is largely an OS-level feature, so enabling it should require few changes from applications. The trade-off is maturity — MPTCP is still somewhat rough around the edges, and whether it beats plain TCP depends heavily on the use case.
Subflows and what they enable
MPTCP's defining innovation is the subflow. Where classic TCP binds one connection to one 5-tuple, MPTCP lets a single connection be backed by multiple TCP subflows, each traversing a different path. On Linux, ss -M shows the subflow activity:
marek$ ss -tMn dport = :443 | cat
tcp ESTAB 0 0 192.168.2.143%enx2800af081bee:57756 104.28.152.1:443
tcp ESTAB 0 0 192.168.1.149%wlp0s20f3:44719 104.28.152.1:443
mptcp ESTAB 0 0 192.168.2.143:57756 104.28.152.1:443Decoupling connection lifetime from flow lifetime unlocks two capabilities that classic TCP lacks:
- Aggregation: Multiple subflows can pool bandwidth across interfaces, such as bonding two NICs in a data center. Linux now ships a BLESS-like MPTCP scheduler, and the macOS stack offers an "aggregation" mode, though how practical link aggregation is in the wild remains debatable. Projects like OpenMPTCProuter are nonetheless pursuing it.
- Mobility: A TCP stream normally dies when its underlying interface disappears — think of walking out of Wi-Fi range with a smartphone. MPTCP can create and destroy subflows over a connection's lifetime, surviving network changes without breaking the session.
The mobility case is the stronger driver. It is especially valuable for protocols like SSH, where maintaining a session while roaming between networks is very desirable — a scenario QUIC cannot cover since its multipath extensions are still a draft.
The state of implementations
Only two major implementations exist: Linux (in-kernel since v5.6, but practically requiring v6.1 or newer) and Apple platforms (iOS since 7, macOS since 10.10). Android is not supported. Linux is typically the server-side stack and Apple devices the client side; configuring Linux as a client is possible but not trivial. Two different Linux APIs have existed historically, and a lot of online documentation is stale — the Linux kernel docs and the mptcp.dev site are the reliable references.
Linux as a server
MPTCP's design is conceptually straightforward. During the initial TCP handshake, each peer can advertise additional addresses and ports for future subflows, via two mechanisms. First, the handshake SYN carries an MPTCP option with a flag bit [C] — the "Do not attempt to establish new subflows to this address and port" bit.
When that bit is cleared, the receiving peer may reconnect to the sender's IP/port. Typically, a server allows clients to reuse its IP/port, while the client does not listen and rejects inbound reconnections. In anycast setups such as Cloudflare's, reconnecting to the same server IP/port may land on a different physical machine, so the server sets the [C] flag. On Linux:
# Linux server sysctl - useful for ECMP or Anycast servers
$ sysctl -w net.mptcp.allow_join_initial_addr_port=0
The second advertising mechanism is the ADD-ADDR MPTCP signal, sent during a connection's lifetime to announce additional listening IP/port pairs. Linux manages this with the ip mptcp endpoint ... signal command:
# Linux server - extra listening address
$ ip mptcp endpoint add 192.51.100.1 dev eth0 port 4321 signal
An ADD-ADDR is typically relayed in an ACK packet, as captured below:
host > host: Flags [.], ack 1, win 8, options [mptcp 30 add-addr v1 id 1 192.51.100.1:4321 hmac 0x...,nop,nop], length 0
That the client can also issue ADD-ADDR messages may sound unusual, but it is legal — though in practice no one, or only the server, does so.
The code change to launch an MPTCP socket on Linux is minimal — replace IPPROTO_TCP with IPPROTO_MPTCP:
IPPROTO_MPTCP = 262
sd = socket(AF_INET, SOCK_STREAM, IPPROTO_MPTCP)
Some socket API surface is still missing, however: certain setsockopt options like TCP_USER_TIMEOUT are not yet implemented, and MPTCP is incompatible with kTLS.
Path managers and schedulers
Once endpooints know about each other's addresses, MPTCP performs its magic through two independent components. The path manager decides whether to establish additional subflows, while the scheduler decides which subflow to send data on. Both peers have path managers, but in practice only the client uses one.
The path manager walks a tightrope: spawning too few subflows forfeits the benefits of multipath, while spawning too many wastes resources. This balancing act is where MPTCP implementations get complicated.
Linux as a client
On Linux, the path manager is an OS feature rather than an application one. It needs explicit configuration of which IP addresses and interfaces qualify as sources for new subflows. The ip mptcp endpoint ... subflow command handles that:
$ ip mptcp endpoint add dev wlp1s0 192.0.2.3 subflow # Linux client
Two additional flags can extend an endpoint's role: "backup" marks a subflow for standby use and "fullmesh" allows the endpoint to be used in multiple concurrent subflow setups. Since maintaining ip mptcp endpoints on a dynamic client is tedious — entries must be added and removed as networks change — NetworkManager has managed these by default since version 1.40:
ubuntu$ cat /etc/NetworkManager/conf.d/95-mptcp.conf
# set "subflow" on all managed "ip mptcp endpoints". 0x22 is the default.
[connection]
connection.mptcp-flags=0x22
Path-manager "limit" settings cap both how many subflows a peer will create per MPTCP connection and how many received ADD-ADDR messages it will retain:
$ ip mptcp limits set subflow 4 add_addr_accepted 2 # Linux client
Testing the mobility scenario on Ubuntu 22 is telling. On a kernel v6.12 laptop, toggling Wi-Fi and Ethernet on and off repeatedly kept a reliable MPTCP connection alive. On Ubuntu's v6.8 kernel, the same exercise failed — see the reported issue. Part of the problem is that Linux's default client path manager only reacts when the server clears the [C] flag; server-advertised ADD-ADDR messages do not spur new subflows unless the endpoint is flagged fullmesh. The transport code appears solid, but the path manager needs sharper heuristics.
Custom path managers are still elusive
Linux supports two styles of path manager: the in-kernel default and a userspace netlink daemon. In principle, the modular design allows a custom implementation:
$ sysctl -w net.mptcp.pm_type=1 # use userspace path manager
In practice, no serious userspace path manager exists. The available implementations perform minimal work, and the netlink API for MPTCP path management remains immature, with issues still open upstream.
The scheduler front
Linux currently exposes one built-in default scheduler, which handles basic packet-loss driven failover. Extensible BPF-based schedulers are on the roadmap but are still a work in progress.
macOS and iOS
Apple's approach inverts the Linux design. macOS and iOS expose raw MPTCP APIs built on connectx(), leaving path management to the application. The low level grants fine control — see the example that establishes one connection with two subflows:
int sock = socket(AF_MULTIPATH, SOCK_STREAM, 0);
connectx(sock, ..., &cid1);
connectx(sock, ..., &cid2);
That power has a price: applications must listen for network changes themselves. Higher-level APIs alleviate the burden. The NWConnection API in C (via nw_parameters_set_multipath_service) and the Swift Network.framework are the common routes:
let parameters = NWParameters.tcp
parameters.multipathServiceType = .interactive
let connection = NWConnection(host: host, port: port, using: parameters)
The API offers three MPTCP service modes:
- Handover Mode: Prefers Wi-Fi and only uses cellular when the system's Wi-Fi Assist feature determines the Wi-Fi signal is too weak.
- Interactive Mode: Used for Siri — minimizes latency for low-bandwidth flows.
- Aggregation Mode: Enables resource pooling but is restricted to developer accounts and not broadly deployable.
Apple's MPTCP stack deeply integrates with the Wi-Fi Assist feature, even if official docs are thin. In hands-on testing, both the cleared [C] bit approach and the ADD-ADDR scenario worked reliably.
For those who want to experiment with MPTCP today, the setup is approachable: Linux works well as a server, and Apple devices are the better clients out of the box. Linux as a client requires kernel 6.1 or newer and the right path-manager flags, and even then some mobility flows work better than others.
IPv6 and MPTCP: A Practical Constraint
Deploying MPTCP over IPv6 introduces a significant limitation. IPv6 addresses are long, and MPTCP relies on the space-constrained TCP Extensions field. Consequently, there is insufficient room for ADD-ADDR messages when TCP timestamps are enabled. If your network environment requires both MPTCP and IPv6, this is a critical factor to evaluate during planning.
Current State and Recommendations
MPTCP is one of the few serious TCP extensions that can be deployed today, and that alone makes it an exciting development. However, the current implementations carry notable restrictions. Based on practical experimentation, the only scenario where MPTCP delivers clear value today is when Linux operates as the server, macOS or iOS acts as the client, and the workload is interactive. Linux can be configured to work as a client as well, but only with additional effort.
The Linux community has accomplished substantial engineering to reach this point. Yet, for an out-of-the-box experience suited to demanding production use, the ecosystem is not quite ready. The prospect of implementing the Path Manager and Scheduler in BPF is particularly promising, and there is good reason to believe Linux will develop a strong MPTCP client story in the near future.
Looking Ahead
After 15 years of development, MPTCP's long-term success remains an open question. Meanwhile, Multi-Path QUIC is under active development, though it is still further away from practical, deployable use than MPTCP currently is. If you have a specific use case that could benefit from MPTCP support at Cloudflare, we welcome your feedback on the matter. We also extend our thanks to Matthieu Baerts for his substantial help in reviewing this material.
