# mwan4 - Multi-WAN with nftables

## Migration from mwan3 to mwan4

### Configuration File Changes

The mwan4 configuration file format is **mostly backward compatible** with mwan3, with the following changes:

#### 1. `flush_conntrack` Option Format (Breaking Change)

**mwan3 format:**
```
config interface 'wan'
	option flush_conntrack 'always'  # or 'never'
```

**mwan4 format:**
```
config interface 'wan'
	list flush_conntrack 'ifup'    # optional
	list flush_conntrack 'ifdown'  # optional
```

**Migration:** The uci-defaults script (`/etc/uci-defaults/90-mwan4`) automatically converts:
- `always` → adds both `ifup` and `ifdown` to the list
- `never` → removes the option entirely

This change provides more granular control over when connection tracking is flushed.

#### 2. `family` Configuration on Interfaces - Dual-Stack Support (New)

**Single family per interface (legacy approach):**
```
config interface 'wan'
	option enabled '1'
	list family 'ipv4'
	list track_ip '1.1.1.1'

config interface 'wan6'
	option enabled '1'
	list family 'ipv6'
	list track_ip '2606:4700:4700::1111'
```

**Dual-stack on single interface (NEW - recommended):**
```
config interface 'wan'
	option enabled '1'
	list family 'ipv4'
	list family 'ipv6'
	list track_ip '1.1.1.1'           # IPv4 tracking
	list track_ip '2606:4700:4700::1111'  # IPv6 tracking
	option reliability '2'
```

**Benefits of dual-stack configuration:**
- Single configuration for both IPv4 and IPv6
- No duplicate settings to maintain
- Automatic family detection from track IPs
- Independent tracking per family
- Single interface ID with family-specific internal chains

**Backward compatibility:** The old `option family` format is automatically migrated to `list family` by the uci-defaults script.

#### 3. Renamed Config Sections: `member` → `route`, `policy` → `strategy` (Breaking Change)

**mwan3 / old mwan4 format:**
```
config member 'wan_m1_w3'
	option interface 'wan'
	option metric '1'
	option weight '3'

config policy 'balanced'
	list use_member 'wan_m1_w3'

config rule 'https'
	option use_policy 'balanced'
```

**New mwan4 format:**
```
config route 'wan_m1_w3'
	option interface 'wan'
	option metric '1'
	option weight '3'

config strategy 'balanced'
	list use_route 'wan_m1_w3'

config rule 'https'
	option use_strategy 'balanced'
```

**Summary of renames:**
| Old | New | Description |
|-----|-----|-------------|
| `config member` | `config route` | Weighted route entry (interface + metric + weight) |
| `config policy` | `config strategy` | Routing strategy (how to distribute/failover traffic) |
| `list use_member` | `list use_route` | Route references within a strategy |
| `option use_policy` | `option use_strategy` | Strategy reference within a rule |

**Migration:** The uci-defaults script (`/etc/uci-defaults/92-mwan4-rename`) automatically converts old section types and option names in-place.

#### 4. `family` on Rules Uses List Format (New)

**Allows rules to be IPv4 or IPv6 specific (using list format):**
```
config rule 'default_rule_v4'
	option dest_ip '0.0.0.0/0'
	option use_strategy 'balanced'
	list family 'ipv4'

config rule 'default_rule_v6'
	option dest_ip '::/0'
	option use_strategy 'balanced'
	list family 'ipv6'
```

Rules without a `family` list will apply to both IPv4 and IPv6 (if the rule is compatible with both).

**Migration:** The uci-defaults script automatically converts the old `option family` format to `list family`.

#### 5. `mmx_mask` Now Configurable (New)

**Customizable packet marking mask:**
```
config globals 'globals'
	option mmx_mask '0x3F00'
```

Previously hardcoded, the marking mask can now be customized in the globals section. This allows integration with other routing systems that may use different parts of the packet mark.

**Default value:** `0x3F00` (supports up to 63 interfaces)

### Unchanged Configuration Sections

The following configuration sections remain fully compatible:

- **Interface tracking options:** `track_ip`, `reliability`, `count`, `timeout`, `interval`, `down`, `up`, `size`, `max_ttl`, `failure_interval`, `recovery_interval`, `check_quality`, `failure_latency`, `recovery_latency`, `failure_loss`, `recovery_loss`
- **Route definitions (formerly member):** `interface`, `metric`, `weight`
- **Strategy definitions (formerly policy):** `use_route` lists, `last_resort`
- **Rule structure:** `sticky`, `timeout`, `dest_port`, `src_port`, `dest_ip`, `src_ip`, `proto`, `use_strategy`, `ipset`, `logging`

### Example Migration

**mwan3 config:**
```
config interface 'wan'
	option enabled '1'
	list track_ip '1.1.1.1'
	option flush_conntrack 'always'
```

**mwan4 config (after automatic migration):**
```
config interface 'wan'
	option enabled '1'
	list family 'ipv4'
	list track_ip '1.1.1.1'
	list flush_conntrack 'ifup'
	list flush_conntrack 'ifdown'
```

### Dual-Stack Configuration Example

Here's a complete example showing how to configure a dual-stack WAN interface with both IPv4 and IPv6:

```
config interface 'wan'
	option enabled '1'
	list family 'ipv4'
	list family 'ipv6'
	list track_ip '1.1.1.1'
	list track_ip '8.8.8.8'
	list track_ip '2606:4700:4700::1111'
	list track_ip '2001:4860:4860::8888'
	option reliability '2'
	option count '1'
	option timeout '2'
	option interval '5'
	option down '3'
	option up '8'

config route 'wan_m1_w1'
	option interface 'wan'
	option metric '1'
	option weight '1'

config strategy 'balanced'
	list use_route 'wan_m1_w1'

config rule 'default_rule_v4'
	option dest_ip '0.0.0.0/0'
	option use_strategy 'balanced'
	list family 'ipv4'

config rule 'default_rule_v6'
	option dest_ip '::/0'
	option use_strategy 'balanced'
	list family 'ipv6'
```

**How it works:**
- The interface spawns two tracking instances: `track_wan_ipv4` and `track_wan_ipv6`
- IPv4 track IPs (1.1.1.1, 8.8.8.8) are automatically used for IPv4 tracking
- IPv6 track IPs (2606:..., 2001:...) are automatically used for IPv6 tracking
- Each family maintains independent online/offline status
- nftables chains are created per family: `mwan4_iface_in_wan_ipv4` and `mwan4_iface_in_wan_ipv6`
- Status is reported separately per family: `mwan4 status` will show both

## nftables Integration

mwan4 integrates with firewall4 by writing nftables rules into the shared `inet fw4` table. All chains and sets use the `mwan4_` prefix.

### Generated nft Files

mwan4 generates two persistent nftables files that are loaded by fw4 via the `ruleset-post` mechanism:

| File | Description |
|------|-------------|
| `/usr/share/nftables.d/ruleset-post/10-mwan4.nft` | Dynamic content rebuilt on hotplug events. Combines base chains and sets (`mwan4_prerouting`, `mwan4_output`, `mwan4_ifaces_in`, connected/custom/dynamic IP sets for both IPv4 and IPv6), per-interface chains (e.g. `mwan4_iface_in_wan_ipv4`, `mwan4_iface_in_wan_ipv6`), and strategy chains (e.g. `mwan4_strategy_balanced_ipv4`). |
| `/usr/share/nftables.d/ruleset-post/11-mwan4-rules.nft` | User rule chains (`mwan4_rules_ipv4`, `mwan4_rules_ipv6`) that map traffic matching criteria (dest/src IP, port, protocol) to strategy chains. Generated at init time and includes sticky session sets when configured. |

### Runtime Temp File

| File | Description |
|------|-------------|
| `/var/run/mwan4.nft.combined` | Temporary file used to validate the combined ruleset (`nft -c -f`) before installing the per-target files. Removed after each operation. |

### nftables Chain Naming

| Chain Pattern | Purpose |
|---------------|---------|
| `mwan4_prerouting` | Main hook at mangle priority — restores marks, processes interfaces, checks sets, applies rules |
| `mwan4_output` | Output hook at mangle priority — same logic for locally-generated traffic |
| `mwan4_ifaces_in` | Dispatches to per-interface chains based on input device |
| `mwan4_iface_in_{iface}_{family}` | Marks traffic arriving on a specific interface/family |
| `mwan4_strategy_{name}_{family}` | Implements a routing strategy (load balance / failover) |
| `mwan4_rules_{family}` | Evaluates user-defined rules and jumps to strategy chains |
| `mwan4_rule_{name}_{family}` | Per-rule sticky session chain (only when sticky is enabled) |

### nftables Set Naming

| Set Pattern | Purpose |
|-------------|---------|
| `mwan4_connected_{family}` | Directly connected networks — traffic to these is marked default |
| `mwan4_custom_{family}` | Networks from custom routing table lookups (`rt_table_lookup`) |
| `mwan4_dynamic_{family}` | Dynamically added networks |
| `mwan4_rule_{family}_{rule}` | Sticky session map (IP + mark → timeout) for a specific rule |

### Documentation

For full mwan3 configuration documentation (still largely applicable):
https://openwrt.org/docs/guide-user/network/wan/multiwan/mwan3#mwan3_configuration

## Getting Help

If things are not working as intended, please run the following commands and include their output in your post/issue (you can mask sensitive information):

```sh
ubus call system board
uci export network
uci export mwan4
mwan4 status
nft list table inet fw4
```

mwan4 integrates with firewall4 and writes its rules, chains and sets (all `mwan4_`-prefixed) into the shared `inet fw4` table — that's why `nft list table inet fw4` is the relevant dump rather than a separate mwan4 table.

If the issue is specific to interface tracking or failover, also include:

```sh
ls /var/run/mwan4/
logread -e mwan4 | tail -100
```