Router setup

MikroTik RouterOS

Export NetFlow v9 from a MikroTik router to Heimnull, let Heimnull blackhole attacked addresses on it over BGP, and avoid the RouterOS pitfalls.

RouterOS exports NetFlow v9 and IPFIX and accepts blackhole routes. It has no sFlow export and no FlowSpec, so on a MikroTik, Heimnull mitigates attacks with blackholes.

The examples use these addresses. Replace them with your own:

In the examples	Value
Heimnull	`10.0.0.10`
MikroTik	`10.0.0.1`
Your AS	`64512`
Your networks	`203.0.113.0/24` and `2001:db8:42::/48`

Before you start: FastTrack

FastTrack speeds up established connections by sending their packets around most of the router’s processing, and traffic-flow then sees little or none of that forwarded traffic. The default configuration turns it on (a fasttrack-connection rule in the firewall filter).

The symptom is traffic in Heimnull far below what the interfaces carry: on the Interfaces page, the flows-to-SNMP ratio sits well under 1, and long TCP downloads are missing. Attacks on established connections can go unseen too.

To see all forwarded traffic, disable the FastTrack rules:

/ip firewall filter disable [find action=fasttrack-connection]
/ipv6 firewall filter disable [find action=fasttrack-connection]

This raises the router’s CPU load, since every packet now goes through the full path. Check /system resource monitor at your busiest time before and after. Hardware-offloaded (L3 HW) traffic on switch chips doesn’t reach traffic-flow either. Check on your version whether your router offloads routing.

Flow export (RouterOS 7)

/ip traffic-flow
set enabled=yes interfaces=ether1 cache-entries=1M packet-sampling=yes sampling-interval=1 sampling-space=99 \
    active-flow-timeout=15s inactive-flow-timeout=5s
/ip traffic-flow ipfix
set dst-mac-address=no src-mac-address=no tcp-ack-num=no tcp-seq-num=no tcp-window-size=no ttl=no \
    ip-header-length=no ip-total-length=no udp-length=no igmp-type=no is-multicast=no ipv6-flow-label=no
/ip traffic-flow target
add dst-address=10.0.0.10 port=2055 src-address=10.0.0.1 version=9 v9-template-refresh=20 v9-template-timeout=1m

Interfaces. Only the WAN (ether1 here), so each packet is counted once. Both directions through it are exported, for IPv4 and IPv6, each in its own template.
Sampling. sampling-interval is how many packets are sampled in a row, and sampling-space how many are skipped. 1 and 99 sample 1 in 100, and the record then carries 100, which Heimnull applies.
- Always use an interval of 1 and a space of N−1. RouterOS reports the ratio as a whole number, rounded down: an interval of 5 and a space of 49 (1 in 10.8) is reported as 10, and traffic then reads about 7% low.
- At 1 in 500, a 10 Mbit/s link gives under one record a second, so the Live page has to average over about 30 seconds; at 1 in 100 that window is about 6 seconds. Check the router’s CPU before and after.
- On a router with little traffic, you can turn packet sampling off (packet-sampling=no): every packet is then counted.
Timeouts. The router exports a flow only when it has been idle for inactive-flow-timeout, or every active-flow-timeout while it lasts. Heimnull counts late records from their arrival, so the lag is the timeout:
- 5 seconds inactive gives about 6 seconds of delay on Live and about 10 seconds to detect an attack, against about 20 seconds at the 15-second default.
- Keep the active timeout at 30 seconds or less, or long flows show up in bursts.
Templates. v9-template-refresh counts packets, not time. On a quiet link that can be minutes apart, and records are dropped until the template arrives after a Heimnull restart. v9-template-timeout=1m bounds that.
Fields. Heimnull uses addresses, ports, protocol, TCP flags, ToS, ICMP type and code, interfaces, timestamps, the gateway, and the NAT fields, which name the inside host behind your WAN address. The fields turned off above are never used, so records stay smaller. Check on your version that this menu also shapes NetFlow v9 records; on the router we tested, it does.
No AS numbers. RouterOS doesn’t include AS numbers in its flow records. Heimnull resolves every remote address’s AS and routed prefix itself, and looks up WHOIS, registrant and RPKI details with NoToolkit. See Enrichment.
Source address. Heimnull identifies the router by the datagrams’ source address, so set src-address.
version=ipfix also works.

RouterOS counts received TCP packets after the kernel merges them, so a record can claim one packet of nearly 3,000 bytes. Heimnull recounts such records from their bytes (the exporter’s Largest packet setting, 1500 by default), so packet rates stay true.

RouterOS 6

RouterOS 6’s traffic-flow works with Heimnull, with one exception: packet sampling is unreliable. Turn it off.

/ip traffic-flow
set enabled=yes interfaces=ether1 packet-sampling=no active-flow-timeout=30s inactive-flow-timeout=5s
/ip traffic-flow target
add dst-address=10.0.0.10 port=2055 src-address=10.0.0.1 version=9

Why. RouterOS 6 writes the sampling rate in its records with its bytes in the wrong order: 1 in 10 arrives as a number above 160 million. Before v3.1.1, Heimnull multiplied every record by it, so a few control-plane packets on an idle link showed as gigabits. Heimnull v3.1.1 and later recognize the swapped value and read it the right way round, and the exporter’s page says so, but sampling without it is the dependable setup.
After turning sampling off, check the exporter in Heimnull (Configure → Exporters). If it shows a sampling rate other than 1, or says rates arrived byte-swapped, set Override rate to 1.
Check on your version which properties your RouterOS 6 release offers: packet-sampling isn’t in every release, and older ones don’t sample at all.
The FastTrack note above applies to RouterOS 6 too.
Upgrade to RouterOS 7 where you can: it’s what the rest of this guide is tested with.

BGP session with Heimnull (RouterOS 7)

Add the session in Heimnull (Configure → BGP) with the MikroTik’s address, the remote AS, Announce mitigations to this session on, and the IPv4 and IPv6 next hops the router should use for blackholes.

On the MikroTik (iBGP in this example):

/routing/bgp/connection
add name=heimnull remote.address=10.0.0.10 remote.as=64512 as=64512 local.role=ibgp \
    input.filter=heimnull-in output.filter-chain=heimnull-out

Check on your version: the property names remote.as, as, input.filter and output.filter-chain come from RouterOS v7 examples. MikroTik’s BGP page shows the add name=… remote.address=… local.role=… form, but not every property.
Send nothing. heimnull-out must reject everything. Heimnull accepts no routes from any session, but anything sent still sits in its BGP speaker’s memory (about 0.8 GiB per million routes).

Blackhole filter

Accept from Heimnull only host routes inside your networks that carry the blackhole community, and make them blackhole routes. Reject everything else.

/routing/filter/rule
add chain=heimnull-in rule="if (dst in 203.0.113.0/24 && dst-len == 32 && bgp-communities includes blackhole) { set blackhole yes; accept; }"
add chain=heimnull-in rule="if (dst in 2001:db8:42::/48 && dst-len == 128 && bgp-communities includes blackhole) { set blackhole yes; accept; }"
add chain=heimnull-in rule="reject"
add chain=heimnull-out rule="reject"

blackhole above is RFC 7999’s well-known community, 65535:666. It’s the default we recommend: set it on Heimnull’s BGP page as the default community.
If you use your own community, match it with a community list or a literal. Check the literal form on your version, for example bgp-communities includes 64512:666.
blackhole is a writable route property in RouterOS v7 filters. Check on your version whether it wants set blackhole yes or set blackhole true: both forms appear in RouterOS v7 examples.
Upstream RTBH. To pass the blackhole on to your transit providers, add their blackhole community in your export chain toward them, following each provider’s documentation.

Check it

In Heimnull, Mitigations → Mitigate an address → Preview shows exactly what the MikroTik will receive. Use Dry run first.
On the MikroTik, /routing/route print where blackhole should list the /32 while the blackhole is active, and nothing after it’s withdrawn. Check on your version for the exact print filter.
In Heimnull, each mitigation’s page shows what the session actually received, and every change is in the audit log.
On the Interfaces page (with SNMP set up on the exporter), the WAN’s flow totals should be close to its SNMP counters. If they’re far below, see FastTrack.