Enrichment

Every flow record is decoded, scaled and enriched in memory as it arrives, before it’s stored or counted. None of this waits on a database or a lookup over the network.

Formats

NetFlow v9, IPFIX and sFlow v5, on any of the flow ports. Heimnull learns and keeps each exporter’s templates, so records decode right after a restart without waiting for the router to send its templates again.

Two kinds of records come in:

• Flow records (NetFlow v9, IPFIX flows): a flow’s addresses, ports, protocol, bytes and packets over a period, from the router’s flow cache.
• Packet samples (IPFIX inline monitoring, sFlow): the first bytes of one sampled packet. Heimnull parses the headers (Ethernet, VLAN and MPLS, IPv4 and IPv6 with extension headers, TCP, UDP, SCTP, ICMP, fragments) and stores each as a one-packet flow.

Sampling

Bytes and packets are multiplied by the sampling rate when they’re stored, so every chart shows estimated real traffic. The rate comes from the router (in the records or an options template) or from the exporter’s settings: see Exporters.

Timestamps

• Clock skew. Each exporter’s clock is compared with the server’s. If it’s off by more than the threshold (2 seconds), its timestamps are corrected, and the exporter shows a warning.
• Implausibly old records (ended more than 2 hours before they arrived: a broken clock) are stored at their arrival time and counted.
• Export lag, from a flow’s end to its arrival, is measured per exporter.

Direction

From your networks: inbound, outbound, internal or transit. When neither address is yours, interface roles decide: traffic entering on a Transit or Peering interface is inbound, and leaving on one is outbound.

AS numbers and prefixes

For each remote address, Heimnull finds the routed prefix and its origin AS:

1. from the exporter’s records, if you trust its AS numbers (Configure → Exporters);
2. otherwise from routed-prefix tables built from full BGP tables by NoToolkit’s collectors, refreshed every 6 hours and bundled in the image.

AS names come from PeeringDB, then the registries. Heimnull doesn’t learn routes from your routers: the datasets cover the internet’s routing table without the memory a full table costs.

Countries

From DB-IP’s free country database (IP Geolocation by DB-IP, CC BY 4.0), bundled and refreshed monthly. Stored for the remote address, and for source and destination.

Applications

Each flow is labeled with an application, a category and the protocol under it by the classification rules, using the addresses, AS, ports, protocol, direction, interfaces and the record’s shape. Reverse DNS names of the busiest remote addresses are resolved in the background and used by reverse DNS rules for later flows; DNS never holds up ingest.

NAT

• MikroTik records carry the NAT translation, so flows through NAT name the internal host behind your public address.
• Juniper SRX firewalls can stream their session logs, which Heimnull matches to the flows: see Juniper SRX.

The internal host is stored as the flow’s host behind NAT (local_host_ip, nat-host in filters). Detection and mitigation still act on the public address, which is what’s routed to you.

Packet counts

Linux-based routers (MikroTik) count TCP segments the kernel merged as one packet, so a record can claim one packet of nearly 3,000 bytes. A record averaging more bytes per packet than the exporter’s Largest packet (1500) gets its packet count recomputed from its bytes. Bytes are never changed.

SNMP

With SNMP set up on an exporter, Heimnull polls its interface table every minute: names, descriptions, speeds, status, and traffic, error and discard counters. Interfaces are keyed by name, so history follows an interface when a reboot renumbers it, and a reboot is noticed within seconds. SNMP gives the Interfaces page its utilization and 95th percentiles, the accuracy check of flows against counters, Planning, and congestion escalation.

Datasets

Dataset	From	Refreshed
Routed prefixes and origin ASes	NoToolkit	Checked every 6 hours
AS names and countries	iptoasn.com, PeeringDB	Daily
Countries	DB-IP Lite	Monthly (checked weekly)
Cloud provider ranges	AWS, Google, Azure, Cloudflare, Fastly, Oracle, GitHub, DigitalOcean	Daily (Azure weekly)
Service address lists	Microsoft 365, Zoom, Steam, Telegram, Tailscale, Atlassian, Zscaler, Okta, Mullvad	Daily

Every dataset except PeeringDB is bundled in the image, so a new server works offline from the start. A download is checked before it’s used (it must parse, and its size can’t change by more than a set share at once) and the previous copy is kept. A failed download is retried every minute when a retry can help. System → Datasets shows each one’s state, and a failure can be notified (dataset_failed).