Set up network flow data monitoring | New Relic Documentation

Set up your network devices so they send network data to New Relic.

Add network flow data

Prerequisites and supported types of network flow data

New Relic prerequisites

A New Relic account. Don't have one? Sign up for free! No credit card required.
A New Relic account ID.
A New Relic license key.

Linux host prerequisites

Docker installed in a Linux host.
SSH access to the Docker host, with the ability to launch new containers.

Network flow data devices prerequisites

Configured network devices to send flow data to the host running the ktranslate docker container. Here's how to configure network flow data collection in some devices:
- NetFlow data
- sFlow data
  - F5 - BIG-IP
- jFlow data
  - Juniper - Junos

Network security prerequisites

Direction	Source	Destination	Ports	Protocol
Outbound	Docker host	`ktranslate` image on Docker Hub or Quay.io	443	TCP
Outbound	Docker host	New Relic Event API US Endpoint: `https://insights-collector.newrelic.com` EU Endpoint: `https://insights-collector.eu01.nr-data.net`	443	TCP
Outbound	Docker host	New Relic Log API US Endpoint: `https://log-api.newrelic.com` EU Endpoint: `https://log-api.eu.newrelic.com`	443	TCP
Inbound	Source devices for network flow data	Docker host	9995 (default)	UDP

Supported types of network flow data

Network flow monitoring supports the four primary types of network flow data and their derivatives. When running the ktranslate container, you will specify which major type you want to monitor using the -nf.source option.

Important

The ktranslate container only supports monitoring one type of network flow data type at a time. If you want to monitor several types, each will require a container.

IPFIX and NetFlow v9 can be sent to the same container, but we recommend running a separate container as a best practice.

Network flow data type	`-nf.source` value
IPFIX	`ipfix`
NetFlow v5	`netflow5`
NetFlow v9	`netflow9`
sFlow	`sflow`
AppFlow	`netflow5`
Argus	`netflow5`
cflowd	`netflow5`
J-Flow	`netflow5`
NetStream	`netflow5`
RFlow	`netflow5`
Cisco NSEL	`netflow9`

Scaling network flow collection

When planning your strategy for collecting network flows at scale, New Relic recommends 1 CPU per 2000 flows-per-second (120,000 flows-per-minute). Deciding whether to run more small containers to distribute load or fewer large containers to consolidate management is a matter of personal preference.

Set up network flow data monitoring in New Relic

Go to one.newrelic.com and click Add more data.
Scroll down until you see Network monitoring and click Network Flows.
Follow the steps in New Relic.
one.newrelic.com > Add more data > Network monitoring > Network Flows to set up network flow data monitoring.
Visualize your network performance data in New Relic.

If you prefer to do the setup manually, proceed with the following steps.

In your local machine, from a Linux host with Docker installed, download the ktranslate image by running one of the following:
- Docker Hub
  bash
```
$docker pull kentik/ktranslate:v2
```
- Quay.io
  bash
```
$docker pull quay.io/kentik/ktranslate:v2
```
Copy the snmp-base.yaml file to the local $HOME directory of your Docker user, and discard the container by running the following:
bash
```
$cd .
$id=$(docker create kentik/ktranslate:v2)
$docker cp $id:/etc/ktranslate/snmp-base.yaml .
$docker rm -v $id
```
In the snmp-base.yaml file, add your network flow devices inside the devices key with the following structure:
```
devices:
  flowDevice:
    device_name: edge-router
    device_ip: 10.10.1.254
    flow_only: true
    # Optional user tags
    user_tags:
      owning_team: net_eng
      environment: production
```
Tip
If you're already monitoring SNMP data devices that send network flow data, you don't need to add them in your snmp-base.yaml file a second time.

Run ktranslate to listen for network flows by running:

bash

$docker run -d --name ktranslate-sflow --restart unless-stopped --net=host \
>-v `pwd`/snmp-base.yaml:/snmp-base.yaml \
>-e NEW_RELIC_API_KEY=$YOUR_NR_LICENSE_KEY \
>kentik/ktranslate:v2 \
>  -snmp /snmp-base.yaml \
>  -nr_account_id=$YOUR_NR_ACCOUNT_ID \
>  ## If your account is located in Europe, add the following flag:
$  ## -nr_region=EU \
$  ## If you want to use FedRAMP, add the following flag to use the FedRAMP authorized endpoints:
$  ## -nr_region=GOV \
$  -metrics=jchf \
>  -tee_logs=true \
>  -flow_only=true \
>  -nf.source=sflow \
>  -service_name=sflow \
>  nr1.flow

Tip

This command assumes collection of sflow data. If you are collecting other flow types, you should change the suffix in the --name flag for the container and update the -nf.source and -service_name flags as necessary.

To get insights into system messages from your devices, setup network syslog collection.
Visualize your network performance data in New Relic.

Find and use your metrics

All network flow logs exported from the ktranslate container use the KFlow namespace, via the New Relic Event API. Currently, these are the default fields populated from this integration:

Attribute	Type	Description
`application`	String	The class of program generating the traffic in this flow record. This is derived from the lowest numeric value from `l4_dst_port` and `l4_src_port`. Common examples include `http`, `ssh`, and `ftp`.
`device_name`	String	The display name of the sampling device for this flow record.
`dst_addr`	String	The target IP address for this flow record.
`dst_as`	Numeric	The target Autonomous System Number for this flow record.
`dst_as_name`	String	The target Autonomous System Name for this flow record.
`dst_endpoint`	String	The target `IP:Port` tuple for this flow record. This is a combination of `dst_addr` and `l4_dst_port`.
`dst_geo`	String	The target country for this flow record, if known.
`in_bytes`	Numeric	The number of bytes transferred for ingress flow records.
`in_pkts`	Numeric	The number of packets transferred for ingress flow records.
`input_port`	Numeric	`If_Index` value for the interface at the source of this flow record.
`l4_dst_port`	Numeric	The target port for this flow record.
`l4_src_port`	Numeric	The source port for this flow record.
`output_port`	Numeric	`If_Index` value for the interface at the destination of this flow record.
`protocol`	String	The display name of the protocol used in this flow record, derived from the numeric IANA protocol number.
`provider`	String	This attribute is used to uniquely identify various sources of data from `ktranslate`. Network flow logs will always have the value of `kentik-flow-device`.
`sample_rate`	Numeric	Sampling rate applied by either the sampling device configuration, or the `sample_rate` argument in `ktranslate`.
`src_addr`	String	The source IP address for this flow record.
`src_as`	Numeric	The source Autonomous System Number for this flow record.
`src_as_name`	String	The source Autonomous System Name for this flow record.
`src_endpoint`	String	The source `IP:Port` tuple for this flow record. It's a combination of `src_addr` and `l4_src_port`.
`src_geo`	String	The source country for this flow record, if known.
`tcp_flags`	Numeric	TCP flags in this flow record.
`timestamp`	Numeric	The time, in Unix seconds, when this flow record was received by the New Relic Event API.