Arkime Architecture

Here are some sample deployments of Arkime for different network architectures. Most folks will probably run a hybrid of the following since no one solution fits all. The ability to scale capturing can be done horizontally by adding more capture machines, vertically by adding more cpus/disk, or both. We usually recommend scaling horizontally unless physically space constrained, and using a network packet broker in front of multiple machines. However it is possible to use big machines, with lots of cpu/disk, and run arkime-capture with more threads. There is a companion video for this page.

Legend/Info:

• A box represents a physical machine.
• It is possible to run multiple capture processes per machine, or have a single capture process listen to multiple interfaces - (FAQ Answer)
• Recommend “Big Data” style boxes for capture - (FAQ Answer)
• Run multiple OpenSearch/Elasticsearch processes per machine since each ES node should be configured at most to 30G - (FAQ Answer)
• Except for single host deployments, it is recommended/useful that all operator access flows through a single apache/viewer combination that can provide better authentication, logging, and a single choke point - (FAQ Answer)

Security

• All ES instances should have iptables for port 9200-920N and 9300-930N, where N is the number of ES instances per machine, and only allow the other OpenSearch/Elasticsearch, capture and viewer machines to connect
• All viewer hosts, except the apache/viewer box, should have iptables for port 8005 and only allow other viewer machines to connect. The viewer must listen on OS interface if using multiple machines
• The shared viewer instances can listen on localhost since only apache talks to it

Single Host

A single host deployment should usually only be used for demos and extremely low traffic networks. The read/write patterns of Arkime vs OpenSearch/Elasticsearch will tax most systems using spinning disks and is not recommended.

Multiple Hosts Monitoring Multiple Network Segments

Multiple Hosts Monitoring High Traffic Networks

Notes:

• Using a Network Packet Broker (NPB) allows traffic to be load balanced and recombined. This is especially useful in HA or asymmetric routing cases
• By using a NPB, other security devices can see the same traffic arkime sees
• When running multiple arkime-captures on the same host make sure the IO doesn’t over whelm the disk and other subsystems.
• Use a TAP with high traffic networks since many mirror ports drop traffic under heavy load
• Operators use an apache fronted viewer and don’t hit the other viewers directly. The apache provides authentication.
• Lock down ES and arkime viewer with iptables

Multiple Clusters

Notes:

• It is possible to use a single ES cluster using the prefix= ini configuration
• Operator uses apache fronted viewers and doesn’t hit the other viewers directly. The apache provides authentication. Can use virtual paths to route to different clusters.
• NPBs are recommended for high traffic networks

Remote Device capture

In some cases, it’s not practical or possible to physical co-locate a capture server near a device we want to instrument. If the device is reachable via ssh, we can use ssh to login to the remote device, initiate a capture and stream the raw pcap data back to a arkime server. This streaming pcap data is written into a UNIX pipe. One starts a arkime capture process using the pipe as the capture interface, resulting in real-time capture instrumentation of a remote device.

Diagram with Steps to implement:

Steps:

1. Create named pipe on the local capture machine, you can name whatever you want:
   mkfifo /tmp/10.1.2.3-ens3
2. Start capture on on the local capture machine:
   /opt/arime/bin/capture --copy -r /tmp/10.1.2.3-ens3
3. Use ssh to copy the data from remote to localhost machine, exclude capturing the traffic we are sending over ssh:
   ssh user@10.1.2.3 sudo /usr/sbin/tcpdump -s0 -i ens3 -w - not host 192.168.10.5 > /tmp/10.1.2.3-ens3
   (The sudo and full path to tcpdump might not be required, remember to update the ips.)

This is NOT the recommended way to use Arkime.