INI Settings

Arkime uses a tiered system for configuration variables. A tiered system allows Arkime to share one config file for many machines. The ordering of sections within the config file doesn't matter.

Order of config variables:

[optional] The section titled with the node name is used first. Arkime will always tag sessions with node: <node name>
[optional] If a node has a nodeClass variable, the section titled with the nodeClass name is used next. Sessions will be tagged with node:<node class name> which is useful if watching different network classes.
The section titled "default" is used last.

Basic Settings

Setting	Default	Description
bpf	EMPTY	The bpf filter used to reduce traffic. Used both on live and file traffic.
businessDayStart	EMPTY	If both businessDayStart and businessDayEnd are set, it displays a colored background on the sessions timeline graph during business hours. Values are in hours from midnight UTC. For example: 9am EST would be 13.
businessDayEnd	EMPTY	If both businessDayStart and businessDayEnd are set, it displays a colored background on the sessions timeline graph during business hours. Values are in hours from midnight UTC. For example: 5pm EST would be 21.
businessDays	1,2,3,4,5	Displays a colored background on the sessions timeline graph on only these days. businessDayStart and businessDayEnd must be set for these to be of use. Values are comma separted list of days of the week as numbers. The week starts at Sunday = 0 and ends on Saturday = 6. For example: Monday through Friday would be 1,2,3,4,5
certFile	EMPTY	Public certificate to use for https, if not set then http will be used. keyFile must also be set.
cronQueries	FALSE	Set on 1 viewer node per cluster, this viewer node will perform all the cron queries for the cluster.
dontSaveBPFs	EMPTY	Semicolon ';' separated list of bpf filters which when matched for a session causes the remaining pcap from being saved for the session. It is possible to specify the number of packets to save per filter by ending with a :num. For example dontSaveBPFs = port 22:5 will only save 5 packets for port 22 sessions. Currently only the initial packet is matched against the bpfs.
dontSaveTags	EMPTY	Semicolon ';' separated list of tags which once capture sets for a session causes the remaining pcap from being saved for the session. It is likely that the initial packets WILL be saved for the session since tags usually aren't set until after several packets. It is possible to specify the number of packets to save per filter by ending with a :num.
dropGroup	EMPTY	Group to drop privileges to. The pcapDir must be writable by this group or to the user specified by dropUser
dropUser	EMPTY	User to drop privileges to. The pcapDir must be writable by this user or to the group specified by dropGroup
espTimeout	600	For ESP sessions, Arkime writes a session record after this many seconds of inactivity since the last session save.
elasticsearch	http://localhost:9200	A comma separated list of urls to use to connect to the Elasticsearch cluster. If not using a VIP, a different Elasticsearch node can be specified for each Arkime node. If Elasticsearch requires a user/password those can be placed in the url also, http://user:pass@hostname:port
elasticsearchTimeout	300	Approximate timeout for most viewer requests to elasticsearch. Elasticsearch will automatically cancel any request after this expires.
esAdminUsers	EMPTY	A comma separated list of users that are allowed to use the ES Admin stats tab. This tab allows the user to change several of the ES settings from the UI.
esClientCert	EMPTY	(Since 2.0) The public key file to use for tls client authentication with elasticsearch. Must also set esClientKey.
esClientKey	EMPTY	(Since 2.0) The private key file to use for tls client authentication with elasticsearch. Must also set esClientCert.
esClientKeyPass	EMPTY	(Since 2.0) The password for the esClientKey setting.
freeSpaceG	5%	Delete pcap files when free space is lower then this. This does NOT delete the session records in the database. It is recommended this value is between 5% and 10% of the disk. Database deletes are done by the db.pl expire. Can also be specified using a percentage.
geoipASNFile	EMPTY	(Pre 1.0) Path to the maxmind geoip ASN file. Download free version
geoipFile	EMPTY	(Pre 1.0) Path to the maxmind geoip country file. Download free version
geoLite2ASN	/usr/share/GeoIP/GeoLite2-ASN.mmdb;/data/moloch/etc/GeoLite2-ASN.mmdb (Pre 2.2) /data/moloch/etc/GeoLite2-ASN.mmdb	A Maxmind account is required to use this feature. We recommend installing and setting up the geoipupdate program included with most Linux releases. (Since 2.2) Semicolon ';' separated list of maxmind geoip country files. The first file found will be used. If no files are found a warning will be issued. To disable warning set to a blank string. (Since 1.0) Path to the maxmind geoip ASN file. Download free version
geoLite2Country	/usr/share/GeoIP/GeoLite2-Country.mmdb;/data/moloch/etc/GeoLite2-Country.mmdb (Pre 2.2)/data/moloch/etc/GeoLite2-Country.mmdb	A Maxmind account is required to use this feature. We recommend installing and setting up the geoipupdate program included with most Linux releases. (Since 2.2) Semicolon ';' separated list of maxmind geoip country files. The first file found will be used. If no files are found a warning will be issued. To disable warning set to a blank string. (Since 1.0) Path to the maxmind geoip country file. Download free version
httpRealm	Arkime	HTTP Digest Realm - Must be in the default section of configuration file. Changing the value will cause all previous stored passwords to no longer work.
icmpTimeout	10	For ICMP sessions, Arkime writes a session record after this many seconds of inactivity since the last session save.
interface	EMPTY	Semicolon ';' separated list of interfaces to listen on for live traffic.
keyFile	EMPTY	Private certificate to use for https, if not set then http will be used. certFile must also be set.
magicMode	both (since 1.5.0) or libmagic (before 1.5.0)	libfile can be VERY slow. Less accurate "magicing" is available for http/smtp bodies: libmagic - normal libmagic libmagicnotext - libmagic, but turns off text checks molochmagic - (removed in 1.5.0) subset of libmagic input files, and less accurate both - (since 1.5.0) try basic and then libmagic basic - 50+ of most common headers none - no libmagic or basic calls
maxFileSizeG	4	The max raw pcap file size in gigabytes. The disk should have room for at least 10*maxFileSizeG files.
maxFileTimeM	0	The max time in minutes between rotating pcap files. Useful if there is an external utility that needs to look for closed files every so many minutes. Setting to 0 means only use maxFileSizeG
maxPackets	10000	Arkime writes a session record after this many packets since the last save. Arkime is only tested at 10k, anything above is not recommended.
maxPacketsInQueue	200000	How many packets per packet thread that can be waiting to be processed. Arkime will stop dropping packets if the queue fills up.
maxStreams	1500000	An aproximiate maximum number of active sessions Arkime/libnids will try and monitor
minPacketsSaveBPFs		Semicolon ';' separated list of bpf filters which when matched for a session have their SPI data NOT saved to Elasticsearch. PCAP data is still saved however. It is possible to specify the number of min packets required for SPI to be saved by ending with a :num. A use case is a scanning host inside the network that you only want to capture if their is a conversation "tcp and host 10.10.10.10:1".
ouiFile	EMPTY	The mac address lookup for manufactures file Download free version
packetThreads	1	Number of threads to use to process packets AFTER the reader has received the packets. This also controls how many packet queues there are, since each thread has its own queue. Basically how much CPU to dedicate to parsing the packets. Increase this if you get errors about dropping packets or the packetQ is over flowing. If using the simple writer, this also controls how many pcap files are open for writing.
serverSecret	Value of passwordSecret	The server-to-server shared key. It must be set in the [default] section of the config file, and all viewers in the arkime cluster must have the same value. It can be, and should be changed periodically.
passwordSecret	EMPTY	Password hash secret - Must be in [default] section of the config file, and all viewers in the arkime cluster must have the same value. Since elasticsearch is wide open by default, we encrypt the stored password hashes with this so a malicious person can't insert a working new account. It is also used for secure server-to-server communication if serverSecret is not set a different value. Don't set if you do not want user authentication. Changing the value will make all previously stored passwords no longer work.
parseCookieValue	false	Parse HTTP request cookie values, cookie keys are always parsed.
parseQSValue	false	Parse HTTP query string values, query string keys are always parsed.
parseSMB	true	Parse extra SMB traffic info
parseDNSRecordAll	false	(Since 1.6) Parse a full DNS record (query, answer, authoritative, and additional) and store various DNS information (look up hostname, name server IPs, mail exchange server IPs, and so on) into multiple ES fields
parseSMTP	true	Parse extra SMTP traffic info
parseSMTPHeaderAll	false	(Since 1.6) Parse ALL SMTP request headers not already parsed using the [headers-email] section
parseHTTPHeaderRequestAll	false	(Since 1.6) Parse ALL HTTP request headers not already parsed using the [headers-http-request] section
parseHTTPHeaderResponseAll	false	(Since 1.6) Parse ALL HTTP request headers not already parsed using the [headers-http-response] section
pcapDir	EMPTY	Semicolon separated list of directories to save pcap files to. The directory to save pcap to is picked using round robin by default, see pcapDirAlgorithm for more options.
pcapDirTemplate	EMPTY	When set, this strftime template is appended to pcapDir and allows multiple directories to be created based on time.
pcapDirAlgorithm	round-robin	When pcapDir is a list of directories, this determines how Arkime chooses which directory to use for each new pcap file. Possible values: round-robin (rotate sequentially), max-free-percent (choose the directory on the filesystem with the highest percentage of available space), max-free-bytes (choose the directory on the filesystem with the highest number of available bytes).
parsersDir	/data/moloch/parsers ; ./parsers	Semicolon separated list of directories to load parsers from
pluginsDir	EMPTY	Semicolon separated list of directories to load plugins from
plugins	EMPTY	Semicolon separated list of plugins to load and the order to load them in. Must include the trailing .so
rootPlugins	EMPTY	Semicolon separated list of plugins to load as root and the order to load them in. Must include the trailing .so
readTruncatedPackets	FALSE	Capture will try and process truncated packets the best it can. In general it is best to have full packet captures for Arkime to work well.
rirFile	EMPTY	Path of the RIR assignments file. Download
rotateIndex	daily	Specifies how often to create a new index in Elasticsearch. Use daily or a form of hourly for busy live instances, use weekly or monthly for research instances. When choosing a value, the goal is to have the avg shard be around 30G. Prior to 1.5.0 changing the value will cause previous sessions to be unreachable through the interface, since 1.5.0 you can set queryAllIndices if you need to change the value. Prior to 1.5.0 if using the multi viewer then all Arkime clusters must have the same value. Possible values are: hourly, daily, weekly, monthly. 1.5.0 added hourly2, hourly3, hourly4, hourly6, hourly8, hourly12 which will bucket N number of hours together. So hourly3 for example will make it so each shard has 3 hours of data. hourly1 would be the same as hourly and hourly24 would be the same as daily.
rulesFiles	EMPTY	A semicolon separated list of files that contain rules. These rules match against fields set and can set other fields or meta data about the sessions. See RulesFormat for the format of the files.
sctpTimeout	60	For SCTP sessions, Arkime writes a session record after this many seconds of inactivity since the last session save.
smtpIpHeaders	EMPTY	Semicolon separated list of SMTP Headers that have ips, need to have the terminating colon ':'
spiDataMaxIndices	1	Specify the max number of indices we calculate spidata for. Elasticsearch will blow up if we allow the spiData to search too many indices.
supportSha256	FALSE	Generate Sha256 hashes along side of md5 hashes content.
tcpSaveTimeout	480	For TCP sessions, Arkime writes a session record after this many seconds since the last session save, no matter if active or inactive.
tcpTimeout	480	For TCP sessions, Arkime writes a session record after this many seconds of inactivity since the last session save.
titleTemplate	_cluster_ - _page_ _-view_ _-expression_	_cluster_ = ES cluster name _userId_ = logged in User Id _userName_ = logged in User Name _page_ = internal page name _expression_ = current search expression if set, otherwise blank _-expression_ = " - " + current search expression if set, otherwise blank, prior spaces removed _view_ = current view if set, otherwise blank _-view_ = " - " + current view if set, otherwise blank, prior spaces removed
udpTimeout	60	For UDP sessions, Arkime writes a session record after this many seconds of inactivity since the last session save.
userNameHeader	EMPTY	Header to use for determining the username to check in the database for instead of using http digest. Use this if you have apache or something else doing the auth. The Web Auth Header checkbox must be set for user. The userNameHeader setting should be the lower case version of the header apache is setting. ONLY set this header on the viewer the reverse proxy is talking to, it doesn't need to be set on every viewer. It is recommended you set viewHost to localhost when using this setting, or use iptables, otherwise a hacker can just send this header.
requiredAuthHeader	EMPTY	Used for allowing an external system like LDAP or Active Directory to manage user provisioning and activation/deactivation. It is assumed that the header contains a list of user roles (like active directory groups) which are inspected against the value in `requiredAuthHeaderVal` (see below) to verify that the user is in the appropriate group (ie. "ArkimeUsers"). If so, the user is authorized to use the system, and if an account does not already exist for them in the arkime user store, it is created (see `userAutoCreateTmpl`)
requiredAuthHeaderVal	EMPTY	See `requiredAuthHeader` for more information.
userAutoCreateTmpl	EMPTY	When using `requiredAuthHeader` to externalize provisioning of users to a system like LDAP/AD, this configuration parameter is used to define the JSON structure used to automatically create a arkime user in the arkime users database if one does not exist. The user will only be created if the `requiredAuthHeader` includes the expected value in `requiredAuthHeaderVal`, and is not automatically deleted if the auth headers are not present. Values can be populated into the creation JSON to dynamically populate fields into the user database, which are passed in as HTTP headers along with the user and auth headers. The example value below creates a user with a userId pulled from the `http_auth_http_user` HTTP header with a name pulled from the `http_auth_mail` user header. It is expected that these headers are passed in from an apache (or similar) instance that fronts the arkime viewer as described in the documentation supporting `userNameHeader` `{"userId": "${this.http_auth_http_user}", "userName": "${this.http_auth_mail}", "enabled": true, "webEnabled": true, "headerAuthEnabled": true, "emailSearch": true, "createEnabled": false, "removeEnabled": false, "packetSearch": true }`
viewerPlugins	EMPTY	Semicolon separated list of viewer plugins to load and the order to load them in. Must include the trailing .js
viewHost	EMPTY	The ip used to listen. See the `host` section of https://nodejs.org/docs/latest-v8.x/api/net.html#net_server_listen_port_host_backlog_callback
viewPort	8005	Port for viewer to listen on
viewUrl	http[s]://hostname:[viewport]	The URL used to access this viewer instance
webBasePath	/	The base url for Arkime web requests. Must end with a / or bad things will happen.
arkimeWebURL	http[s]://hostName:[viewPort]/[webBasePath]	(since 3.0) The website URL for Arkime. Used to create links to the Arkime web UI. Must end with a / or bad things will happen. You can include the "http(s)://" or exclude it. If excluded, http/https will be set by determining whether a keyFile and certFile exist.
yara	EMPTY	Where to load Yara rules from.
yaraEveryPacket	TRUE	(Since 1.5.0) Apply yara to every packet or just the packets that are classified.
yaraFastMode	TRUE	(Since 1.5.0) Set the Yara Fast Mode flag.
elasticsearchAPIKey	EMPTY	(Since 3.0) Use an Elastisearch API key for access without requiring basic authentication. Once you have created an API Key, you must base64 encode the id and api_key joined by a colon. `echo -n "id:api_key" \| base64` is one way to generate the base64 key. Notice the -n, you have to make sure you don't encode an extra newline.
usersElasticsearchAPIKey	EMPTY	(Since 3.0) Use an Elastisearch API key for users DB access without requiring basic authentication. See elasticsearchAPIKey setting for information on creating and encoding an Elasticsearch API Key. See usersElasticsearch setting for information about the Users DB.

Advanced Settings

Setting	Default	Description
accessLogFile	EMPTY	If left empty, then the viewer will log to stdout. It can be set to a filename and then logging will be directed there.
accessLogFormat	:date :username %1b[1m:method%1b[0m %1b[33m:url%1b[0m :status :res[content-length] bytes :response-time ms	(Since 2.3) Set the log record format -- this uses Express Morgan. The string is URL Encoded and so uses %xx to escape special characters. See the Morgan documentation for the list of keywords.
accessLogSuppressPaths	EMPTY	(Since 2.3) This is a semi-colon seperated list of URL paths which should not be logged. Setting this to /eshealth.json will suppress logging of all calls to that endpoint.
aes256Encryption	true (since 2.4.0) false (before 2.4.0)	(Since 2.2.0) Use better encryption when talking between viewer nodes
autoGenerateId	false	(Since 1.6) Use Elasticsearch auto generated ids
caTrustFile	Empty	Optional file with PEM encoded certificates to use when validating certs.
compressES	false	Compress requests to ES, reduces ES bandwidth by ~80% at the cost of increased CPU. MUST have "http.compression: true" in elasticsearch.yml file
dbBulkSize	20000	Size of indexing request to send to Elasticsearch. Increase if monitoring a high bandwidth network.
dbEsHealthCheck	TRUE	Perform an Elasticsearch health check every 30 seconds and report on slowness or if not green. For big clusters this should be disabled.
dbFlushTimeout	5	Number of seconds before we force a flush to Elasticsearch
enablePacketLen	FALSE (2.0 and later), TRUE (before 2.0)	(Since 2.0) Index all the packet lengths in elasticsearch
filenameOps	EMPTY	(Since 1.5.0) A semicolon separated list of operations that use the filename. Format is `fieldexpr=match%value` so if you wanted to set a tag based on part of filenames that start with gre use `tags=/gre-(.*)\\.pcap%gretest-\\1`. Notice two backslashes are required everywhere you want one because of ini formatting.
fragsTimeout	480	Number of seconds to keep around an ip fragment and try an reassemble it
hstsHeader	FALSE	From viewer, return a hsts header on responses
huntWarn	100000	(Since 1.6.0) Warn users when creating a hunt if more then this many sessions will be searched
huntLimit	1000000	(Since 1.6.0) Do not create hunts for non admin users if more then this many sessions will be searched
huntAdminLimit	10000000	(Since 1.6.0) Do not create hunts for admin users if more then this many sessions will be searched
iframe	deny	Used to set the `X-Frame-Options` header for putting Arkime in an iFrame. Options include "deny", "notallowed", or a URL to allow from.
includes		Semicolon ';' separated list of files to load for config values. Files are loaded in order and can replace values set in this file or previous files. Setting includes is only supported in the top level config file.
interfaceOps	EMPTY	(Since 1.5.0) A semicolon separated list of a comma separted list of operations to set for each interface. The semicolon list must have the same number of elements as the interface setting. The format is `fieldexp=value`. So for example if you have `interface=eth1;eth2` you could set a tag with `interfaceOps=tags=eth1;tags=eth2`.
isLocalViewRegExp	EMPTY	(Since 1.6.1) If the node matches the supplied regexp, then the node is considered local.
ja3Strings	FALSE	(Since 2.0) Index the raw JA3 strings before hashing them
maxESConns	20	Max number of connections to Elasticsearch from capture process
maxESRequests	500	Max number of Elasticsearch requests outstanding in queue
maxFreeOutputBuffers	50	For capture, the max number of free output buffers we keep around before returning them to the OS
maxFrags	10000	Max number of ip fragment packets to save and try and match at once
maxMemPercentage	100	If arkime exceeds this amount of memory it will log an error and send itself a SIGSEGV that should produce a core file.
maxReqBody	256	For HTTP requests, store this many bytes in the http.requestBody field. Can be disabled by setting to 0.
reqBodyOnlyUtf8	TRUE	Only store request bodies that are utf8
maxTcpOutOfOrderPackets	256	(Since 1.5.0) Max number of tcp packets to track while trying to reassemble the TCP stream
offlineDispatchAfter	pre 2.1 - 5000 (unchangeable) 2.1,2.2 - 1000 2.3 - 2500	How many packets to read from offline pcap files at once.
offlineFilenameRegex	(?i)\.(pcap\|cap)$	Regexp to control which filenames are processed when using the -R option to moloch-capture.
pcapBufferSize	100000	pcap library buffer size, see man pcap_set_buffer_size
pcapReadMethod	libpcap	Specify how packets are read from network cards: libpcap = Use libpcap pfring = Use pfring directly, requires rootPlugins=reader-pfring.so daq = Use daq, requires rootPlugins=reader-daq.so snf = Use Myricom snf, requires rootPlugins=reader-snf.so tpacketv3 = Use linux tpacketv3 (afpacket) interface
pcapWriteMethod	normal	Specify how packets are written to disk: simple = what you should probably use simple-nodirect = use this with zfs/nfs s3 = write packets into s3 null = don't write to disk at all
pcapWriteSize	262144	Buffer size when writing pcap files. Should be a multiple of the raid 5/xfs stripe size and multiple of 4096 if using direct/thread-direct pcapWriteMethod
prefix	EMPTY	It is possible for multiple Arkime clusters to live inside the same Elasticsearch cluster by giving each Arkime cluster a different prefix value.
queryAllIndices	FALSE for viewer, TRUE for multiviewer	(Since 1.5.0) Always query all indices instead of trying to calculate which ones. Use this if searching across that use different rotateIndex values.
saveUnknownPackets	EMPTY	(Since 1.5.2) Save unknown ether, ip, or corrupt packets into separate files. This variable takes a semicolon separated list of the following values (applied in order): all = save all unknown ip and ether packets ip:all = save all unknown ip packets ether:all = save all unknown ether packets ip:N = save all unknown ip packets with type of N -ip:N = don't save all unknown ip packets with type of N ether:N = save all unknown ether packets with type of N -ether:N = don't save all unknown ether packets with type of N corrupt = save all corrupt packets (Since 1.5.3)
simpleEncoding	EMPTY	Arkime support PCAP at Rest Encryption.
simpleMaxQ	2000	(since 2.0.1) The maximum number of disk queue entries per capture node. Once this limit is hit, packets will not be saved to disk until the disk queue falls below the threshold. The packets will still be processed, so all meta data is still extracted. Sessions that have packets not written to disk will be tagged with pcap-disk-overload. Increasing this value will use more memory when there is disk congestion.
snapLen	16384	The maximum size of a packet Arkime will read off the interface. This can be changed to fix the "Arkime requires full packet captures" error. It is recommend that instead of changing this value that all the card "offline" features are turned off so that arkime gets a picture of whats on the network instead of what the capture card has reassembled. For VMs, those features must be turned off on the physical interface and not the virtual interface. This setting can be used when changing the settings isn't possible or desired.
trackESP	FALSE	(Since 1.5.0) Add ESP sessions to Arkime, no decoding
uploadCommand	EMPTY	If set uploads from the UI are allowed. An upload saves the file and runs capture on it, so its better to just run capture instead of using upload if you can. Example setting would be `/data/moloch/bin/moloch-capture --copy -n {NODE} -r {TMPFILE} -c {CONFIG} {TAGS}`. The following templated values will be filled in for you: NODE - The node name of the viewer that received the upload TMPFILE - The tmp file name uploaded to CONFIG - The config file path used to start viewer TAGS - The tags set with the upload
uploadFileSizeLimit	EMPTY	(Since 2.0) If set, the max size of pcap files that can be uploaded from the UI
usersElasticsearch	EMPTY	Set this option to which single elasticsearch cluster to use for the users index. It is possible for multiple Arkime clusters to share the same users table, so that the same accounts and settings work across all clusters. So the elasticsearch setting would point to a unique elasticsearch cluster per Arkime cluster, while the usersElasticsearch setting would point to a single shared Elasticsearch cluster. Same format as the elasticsearch option.
usersPrefix	[PREFIX] if set otherwise EMPTY	Like prefix but only for the users table.
valueAutoComplete	![multiES] otherwise true	Autocomplete field values in the search expression.

Packet Deduplication Settings

Arkime since version 2.7.1 has supported basic packet deduplication. The deduplication is done early in the pipeline before a packet queue is assigned. Currently only the iphdr + tcphdr or iphdr + udphdr is used to find duplicate packets, ignoring the TTL field. The deduplication will only look back a configured number of seconds.

The packet deduplication is an expensive CPU task, so you should monitor the "Dup Drops/s" column in capture stats to see if worthwhile.

Setting	Default	Description
enablePacketDedup	false	Enable packet deduplication
dedupSeconds	2	The number of seconds to keep packet information for.
dedupPackets	1048575	The approximate number of packets to keep information on per second. Set this number to the max number of packets per second you expect.

Reader tpacketv3 Settings

Tpacketv3, also known as AFPacket v3, is the preferred reader for arkime and can be used on most 3.x or later kernels. Configure moloch-capture to use tpacketv3 as the reader method with pcapReadMethod=tpacketv3 in your configuration file. This is also know as afpacket.

Setting	Default	Description
tpacketv3BlockSize	2097152	The block size in bytes used for reads from each interface. There are 120 blocks per interface.
tpacketv3NumThreads	2	The number of threads used to read packets from each interface. These threads take the packets from the AF packet interface and place them into the packet queues.
tpacketv3ClusterId	0	(Since 2.0) The cluster id for use with PACKET_FANOUT

Example:

[default]
pcapReadMethod=tpacketv3
tpacketv3BlockSize=2097152
interface=eth0
tpacketv3NumThreads=2

Reader daq Settings

To use daq:

load the daq plugin by changing configuration file so it has reader-daq.so as a rootPlugins rootPlugins=reader-daq.so
tell moloch-capture to use daq as the reader method with pcapReadMethod=daq in your configuration file
optionally change interface to any special daq interface value

Setting	Default	Description
daqModuleDirs	/usr/local/lib/daq	Directories where the daq modules live
daqModule	pcap	The daq module to use

Reader pfring Settings

We suggest you try tpacketv3 (afpacket) first if available on the host
install the pfring package on all hosts that will run moloch-capture from http://packages.ntop.org/
load the pfring plugin by changing configuration file so it has reader-pfring.so as a rootPlugins rootPlugins=reader-pfring.so
Configure moloch-capture to use pfring as the reader method with pcapReadMethod=pfring in your configuration file
optionally change interface to any special pfring interface value

Setting	Default	Description
pfringClusterId	0	The pfring cluster id

Reader snf Settings

install the snf package on all hosts that will run moloch-capture
load the snf plugin by changing configuration file so it has reader-snf.so as a rootPlugins rootPlugins=reader-snf.so
Configure moloch-capture to use snf as the reader method with pcapReadMethod=snf in your configuration file
optionally change interface to any special snf interface value

Setting	Default	Description
snfNumRings	1	Number of rings per interface
snfDataRingSize	0	The data ring size to use, 0 means use the SNF default
snfFlags	-1	Controls process-sharing (1), port aggregation (2), and packet duplication (3). (Default value uses SNF_FLAGS environment variable)
snfNumProcs	1	(Since 2.0) The number of capture processes listening on the shared interface
snfProcNum	0	(Since 2.0) Which capture process this is if using a shared interface

Reader PCAP-Over-Ip Settings

Since Arkime 2.7.0, we can process PCAP-Over-Ip requests. Two modes are support client and server. Client mode will connect to a pcap-over-ip service, server will listen for pcap-over-ip connections. Read more information about PCAP-Over-Ip.

Client mode is set by using pcapReadMethod=pcap-over-ip-client. The interface setting must have a list of hosts and optional ports to connect to.

Server mode is set by using pcapReadMethod=pcap-over-ip-server. The interface setting is ignored and should just be set to dummy.

Writer S3 Settings

See S3 document for more information about using S3 for PCAP storage.

Setting	Default	Description
s3Region	us-east-1	S3 Region to send data to
s3Bucket	none	S3 Bucket to store pcap info
s3AccessKeyId	(Since 2.1) Obtained from the EC2 IAM Role.
s3SecretAccessKey	(Since 2.1) Obtained from the EC2 IAM Role.
s3Compress	FALSE	Should traffic be compressed before sending
s3WriteGzip	FALSE	(Since 2.1) Should the PCAP files be stored on S3 in gzipped form
s3StorageClass	STANDARD	(Since 2.1) The S3 storage class to use&emdash;this has pricing implications
s3PathAccessStyle	TRUE if there is a period in the bucket name.	(Since 2.1) If true use s3.amazonaws.com/s3Bucket if false use s3Bucket.s3.amazonaws.com
s3MaxConns	20	Max connections to S3
s3MaxRequests	500	Max number of outstanding S3 requests
s3ExpireDays	none	Expiration days for S3 stored PCAP files. Expired PCAP files will be deleted.
s3Host	none	Override the default endpoint URL for the specified Bucket and Region. This is only used if you want to use a third party s3 server, like a MinIO or pithos instance.
s3UseHttp	FALSE	Use HTTP instead of HTTPS to connect to S3 host.

Debug Settings

Setting	Default	Description
logESRequests	false	Write to stdout Elasticsearch requests
logEveryXPackets	50000	Write to stdout info every X packets. Set to -1 to never log status.
logFileCreation	false	Write to stdout file creation information
logHTTPConnections	true for online captures	Log http connection attempts and information
logUnknownProtocols	false	Write to stdout unknown IP protocols

custom-fields

You can add custom fields to Arkime several ways, including the wise and tagger plugins. Since Arkime 1.5.2 the easiest way is to use a [custom-fields] section in the ini file. At capture startup it will check to make sure all those fields exist in the database. The format of the line is the similar as used in wise and tagger, except you use expression=definition. You will need to also create a [custom-views] section to display the data on the Sessions tab.

Field definition:

Setting	Default	Description
field	The field expression, overrides the key in the ini file
kind	REQUIRED	integer ip lotermfield - lowercased string termfield - string uptermfield - uppercased string
count	false	Track number of items with a .cnt field auto created
friendly	fieldname	A SHORT description, used in SPI View
db	REQUIRED	The DB field name
group	Before first dot in field or general	Category for SPI view
nolinked	false	(Since 2.1.0) Set to true for linked sessions to have independent values for this field
noutf8	false	(Since 2.1.1) Set to true for the field to be 8 bit instead of utf8
help	fieldname	Help to display in about box or help page

[custom-fields]
# Format is FieldExpr=text format
theexpression=db:theexpression

sample.md5=db:sample.md5;kind:lotermfield;friendly:Sample MD5;count:true;help:MD5 of the sample

custom-views

With Arkime "views" are how the SPI data is displayed in the Sessions tab. Usually there is a unique "view" for each category of data. You can add custom views to Arkime several ways, including the wise and tagger plugins. Since Arkime 1.5.2 the easiest way is to use a [custom-views] section in the ini file. At viewer startup, a new section will be created for each entry. The format of the line is name=definition. Viewer will sort all views by name when choosing the order to display in.

Field definition:

Setting	Default	Description
fields	REQUIRED	A comma separated list of field expression to display. They will be displayed in order listed.
title	Defaults to name	The title to get the section on Sessions tab
require	REQUIRED	The db session name to require be set to show the section.

[custom-views]
sample=title:Samples;require:sample;fields:sample.md5,sample.house

[custom-fields]
sample.md5=db:sample.md5;kind:lotermfield;friendly:Sample MD5;count:true;help:MD5 of the sample
sample.house=db:sample.house;kind:termfield;friendly:Sample House;count:true;help:House the sample lives in

moloch-clusters

The moloch-clusters section is used to describe the various moloch-clusters that are available to forward traffic to either manually or through the CRON functionality. Each line represents a single cluster, with the name just being any unique string.

[moloch-clusters]
cluster1=url:https://arkime.example.com:8005;passwordSecret:password;name:Cluster
cluster2=url:https://cluster2.example.com:8005;passwordSecret:foo;name:Test Cluster

url	The base url to use to contact cluster
passwordSecret	The passwordSecret for the remote cluster, if it is different from current cluster
name	Friendly name to display in UI

Multi Viewer Settings

The multi viewer is useful when you have multiple Arkime clusters that you want to search across. To use the multi viewer, an extra viewer process and a multies process must be started. The viewer process works like a normal viewer process, except instead of talking to a elasticsearch server, it talks to a multies server that proxies the queries to all the real elasticsearch servers. These two processes can share the same config file and node name section. The viewer part uses the SAME configuration values as above if you need to set anything.

A sample configuration for multi viewer (the elasticsearch variable points to the multies.js process):

[multi-viewer]
elasticsearch=127.0.0.1:8200
viewPort = 8009
viewHost = localhost
multiES = true
multiESPort = 8200
multiESHost = localhost
multiESNodes = escluster1.example.com:9200,name:escluster1,prefix:PREFIX,elasticsearchAPIKey:testkey1;escluster2.example.com:9200,name:escluster2

Which you would then use by starting both the multiviewer and multies. This is a sample for running manually (but you should setup startup scripts to run for real):

cd /data/moloch/viewer
/data/moloch/bin/node multies.js -n multi-viewer
/data/moloch/bin/node viewer.js -n multi-viewer

Setting	Default	Description
multiES	false	This is the multiES node
multiESPort	8200	Port that multies.js should listen on
multiESHost	EMPTY	Host interface that multies.js should listen on
multiESNodes	EMPTY	Semicolon separated list of Elasticsearch nodes that MultiES should connect to. The first node listed will be considered the primary node and is used for users/views/queries. Example: `escluster1.example.com:9200,name:escluster1,prefix:PREFIX,elasticsearchAPIKey:testkey1;escluster2.example.com:9200,name:escluster2` Components (comma separated): (required) the first part is the node `host:port` (required since 2.7.0) `name:` element to name the Elasticsearch node. (optional) `prefix:` element can follow each host if that cluster was setup with an Elasticsearch prefix. (optional since 3.0) `elasticsearchAPIKey:` element to set an Elasticsearch API Key per node. See elasticsearchAPIKey settings for more information on how to configure ES API Keys.

override-ips

override-ips is a special section that overrides the MaxMind databases for the fields set, but fields not set will still use MaxMind (example if you set tags but not country it will use MaxMind for the country) Spaces and capitalization are very important.

[override-ips]
10.1.0.0/16=tag:ny-office;country:USA;asn:AS0000 This is neat

tag	A single tag to set for matches
country	A 3 character country code to set for matches
asn	An ASN value to set for matches
rir	A RIR value to set for matches

headers-http-request

This section makes it easy to specify HTTP Request headers to index. They will be searchable in the UI using http.[HEADERNAME]

[headers-http-request]
referer=type:string;count:true;unique:true

type	string - index as a string integer - index as an integer ip - index as an IP
unique	true by default, only index unique values
count	false by default, create a second field http.[HEADERNAME].cnt with the number items found

headers-http-response

This section makes it easy to specify HTTP Request headers to index. They will be searchable in the UI using http.[HEADERNAME]

[headers-http-response]
location=type:string
server=type:string

headers-email

This section makes it easy to specify email headers to index. They will be searchable in the UI using email.[HEADERNAME]

[headers-email]
x-priority=type:integer

right-click

It is possible to configure right click actions on any SPI data fields. Right click actions can be added based on either the field name or the category of the data. They can be added either in the configuration file or by enabled WISE data sources.

Configuration File Sample:

[right-click]
VTIP=url:https://www.virustotal.com/en/ip-address/%TEXT%/information/;name:Virus Total IP;category:ip
VTHOST=url:https://www.virustotal.com/en/domain/%HOST%/information/;name:Virus Total Host;category:host
VTURL=url:https://www.virustotal.com/latest-scan/%URL%;name:Virus Total URL;category:url

Each line in the right-click section contains possible actions:

Piece	Notes
[key]=	The key must be unique and is also used as the right click menu name if the name field is missing
actionType	Needs a url. Supported actionTypes: fetch: Information will be fetched and displayed in the rightclick menu for 5 seconds
func	A javascript function body (feeds into new Function()) that returns a right click action formatted as an object. This is the recommended way if you need to omit the url from an action and can be used to place plain text in the menu. Value is not calculated so one of the keys in the returned action object must be "value" if you want to show name:value pair in the menu (but not req). Formatting substitutions will also be ignored.
url	The url to open if selected. There are some basic URL substitutions defined below.
name	The menu text to display
category	If the field that is right clicked on has this category then display this menu item. All right click entries must have either a category or fields variable defined.
fields	A comma separated list of field names. If the field that is right clicked on has one the field expressions in the list then display this menu item. All right click entries must have either a category or fields variable defined.
regex	A regex to use on the right clicked text to extract a piece for the URL. The first matching group is substituted for %REGEX% in the url. If the regex doesn't match at all then the menu isn't displayed.
users	A comma separated list of user names that can see the right click item. If not set then all users can see the right click item.

The possible URL Substitutions are:

URL Substitutions	Notes
%TEXT%	The text clicked on in raw form
%UCTEXT%	The text clicked on, upper-cased
%URL%	A URL extracted from the text clicked on
%HOST%	A hostname extracted from the text clicked on. Sometimes the same as %TEXT%, sometimes a subset.
%REGEX%	The first regex group match
%EXPRESSION%	The search expression, URI encoded
%DATE%	The search date/time range, eg., `startTime=1567447943&stopTime=1569607943` (UNIX seconds) for a custom date range or `date=336` (hours) for times relative to now
%ISOSTART%	The beginning of the search date/time range, ISO_8601 formatted
%ISOSTOP%	The end of the search date/time range, ISO_8601 formatted
%FIELD%	The field name as it would be referenced in a Arkime search expression
%DBFIELD%	The field name as referenced in the underlying Elasticsearch database
%NODE%	The name of the capture node
%ID%	The session ID

The categories that Arkime uses are:

Category	Notes
asn	An ASN field
country	A three letter country code
host	A domain or host name
ip	An ip address
md5	A md5 of a payload, such as for http body or smtp attachment
port	The TCP/UDP port
rir	The Regional Internet Registry
url	A URL
user	A user name or email address

packet-drop-ips

This section allows you to specify ips or cidrs to drop from being processed. This is different from a bpf filter since the packets will actually reach moloch-capture (and counted) but won't be fully processed. However if you have many ranges/ips to drop it can be more efficient then bpfs. It is possible to also allow ranges inside of dropped ranges using the "allow" keyword. Order added doesn't matter, searching always finds the best match.

[packet-drop-ips]
10.0.0.0/8=drop
10.10.0.0/16=allow
10.10.10.0/24=drop
10.10.10.10=allow

PCAP at Rest Encryption

Arkime provides support for PCAP at rest encodings. Two forms of encodings are currently supported: "aes-256-ctr" and "xor-2048". Please note that xor-2048 is not actually secure and is only for testing.

The current implementation is based around openssl. The encoding that you wish to use is configured in the config file by setting the simpleEncoding variable. The simpleEncoding may be set either globally or per node.

Each file on disk is encoded by a unique data encryption key (dek). The dek is encrypted using a key encryption key (kek) when stored. The encrypted dek, the id of the kek, and the initialization vector (IV) are all stored per file in Elasticsearch. Which kek is used when creating files, is selected with the simpleKEKId variable. The simpleKEKId may be set either globally or per node.

The kek passwords that may be used should be placed in a [keks] section of the config file. There is one line for each kekid to kek mapping. An easy way to create kek passwords is openssl rand -base64 30. Remember a kek is password used to encrypt the dek and NOT the password used to encrypt the files. The dek is what is used to encrypt the files, and is unique per file.

You MUST secure your config file.

You are not required to use the same keks on all nodes, however, you can if you wish. It is recommended that you rotate your keks occasionally (timing dependent on your risk tolerance) and create new keks to be used. Do NOT delete the old keks until all pcaps which have been encoded with those keks have been expired.

Currently it is not possible to re-encrypt a data encryption key, however, this should be possible in the future with a db.pl command.

Example:

[default]
pcapWriteMethod=simple
simpleEncoding=aes-256-ctr
simpleKEKId=kekid1

[keks]
kekid1=Randomkekpassword1
kekid2=Randomkekpassword2

Advantages:

Don't need to mess with disk encryption
Access to just the host doesn't give you access to the pcap file

Disadvantages:

Normal pcap tools will no longer work
If the files index in elasticsearch gets corrupt, you can not reingest the files

Plugins

WISE

See the WISE page

Netflow

Arkime can generate netflow for all sessions it saves SPI data for. Add netflow.so to the plugins= line in the config file.

Setting	Default	Description
netflowSNMPInput	0	What to fill in the input field
netflowSNMPOutput	0	What to fill in the output field
netflowVersion	5	Version of netflow to send: 1, 5, 7
netflowDestinations	EMPTY	Semicolon ';' separated list of host:port destinations to send the netflow

Suricata

Arkime can enrich sessions with Suricata alerts. Suricata and Arkime must see the same traffic and share the same eve.json/alert.json for the plugin to work. Sessions that have been enriched will have several new fields, all starting with suricata, and will be displayed in a Suricata section of the standard Arkime session UI. Arkime matches sessions based on the 5 tuple from the alert.json or eve.json file, only using the items with event_type of alert. A very simple query to find all sessions that have Suricata data is suricata.signature == EXISTS!.

Note: there isn't a special Suricata UI inside Arkime, this is just adding new fields to Arkime sessions like wise or tagger do. The Suricata enrichment is done by moloch-capture, so moloch-capture must see the traffic.

Add suricata.so to the plugins= line in the config file.

Setting	Default	Description
suricataAlertFile	REQUIRED	The full path to either the alert.json or eve.json file, make sure the `dropUser` or `dropGroup` can open the file
suricataExpireMinutes	60	(Since 1.5.1) The number of minutes to keep Suricata alerts in memory before expiring them based on the Suricata alert timestamp. For example if a Suricata alert has a timestamp of 1am, the default is to keep looking for matching traffic until 2am (60 minutes).

Sample Config:

# Add suricata.so to your plugins line, or add a new plugins line
plugins=suricata.so

# suricataAlertFile should be the full path to your alert.json or eve.json file
suricataAlertFile=/nids/suricata/eve.json

Tagger

See the Tagger page and the Tagger Format page

Lua

See the Lua page

Setting	Default	Description
luaFiles	EMPTY	The Lua Files to load

TCP Health Check

TCP Health Check plugin enables the capture module to listen on a specific TCP port and immediately close accepted connections. This is useful to keep load balancer health checks informed about the capture module working properly. One specific use case is Health Checks for Target Groups in AWS Network Load Balancer.

Add tcphealthcheck.so to the plugins= line in the config file.

Setting	Default	Description
tcpHealthCheckPort	EMPTY	Make the capture module listen on this TCP port for health checks.

High Performance Settings

Sample config items for max performance. Most of the defaults are fine. Reading Arkime FAQ - Why am I dropping packets and https://github.com/pevma/SEPTun or https://github.com/pevma/SEPTun-Mark-II may be helpful.

# MOST IMPORTANT, use basic magicMode, libfile kills performance
magicMode=basic

pcapReadMethod=tpacketv3
tpacketv3BlockSize=8388608

# Increase by 1 if still getting Input Drops
tpacketv3NumThreads=2

pcapWriteMethod=simple
pcapWriteSize=4194304

# Start with 5 packet threads, increase by 1 if getting thread drops.  Should be about 1.5 x # Gbps that need to be captured
packetThreads=5

# Increase the size of ES messages and compress them for lower traffic rates
dbBulkSize=4000000
compressES=true
dbEsHealthCheck=false

# Set to number of packets a second, if still overflowing try 400k
maxPacketsInQueue = 300000

# Uncomment to disable features you don't need
# parseQSValue=false
# parseCookieValue=false

The following rules can help greatly reduce the number of SPI sessions being written to disk:

---
version: 1
rules:
- name: "Dont write tls packets or check yara after 10 packets, still save SPI"
  when: "fieldSet"
  fields:
    protocols:
    - tls
  ops:
    _dontCheckYara: 1
    _maxPacketsToSave: 10

- name: "Dont save SPI sessions to ES with only 1 src packet"
  when: "beforeFinalSave"
  fields:
    packets.src: 1
    packets.dst: 0
    tcpflags.syn: 1
  ops:
    _dontSaveSPI: 1

- name: "Dont save SPI data for listed hostnames tracked by dst ip:port, use on cloud destinations"
  when: "fieldSet"
  fields:
    host.http:
    - ad.doubleclick.net
    protocols:
    - tls
  ops:
    _dontSaveSPI: 1
    _maxPacketsToSave: 1
    _dropByDst: 10