General

Why should I use Arkime?

If you want a standalone open source full packet capture (FPC) system with meta data parsing and searching, then Arkime may be your answer! Arkime allows you complete control of deployment and architecture. There are other FPC systems available.

Why change our name?

This project has experienced significant growth, adoption, and change over the last eight years. We are now at a new milestone and believe it’s the right time to rename our project to Arkime! Read more about why we made this change here.

How do you pronounce our name?

(/ɑːrkɪˈmi/)? Read more about why we changed our name here.

Upgrading Arkime

Upgrading Arkime requires you install versions in order, as described in the chart below. If the version you are currently on isn’t listed please upgrade to the next higher version in the chart, you can then install the major releases in order to catch up. New installs can start from the latest version.

Name Version Min upgrade from ES Versions Special Instructions Notes
Arkime 2.7+ 2.0.0 7.4+ (7.9.0+ recommended, 7.7.0 broken) ES 7 instructions
Moloch 2.2+ 1.7.0 (1.8.0 recomended) 6.8.2+ (6.8.6+ recommended), 7.1+ (7.8.0+ recommended, 7.7.0 broken) Moloch 2.0 instructions Must already be on 6.8.x or 7.1+ before upgrading to 2.2
Moloch 2.0, 2.1 1.7.0 (1.8.0 recomended) 6.7, 6.8, 7.1+ Moloch 2.0 instructions Must already be on ES 6.7 or 6.8 (ES 6.8.6 recommended) before upgrading to 2.0
Moloch 1.8 1.0.0 (1.1.x recommended) 5.x or 6.x ES 6 instructions Must have finished the 1.x reindexing, stop captures for best results
Moloch 1.1.1 0.20.2 (0.50.1 recommended) 5.x or 6.x (new only) Instructions Must be on ES 5 already
Moloch 0.20.2 0.18.1 (0.20.2 recomended) 2.4, 5.x ES 5 instructions

What OSes are supported?

We have RPMs/DEBs available on the downloads page Our deployment is on RHEL 7, using both the pcap and afpacket reader depending on deployment. We recommend using afpacket (tpacketv3) whenever possible. A large amount of development is done on macOS 10.15 using MacPorts or Homebrew, however, it has never been tested in a production setting. :) Arkime is not supported on 32 bit machines.

The following OSes should work out of the box for compiling yourself:

Arkime is not working

Here is the common check list:

  1. Check that Elasticsearch is running and green using
    curl http://localhost:9200/_cat/health
    on the machine running Elasticsearch.
  2. Check that the db has been initialized with /data/moloch/db/db.pl http://elasticsearch.hostname:9200 info
  3. Check that viewer is reachable by visiting http://arkime-viewer.hostname:8005 from your browser
    1. If it doesn’t render, looks strange or warns of an old browser, use a newer supported browser
  4. Check for errors in /data/moloch/logs/viewer.log and that viewer is running with pgrep -lf viewer
  5. Check for errors in /data/moloch/logs/capture.log and that capture is running with pgrep -lf capture
  6. Check that the stats page shows the capture nodes you are expecting, visit http://arkime-viewer.hostname:8005/stats?statsTab=1 in your browser.
    1. Make sure the nodes are showing packets being received
    2. Make sure the timestamp for nodes is recent (within 5 seconds)
  7. Disable any bpf= in /data/moloch/etc/config.ini, if that fixes the issue read BPF FAQ answer
  8. If the browser has "Oh no, Arkime is empty! There is no data to search." but the stats tab shows packets are being captured:
    1. Live capture Arkime only writes records when a session has ended, it may take several minutes for session to show up after a fresh start, see /data/moloch/etc/config.ini to shorten the timeouts
    2. Elasticsearch will only refresh the indices once a minute with the default Arkime config, force a refresh with curl http://elasticsearch.hostname:9200/_refresh
    3. Verify your time frame for search covers the data (try switching to ALL)
    4. Check that you don’t have a view set
    5. Check that your user doesn’t have a forced expression set, might need to ask your Arkime admin
  9. Restarting moloch-capture after adding a --debug option may print out useful information what is wrong if you are having packet capture issues. You can add multiple --debug options to get even more information. Capture will print out the config settings it is using, verify they are what you expect. Usually this setting is changed in /etc/systemd/system/molochcapture.service.
  10. Restart viewer after adding a --debug option may print out useful information what is wrong if you are having issues viewing packets that were captured. Usually this setting is changed in /etc/systemd/system/molochviewer.service.
    1. Make sure the plugins and parsers directories are correctly set in /data/moloch/etc/config.ini and readable by the viewer process

How do I reset Arkime?

  1. Leave Elasticsearch running
  2. Shutdown all running viewer or capture processes so no new data is recorded.
  3. To delete all the SPI data stored in Elasticsearch, use the db.pl script with either the init or wipe commands. The only difference between the two commands is that wipe leaves the added users so they don’t need to be re-added.
    /data/moloch/db/db.pl ESHOST:ESPORT wipe
  4. Delete the PCAP files. The PCAP files are stored on the file system in raw format. You need to do this on all of the capture machines.
    /bin/rm -f /data/moloch/raw/*

Self-Signed SSL/TLS Certificates

It is possible to get self signed certificates to work in the following scenarios:

Usually the easiest way is to add the self signed cert to the OS's list of valid certificates or chains. Googling is the best way to figure out how to do this. Viewer also supports a caTrustFile option.

The core Arkime team does not support or recommend self signed certs. Use the money you are saving on a commercial product and go buy real certs. Wildcard certs are now cheap and you can even go with free Lets Encrypt certs. There may be folks on the Arkime slack workspace willing to help out.

Both capture and viewer can run with --insecure to turn off cert checking. You will need to add this option to the startup command for both capture and viewer. For example change in the /etc/systemd/system/arkimecapture.conf file, the ExecStart line from ... capture -c ... to ... capture --insecure -c ....

How do I upgrade to Moloch 1.0

Moloch 1.0 has some large changes and updates that will require all session data to be reindexed. The reindexing is done in the background AFTER upgrading so there is little downtime. Large changes in 1.0 include:

If you have any special parsers, tagger, plugins or wise source you may have to change configurations.

To upgrade:

Once 1.1.1 is working, its time to reindex the old session data:

How do I upgrade to Moloch 2.0

Upgrading to Moloch 2.0 is a multistep process that requires an outage. An outage is required because all the captures must be stopped before upgrading the database so there are no schema issues or corruption. Most of the administrative indices will have new version numbers after this upgrade, so that elasticsearch knows they were created with 6.7 or 6.8. This is very important when upgrading to ES 7.x later.

How do I upgrade to Arkime 3.0

3.0 has not been released yet, this are preliminary instructions! Upgrading to Arkime 3.0 is a multistep process that requires an outage. An outage is required because all the captures must be stopped before upgrading the database so there are no schema issues or corruption. Most of the administrative indices will have new version numbers after this upgrade, so that elasticsearch knows they were created with 7. This is very important when upgrading to ES 8.x later.

Arkime Logo