base/frameworks/logging/writers/elasticsearch.bro

LogElasticSearch

Log writer for sending logs to an ElasticSearch server.

Note: This module is in testing and is not yet considered stable!

There is one known memory issue. If your elasticsearch server is running slowly and taking too long to return from bulk insert requests, the message queue to the writer thread will continue growing larger and larger giving the appearance of a memory leak.

Namespace:LogElasticSearch
Source File:/scripts/base/frameworks/logging/writers/elasticsearch.bro

Summary

Options

LogElasticSearch::cluster_name: string &redef Name of the ES cluster.
LogElasticSearch::index_prefix: string &redef Name of the ES index.
LogElasticSearch::max_batch_interval: interval &redef The maximum amount of wall-clock time that is allowed to pass without finishing a bulk log send.
LogElasticSearch::max_batch_size: count &redef The batch size is the number of messages that will be queued up before they are sent to be bulk indexed.
LogElasticSearch::max_byte_size: count &redef The maximum byte size for a buffered JSON string to send to the bulk insert API.
LogElasticSearch::server_host: string &redef ES server.
LogElasticSearch::server_port: count &redef ES port.
LogElasticSearch::type_prefix: string &redef The ES type prefix comes before the name of the related log.

Constants

LogElasticSearch::transfer_timeout: interval The time before an ElasticSearch transfer will timeout.

Detailed Interface

Options

LogElasticSearch::cluster_name
Type:string
Attributes:&redef
Default:"elasticsearch"

Name of the ES cluster.

LogElasticSearch::index_prefix
Type:string
Attributes:&redef
Default:"bro"

Name of the ES index.

LogElasticSearch::max_batch_interval
Type:interval
Attributes:&redef
Default:1.0 min

The maximum amount of wall-clock time that is allowed to pass without finishing a bulk log send. This represents the maximum delay you would like to have with your logs before they are sent to ElasticSearch.

LogElasticSearch::max_batch_size
Type:count
Attributes:&redef
Default:1000

The batch size is the number of messages that will be queued up before they are sent to be bulk indexed.

LogElasticSearch::max_byte_size
Type:count
Attributes:&redef
Default:1048576

The maximum byte size for a buffered JSON string to send to the bulk insert API.

LogElasticSearch::server_host
Type:string
Attributes:&redef
Default:""

ES server.

LogElasticSearch::server_port
Type:count
Attributes:&redef
Default:9200

ES port.

LogElasticSearch::type_prefix
Type:string
Attributes:&redef
Default:""

The ES type prefix comes before the name of the related log. e.g. prefix = “bro_” would create types of bro_dns, bro_software, etc.

Constants

LogElasticSearch::transfer_timeout
Type:interval
Default:2.0 secs

The time before an ElasticSearch transfer will timeout. Note that the fractional part of the timeout will be ignored. In particular, time specifications less than a second result in a timeout value of 0, which means “no timeout.”

Copyright 2013, The Bro Project. Last updated on June 15, 2015. Created using Sphinx 1.2.2.