Tag: filebeat

How to check socket connection ?

How to check socket connection between filebeat, logstash and elasticseearch ?

netstat -anp | grep 9200
netstat -anp | grep 5044

a – Show all listening and non-listening sockets
n – numberical address
p – process id and name that socket belongs to

9200 – Elasticsearch port
5044 – Filebeat port

ESTABLISHED” status for the sockets that established connection between logstash and elasticseearch / filebeat.

LISTEN” status for the sockets that listening for incoming connections.

To view the count of socket, use

 wc -l

command

Filebeat

Logstash
Grok
Elasticsearch
Kibana

Filebeat

./bin/plugin install logstash-input-beats
Update the beats plugin if it is 92 then it should be to 96
If [fields][appid] == appid
No tab allowed filebeat.yml file
curl -L -O https://download.elastic.co/beats/filebeat/filebeat-1.0.1-x86_64.rpm
/etc/filebeat.yml
Output to Logstash
curl -XPUT ‘http://localhost:9200/_template/filebeat?pretty’ -d@/etc/filebeat/filebeat.template.json

/usr/bin/filebeat -c /etc/filebeat/filebeat.yml –e

Install Filebeat
In the remote host, download filebeat-1.0.1 from elastic link below and install the RPM with root access.
curl -L -O https://download.elastic.co/beats/filebeat/filebeat-1.0.1-x86_64.rpm

rpm -ivh filebeat-1.0.1-x86_64.rpm

Contents of filebeat
ilamuruhan@ubuntu:/opt$ rpm -qlp filebeat-1.0.1-x86_64.rpm
/etc/filebeat/filebeat.template.json
/etc/filebeat/filebeat.yml
/etc/init.d/filebeat
/usr/bin/filebeat
/usr/bin/filebeat-god

Configure filebeat
File Name : /etc/filebeat/filebeat.yml
Prospector – where from file to read
prospectors:

paths:
– /var/log/*.log
input_type: log
document_type: beat
registry: /var/lib/filebeat/registry

Output – where to send file
output:
logstash:
hosts: [“10.0.0.1:5044”]

Logging – filebeat own logs
logging:
to_files: true
files:
path: /var/log/filebeat
name: filebeat
rotateeverybytes: 10485760
level: error

Configure the path

– Get everything sub-directories
– Track the progress of the file and recognizes the log rotation
– Send logs from remote to Logstash or Elasticsearch