2.9. logstash扩展配置
首先得明确以下几点
1、logstash安装在/opt/logstash
2、logstash配置目录为/etc/logstash/conf.d
3、确定存在名为02-beats-input.conf配置文件，该文件在上文之前创建配置过
4、确定存在名为30-elasticsearch-output.conf配置文件，该文件在上文之前创建配置过
创建patterns
$ sudo mkdir -p /opt/logstash/patterns
$ sudo chown logstash: /opt/logstash/patterns
修改/etc/filebeat/filebeat.yml文件 filebeat:
prospectors: -
document_type: syslog paths:
- /var/log/secure - /var/log/messages -
document_type: sys-log input_type: log paths:
- /var/log/*.log
registry_file: /var/lib/filebeat/registry logging: files:
rotateeverybytes: 10485760 output: logstash:
bulk_max_size: 1024 hosts:
- \ tls:
certificate_authorities:
- /etc/pki/tls/certs/logstash-forwarder.crt shipper: ~
2.9.1. Nginx日志配置
2.9.1.1. Logstash Patterns: Nginx
$ sudo mkdir -p /opt/logstash/patterns
sudo vim /opt/logstash/patterns/nginx NGUSERNAME [a-zA-Z\\.\\@\\-\\+_%]+ NGUSER %{NGUSERNAME} NGINXACCESS %{IPORHOST:clientip} %{NGUSER:ident} %{NGUSER:auth} \\[%{HTTPDATE:timestamp}\\] \%{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\%{NUMBER:response} (?:%{NUMBER:bytes}|-) (?:\$ sudo chown logstash: /opt/logstash/patterns/nginx
2.9.1.2. Logstash Filter: Nginx
$ sudo vim /etc/logstash/conf.d/11-nginx-filter.conf filter {
if [type] == \ grok {
match => { \ } } }
2.9.1.3. 重启logstash
$ sudo service logstash restart
2.9.1.4. Filebeat Prospector: Nginx
修改/etc/filebeat/filebeat.yml配置 $ sudo vim /etc/filebeat/filebeat.yml filebeat:
prospectors: -
document_type: nginx-access paths:
- /var/log/nginx/access.log registry_file: /var/lib/filebeat/registry logging: files:
rotateeverybytes: 10485760 output: logstash:
bulk_max_size: 1024 hosts:
- \ tls:
certificate_authorities:
- /etc/pki/tls/certs/logstash-forwarder.crt shipper: ~
2.9.1.5. 重启filebeat
$ sudo service filebeat restart
2.9.1.6. kibana搜索效果图
2.9.2. Apache HTTP Web Server日志配置
2.9.2.1. Logstash Filter: Apache
$ sudo vi /etc/logstash/conf.d/12-apache.conf filter {
if [type] == \ grok {
match => { \ } } }
2.9.2.2. 重启logstash
$ sudo service logstash restart
2.9.2.3. Filebeat Prospector: Apache
$ sudo vim /etc/filebeat/filebeat.yml filebeat:
prospectors: -
document_type: apache-access input_type: log paths:
- /var/log/apache2/access.log registry_file: /var/lib/filebeat/registry