关于解析：Logstash日期无效的格式

Logstash date invalid format

尝试从rsylog服务器解析日志并将其插入elasticsearch。
我的传入日志行是

1	Feb 13 01:17:11 xxxx xxx-xxxx_error 2016/02/13 01:17:02 [error] 13689#0: *1956118 open()"xxxxxx" failed (2: No such file or directory), client: xx.xx.xx.xx, server: xxxxx.xx, request:"xxxxxxx HTTP/1.1", host:"xxxxx.xx"

我正在使用以下logstash过滤器提取字段：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

grok {
match => {
"message" => [
"(?<logstamp>\\h{3} \\d{2} \\d{2}:\\d{2}:\\d{2}) %{WORD:hostname} (?<source>[^\\s]+) (?<timestamp>\\d{4}/\\d{2}/\\d{2} \\d{2}:\\d{2}:\\d{2}) %{GREEDYDATA:error_message}"
]
}

date {
locale =>"en"
match => ["timestamp","yyyy/MM/dd HH:mm:ss" ]
}
}

mutate {
remove_field => ["@version","_score","message","host","_type","logstamp" ]
}

基于http://grokdebug.herokuapp.com/，我的语法是理智的。
我在日志行中有两个日期，因为第一个日期是rsyslog接收到该行的日期，第二个日期是来自nginx的日期。我想要的是将第二个传递给"时间戳"。

我在logstash中得到的错误是：

1
2
3
4

@metadata_accessors=#<LogStash::Util::Accessors:0x1d630482 @store={"path"=>"..."}, @lut={"[path]"=>[{"path"=>"..."},
"path"]}>, @cancelled=false>], :response=>{"create"=>{"_index"=>"...","_type"=>"...","_id"=>"...","status"=>400,"error"=>{"type"=>"mapper_parsing_exception",
"reason"=>"failed to parse [timestamp]","caused_by"=>{"type"=>"illegal_argument_exception",
"reason"=>"Invalid format: "2016/02/16 12:25:16" is malformed at "/02/16 12:25:16""}}}}, :level=>:warn}

(我剪切了输出以使其更短)

编辑：工作配置
我最终将时间戳从Nginx日志转换为更标准的时间戳(如ruby部分所示)，并在date匹配项中将其用作@timestamp。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

grok {
match => {
"message" => [
"(?<logstamp>\\h{3} \\d{2} \\d{2}:\\d{2}:\\d{2}) %{WORD:hostname} (?<source>[^\\s]+) (?<ngxstamp>[^\\s]+ [^\\s]+) %{GREEDYDATA:error_message}"
]
}
}

ruby {
code =>"event['ngxstamp'] = event.timestamp.time.localtime.strftime('%Y-%m-%d %H:%M:%S')"
}

date {
match => ["ngxstamp","yyyy-MM-dd HH:mm:ss" ]
locale =>"en"
}

mutate {
remove_field => ["@version","_score","message","host","_type","logstamp" ]
}