Tuesday, November 29, 2022
Converting custom log timestamt or date time format into @timestamp - elastic, logstasch
To convert custom log timestamp into @timestamp you should do next:
variable timestamp has value like and you prepare it or read it with grok.
21-12-2022 14:33:12,001
And this value is transformed into cy, cm, .... and on the end created as date into variable timestampd. Be careful about milliseconds and millisecond delimiter. If you have . you should put . into formater.
grok {
match => { "timestamp" => "%{MONTHDAY:cd}-%{MONTHDAY:cm}-%{YEAR:cy} %{HOUR:chr}:%{MINUTE:cmm}:%{SECOND:cs}" }
overwrite => [ "cy" ]
overwrite => [ "cm" ]
overwrite => [ "cd" ]
overwrite => [ "chr" ]
overwrite => [ "cmm" ]
overwrite => [ "cs" ]
}
}
mutate {
add_field => {
"timestampd" => "%{cy}-%{cm}-%{cd} %{chr}:%{cmm}:%{cs}"
}
remove_field => ["cd", "cm", "cy", "chr", "cmm", "cs"]
}
date{
match => ["timestampd", "YYYY-MM-dd HH:mm:ss,SSS"]
target => "timestampd"
}
At the end you should do the trick about converting date to string and back to @timestamp. Without this I did not have luck.
mutate {
convert => {"timestampd" => "string"}
}
date{
match => ["timestampd", "YYYY-MM-dd'T'HH:mm:ss'.'SSS'Z'"]
timezone => "UTC"
target => "@timestamp"
}
Solution created based on forums:
https://discuss.elastic.co/t/how-to-parse-date-field-into-timestamp/107396/9 by sdussin
https://discuss.elastic.co/t/how-to-convert-the-date-string-into-datetime-format/192941 by Karn_Gusain
# posted by Domen Dolar @ 8:06 AM 
