Escaping characters in parse_cef?

[sb1-nicolai]

Hi.

I am trying to get the parse_cef() function to escape characters, but I am unable to get it to parse fields that include = such as: ... cs4=None request="https://foo.com?id=foo&a=bar" abc=lol

Here is a link to the VRL Playground with example data: link

If I am not mistaken CEF-formatted data should include quotes for data that requires it (as it does in my example) but I cannot get Vector to parse it correctly and have run out of ideas to test.

Currently all CEF-formatted URL logs that include parameters are being dropped by the transformer and is what I am trying to fix.

There is a chance I have overlooked something here, but from what I could see the only option to adjust the parse_cef function is translate_custom_fields which unfortunately does not help much in this case.

Is it possible to use parse_cef or is it better to go with a custom transformer such as the one mentioned in #6451?

[sb1-nicolai]

Here is a dirty hack that can be used temporarily for arbitrary fields containing = or other characters that break the CEF parser before using parse_cef():

.cef_blob = replace_with!(string!(.message), r'request=[^ ]+') -> |match| {
    "request=" + replace(slice!(match.string, length("request="), length(match.string)), "=", "\\=")
}

. = parse_cef!(.cef_blob, translate_custom_fields: true)

Which produces:

{
	"EventID": "52b06812ec3500ed864c461e",
	"ExternalLink": "None",
	"ExtraData": "None",
	"PTAlink": "https://1.1.1.1/incidents/52b06812ec3500ed864c461e",
	"abc": "foobar",
	"cefVersion": "0",
	"detectionDate": "1388577900000",
	"deviceEventClassId": "1",
	"deviceProduct": "PTA",
	"deviceVendor": "CyberArk",
	"deviceVersion": "12.6",
	"dhost": "dev1.domain.com",
	"dst": "2.2.2.2",
	"duser": "andy@dev1.domain.com",
	"name": "Suspected credentials theft",
	"request": "https://foo.com?id=foo&a=bar",
	"severity": "8",
	"shost": "prod1.domain.com",
	"src": "1.1.1.1",
	"suser": "mike2@prod1.domain.com"
}

Edit 2:

Here is also an alternative regex that can be used:

.cef_blob = replace_with!(string!(.cef_blob), r'requestClientApplication=(.*?) \w+') -> |match| {
    "requestClientApplication=" + replace(slice!(match.string, length("requestClientApplication="), length(match.string)), "=", "\\=")
}