Parsing a nested JSON audit feed with conflicting custom properties
A custom SaaS audit application sends JSON over syslog/TCP to a QRadar 7.5 Event Collector. Payload fields are nested: actor.name=jsmith, action=login, result=failure, src.ip=10.20.5.7, and a long session token. QRadar auto-discovered the source as SIM Generic. An admin created a custom property User Name with regex user=(\S+); it returns blank. Another property Source IP uses regex ip=([0-9.]+); it captures the wrong address when the payload contains multiple IPs. Events are searchable only by raw payload. The application cannot change its format or add a delimiter. You must make actor.name, action, result, and src.ip usable in rules and searches without creating an offense for every session token.
1. What is the correct parsing approach, and why is the existing regex failing?
Read the worked answer
Reasoning: the payload is JSON with nested keys, not key=value. The regex user=(\S+) never matches a nested name field, so the property is blank by design. The ip=([0-9.]+) regex also matches an arbitrary ip occurrence, not necessarily src.ip. Decision: parse with the DSM Editor JSON field mapping if the installed DSM supports JSON; otherwise create a log source extension that extracts actor.name, action, result, and src.ip from the JSON structure. Map src.ip to the built-in Source IP field, not a custom property named Source IP. Do not rely on broad regex for nested JSON unless the format is fixed. Test with the exact sample payload in DSM Editor, deploy, then verify Log Activity shows populated fields and rules can match them.
2. How would you control the session token so it does not hurt performance?
Read the worked answer
Reasoning: custom properties are extracted and stored at ingest; a high-cardinality token increases index size and search cost with little detection value. The token is useful only for raw-payload forensics. Decision: do not define the session token as a custom property for rule matching. If a searchable field is mandatory, extract a truncated hash or prefix and mark it not optimized; document that it is not for offenses. Keep actor.name, action, result, and src.ip as the only rule-facing fields. Validate by ingesting a test burst and checking property cardinality, /store/ariel growth, and search time; if cardinality is still high, remove the property and search raw payload instead.
3. How do you test and validate the change end to end?
Read the worked answer
Use a three-stage test: (1) DSM Editor sample test with the exact JSON, including a second event with a different actor.name and src.ip; (2) send from the application to a non-production log source or after hours, then search Log Activity filtered by the log source and add columns for each field; (3) create a disabled test rule that matches action=login AND result=failure and use Rule Test on the collected events. Check that the source is not duplicated, the built-in Source IP is correct, and no session-token property appears. Only then enable and deploy; monitor for 24 hours.