BIPI
BIPI

Log Sources That Actually Matter for IR: A Prioritized List

Cybersecurity

When the incident lands, the SOC discovers which log sources it actually needs. A prioritized list of telemetry that earns its retention cost during real incident response, ranked by frequency of investigative use rather than vendor enthusiasm.

By Arjun Raghavan, Security & Systems Lead, BIPI · September 23, 2023 · 9 min read

#incident-response#siem#log-sources#soc#telemetry

Every SOC has a log ingestion bill they cannot fully justify and a retention policy they cannot fully defend. When the incident hits, half of what they pay for is irrelevant and half of what they need is missing or retained for too short a window. Prioritization by investigative value, not vendor pitch, fixes both problems.

Tier One: Always On, Always Long Retention

These sources show up in nearly every incident investigation and must be retained for at least 12 months, ideally 18. The retention cost is real but the alternative is reconstructing intrusions from incomplete evidence.

  • Endpoint process creation telemetry: Sysmon event ID 1 or EDR equivalent, the single most important data source for IR
  • Authentication events: Windows 4624 and 4625, Entra ID sign in logs, Okta system log, SSH auth logs for Linux fleet
  • DNS query logs: required for retroactive C2 discovery when a new indicator is published
  • Web proxy or DNS firewall logs: outbound HTTP and HTTPS requests, even just the SNI, are critical for exfiltration analysis
  • VPN and remote access logs: required for any external compromise investigation
  • EDR alerts and detections: must be retained even for false positives, because patterns emerge across alerts

Tier Two: Always On, 90 to 180 Day Retention

These sources are crucial for active investigation but rarely needed for incidents older than six months. Retain hot for query speed during incidents.

  1. Sysmon event ID 3 network connections, event ID 10 process access, event ID 22 DNS query at process level
  2. Cloud audit logs: AWS CloudTrail, Azure Activity, GCP Cloud Audit Logs, retain management plane events longer than data plane
  3. Email security logs: gateway delivery, click events, attachment detonation results
  4. Application authentication: SaaS app sign in events, especially M365, Workspace, Salesforce, GitHub
  5. Firewall connection logs: full session metadata, not just denied traffic

Tier Three: On Demand or Sampled

These sources have detection value but are not worth retaining at full fidelity. Sample, summarize, or capture only on demand during incidents.

  • Full packet capture: extraordinarily expensive, justify only for crown jewel network segments and short retention
  • Sysmon event ID 7 image loaded: high volume, capture on a pilot ring or sample at random
  • Verbose application logs from internal apps: retain in app, not in SIEM, with hooks to pull on demand
  • Endpoint registry change events at full fidelity, scope to sensitive keys only

Cloud and SaaS Are Often Underrated

Most modern intrusions touch cloud and SaaS within hours of initial access. T1078.004 valid cloud accounts, T1098.001 additional cloud credentials, T1530 data from cloud storage all require cloud audit telemetry. Yet many SOCs onboard CloudTrail at summary fidelity, drop GetObject events to save volume, and miss the data staging that precedes exfiltration. Onboard full CloudTrail, retain management events for 18 months, and keep at least 90 days of data plane events for sensitive buckets.

Identity Is the New Perimeter

If you can only prioritize three sources, prioritize endpoint process creation, authentication events across all identity providers, and DNS. These three reconstruct most intrusions even when the attacker is careful. Conversely, a SOC with great firewall logs and no identity telemetry will lose every account compromise investigation.

The right test for a log source is not whether it generates dashboards, but whether you would miss it at 2 a.m. on day three of a real intrusion. Retention follows that test.

Build a Quarterly Review

Every quarter, review the log sources used in the past quarter's incidents and hunts. Sources used in three or more investigations move up a tier or extend retention. Sources used in zero investigations across the year are candidates for sampling or removal. The retention bill should be defended by investigative receipts, not by vendor recommendations.

Read more field notes, explore our services, or get in touch at info@bipi.in. Privacy Policy · Terms.