Skip to content

Change tracking and change control

Change tracking answers two questions an auditor and an on-call engineer both ask: what changed on this host, and was it authorised? The first half is passive - the agent records sessions, commands, and file changes whether or not anyone declared them. The second half is the change record: a declared, approved window that the observed activity is reconciled against.

Everything below is captured by the eBPF agent in the kernel. Nothing on the host needs to opt in, no audit daemon is required, and shell history is never trusted.

  • SSH sessions - who logged in, from where, and for how long, with trusted-source allowlists so known bastions do not page. See SSH monitoring.
  • Commands executed - process execution with full arguments, attributed to the login user rather than the effective user, so a change made after sudo still names the person.
  • File changes on watched paths - the process and user that wrote each file, with YARA scans of changed files. See Security events.
  • Out-of-hours activity - logins and changes outside business hours raise their own alerts.

A deploy inside a maintenance window looks routine. The same commands at 02:00 from an unknown IP are alerted as a breach. The change record is what tells the two apart.

A change record lives under Change in the main navigation and carries an ID of the form CHG000042. Each record has:

FieldNotes
Title, descriptionWhat is being done and why
Planned windowwindow_start and window_end; end must be after start
Target hostsOne or more servers from your fleet
Statusrequested -> approved -> started -> completed
ActorsWho requested, approved, started, and completed, with timestamps
Verificationpending, running, clean, changes_detected, or failed

The lifecycle only moves forward. An illegal transition (starting a change that was never approved, approving one already started) is rejected with 409 Conflict. Because the ordering cannot be rewound, an approval can never be back-applied to work that already happened - which is the property a SOX or PCI change-authorisation control depends on.

  1. Create - declare the title, window, and hosts. The record is in requested.
  2. Approve - an admin approves. Title, window, and hosts can still be edited up to this point.
  3. Start - the declared scope is frozen and the maintenance window opens on every target host (see below).
  4. Complete - the hosts re-lock and verification runs automatically.
  5. Verify - re-run verification on demand at any later time.

A change cannot be deleted while it is started.

Starting a change is the sanctioned way to open a maintenance window. It reuses the SSH lockdown unlock mechanism:

  • Start queues an ssh_lockdown.unlock command to each target host’s agent, held open until the change’s window_end. The hold is clamped between 15 minutes and 24 hours, matching the limits an operator has when unlocking by hand. If a host already has a longer unlock, it is never shortened.
  • Complete queues ssh_lockdown.lock_now to each host and clears the release time. The agent then re-baselines file integrity so post-change drift is measured from the new known-good state.
  • If a change is abandoned, the hosts re-lock themselves when window_end passes. Completing is for a clean audit trail; safety does not depend on it.
  • Every unlock and re-lock is written to the host’s lockdown event log tagged with source: change_management and the change UUID.

A host that cannot receive the command (no agent online) records a per-host maintenance error on the change; the change still proceeds for the other hosts.

Lockdown is enforced only on hosts whose lock posture is enabled. On a host left in the default unlocked posture, the unlock and re-lock are recorded but change nothing about access. Turn the posture on for hosts where the window should actually gate SSH.

On completion, verification queries the file-integrity changes recorded on each target host between the actual start and completion times (plus a 10 minute grace period) - not the planned window. It lists the distinct paths that changed, up to 200 per host, and sets the verdict:

  • clean - no file-integrity changes were recorded during the window.
  • changes_detected - files changed; the per-host path list is on the change. This is information, not a failure: it is the evidence of what the change touched.
  • failed - the query could not run for every host.

Verification currently lists changed files only. It does not run health checks or compare against a declared file scope.

Reconciliation: finding unauthorised change

Section titled “Reconciliation: finding unauthorised change”

Verification looks inward at one change. Reconciliation looks at a period and sorts every change-constituting security event on your hosts into three buckets:

  • Authorised - the event fell inside a started change’s actual window (plus grace) for that host.
  • Unauthorised - the exception list, with timestamp, host, path, rule, actor, and process command line.
  • Unobserved - an approved and started change during which nothing changed on disk.

Only rules that constitute a change count: sysctl, sudoers, PAM, cron, audit-log and sensitive-file writes, file-integrity violations, and destructive recursive deletes. SSH logins are not treated as changes. A change planned for 02:00 but executed at 14:00 is an unauthorised change at 14:00; the planned window never launders it.

GET /api/v1/changes/reconciliation?start_date=...&end_date=... returns JSON, or CSV with format=csv. The CSV carries a provenance header (period, generated by, all counts, coverage warnings) so it can be handed to an auditor as-is. The coverage section says plainly whether the period is fully covered by retained events, so “no exceptions” can be distinguished from “no visibility”.

The change-control release added detection rules that fire regardless of any window: destructive recursive delete (pm-008), external download by fetch tool (ns-014), container pulled from an unauthorised registry (cs-005, active once you configure allowed registries), and an extended sensitive-file rule (fi-009) covering /etc/sudoers.d, /etc/systemd, and /etc/docker. They are detection-only by design. See Rules customisation.

Driving changes from an assistant or a pipeline

Section titled “Driving changes from an assistant or a pipeline”

The MCP server exposes the lifecycle as tools, so an AI assistant or a CI job can run a deploy inside a declared change:

ToolEffect
list_changes, get_changeRead records, filter by status or host
create_changeTitle, window, host_uuids (from list_servers)
approve_changerequested -> approved
start_changeOpens the SSH window on the target hosts
complete_changeRe-locks and starts verification
verify_changeRe-runs verification

The same operations exist over the REST API under /api/v1/changes/ and accept a tenant API key, which is how AlertKick’s own deploy pipelines open and close their windows. See AI assistants and the API reference.

Keep the verifier independent: an agent that operates your servers should not be the thing that audits them. AlertKick sits outside the actor and records ground truth at the kernel.

A ticket system proves what was ticketed. The question an auditor asks is how you know the population of changes is complete, and the honest answer needs a second, independently generated population. Kernel capture provides it; the change record provides the authorisation; reconciliation provides the exception list; the lockdown event log proves the window was enforced rather than asserted. See Compliance reports for how this evidence is packaged.