← All projects Case study · SIH 2026 · problem statement SIH26153 (NTRO)

ShadowCat: early attack warning with a log nobody can quietly edit

Problem: Move network defence from signature matching to learned estimates of whether an attack is under way or about to start, with every forecast provable later.
My role: Team lead of a 6-member team that built both ShadowCat and SHIKRA. Here I built the data ingestion into the network-state representation and the cryptographic audit chain.
Links: Repo Every claim, with source

0.990detection F1, 0.0% false alarms

0.907onset (≤ 5 min) F1, 0.4% false alarms

0.707ROC-AUC on attack families never seen

ML lens

Detection, onset and the world model

Representation

Each minute of traffic becomes a 406-feature "unified cyber state": 388 flow features, 12 packet-level features parsed from raw captures, and 6 presence masks so missing telemetry is marked absent instead of zero-filled. An LSTM world model learns how that state evolves over a 30-minute lookback.

Evaluation protocol

37-fold leave-one-episode-out on CSE-CIC-IDS2018, with a 36-window purge around each held-out episode so neighbouring minutes can't leak.
Scaler, PCA and models refit inside every fold.
Decision thresholds chosen without any test labels.
A separate leave-one-day-out stress test, where whole attack families are unseen.

Task	Model	ROC-AUC	F1	False-alarm rate
Detection	Stacked ensemble	1.000	0.990	0.0%
Detection	Logistic regression	1.000	0.980	0.0%
Onset ≤ 5 min	Stacked ensemble	0.962	0.907	0.4%
Onset ≤ 5 min	Logistic regression	0.952	0.896	0.4%

Why the near-perfect detection score is not the headline: even a linear model detects attacks almost perfectly on this benchmark, so the representation does most of the work. The harder test is unseen attack families, where supervised models drop to ROC-AUC 0.666 and the world model's deviation score reaches 0.707. That gap is the case for combining both.

What did not work, and why

The problem statement asks for forecasts several minutes ahead. Across 11 experiments (capacity sweeps, a mixture-density head, Monte Carlo rollouts, 10 capture days instead of 6, and three other datasets), the reliable forecast horizon stayed at zero. Two measured causes: only 71 of 2,787 windows actually precede an attack, and a Gaussian next-state loss collapses toward the majority class. We reported the horizon with confidence intervals instead of tuning until a number looked good.

A GraphSAGE graph-fusion model had lower test error but worse validation loss, so it was held back: choosing it on test error would have leaked the test set.

Systems lens

Ingestion and a provable audit trail

Ingestion

Accepts CICFlowMeter flow CSVs or raw PCAP / PCAPNG uploads; other schemas are mapped or rejected, never guessed.
A packet parser extracts TTL, fragmentation, payload percentiles, retransmissions and a port-scan score from raw captures.
Everything is bucketed into UTC one-minute windows with explicit presence masks.

Audit chain

Every model, report and forecast is appended to a SHA-256 hash chain signed with Ed25519. A signed Merkle root (RFC 6962 hashing) commits to the whole chain, so anyone can check that one forecast is in the log with an O(log n) inclusion proof, and a chain rebuilt with recomputed links is still detected. Hyperledger Fabric notarisation is optional, with automatic fallback.

Try it: tamper with a forecast log

A simplified version of ShadowCat's audit chain: a registered model, a forecast it made, and the report that cites it. Each entry stores the SHA-256 hash of the one before it. Change the forecast's probability, as someone hiding a missed alert might, and every later entry stops verifying. The real chain also signs each entry with Ed25519 and commits to all of them with a Merkle root.

Chain verified · 3 of 3 entries intact

ShadowCat pipeline: dataset feasibility gate, input data, data engineering, unified cyber state, world model and prediction heads, leakage control, audit chain and offline dashboard — Pipeline, from the dataset feasibility check through leakage control to the signed audit trail and offline dashboard.

Quality and reproducibility

75 automated tests, including one that blocks known placeholder values from reaching the dashboard.
python reproduce.py recomputes every headline number in about 5 minutes on a laptop CPU and checks it against the committed artifacts; --full retrains.

What I'd do next

Train the forecasting pipeline on whole-network captures of multi-stage campaigns, which the public datasets we checked do not have.
Publish per-fold variance alongside the pooled scores on the project page.