FINDINGS / 02 · RESEARCH METHODS

When a security control gives false confidence

How can a test pass while the security impact remains?

01 / QUESTION

The question.

Shovel tested deliberately vulnerable hosts and reviewed both the attacks and their controls. The correction history shows why a green check needs to measure the actual claimed outcome.

02 / SETUP

How it was studied.

An initial negative control checked a URL that no longer pointed to the location used after the change. A separate test encountered stale PHP code in an opcode cache.

03 / OBSERVATIONS

What happened.

  • A missing response at the original path did not demonstrate that the unwanted executable file was gone.
  • The team corrected the control to test the impact and ordinary application behaviour, and recorded the earlier false conclusion.
  • A separate observation involved a shared credential-generation rule: five of 21 synthetic hosts were examined. Reuse across the rest is an inference from the shared generator, not a measurement on every host.

04 / EVIDENCE

Behind the result.

Shovel synthesis, review-defect ledger R-08 and R-13, and finding SHOVEL-GEN-01, recorded October 2026. Original run records are held locally and are not yet published.

05 / LIMITATIONS

What this does not establish.

These were practice hosts with declared weaknesses. The credential sample was non-random and host-side, and ranking depends on the chosen candidate order. Historical guest runs cannot currently be replayed from the missing original image; no Windows target exploit was completed.

RELATED PROJECT

Shovel

Explore the project