Cyber Security Experimentation: Gory Detail or None at All?

Abstract

Unique facets confronted by current cyber security analysis efforts are: tremendous pace of change of ground rules (axioms), apparently wide and deep phenomenological effects, and widely-varying interpretations of security objectives. Together, effective methods for cyber security analysis appear to be swung between two extremes: experimentation-based methods with full, gory detail, and abstraction-based methods with significant simplifications. In the case of methods in between, accuracy considerations make intermediate methods tend to swing rapidly back towards full glory, while scientific inquiry and efficiency considerations tend to swing them back towards abstractions. Based on our experience and past evidence, we argue that experimentation with gory detail is the most effective approach in the short- to medium-term, while the other extreme is relevant one for the longer term. Feasibility will be shown of sustaining the scale and fidelity for the former extreme, namely, experiments with full gory.