The development of Artificial General Intelligence (AGI) is widely considered inevitable. However, the actions of a self-aware AGI are uncertain, with significant research suggesting a non-trivial likelihood of hostility towards humans. This paper addresses the timely and critical research topic of AGI containment. While existing work explores strategies like safe language semantics and sandboxing, it is often bounded by its originating field. This study constructs a foundational domain ontology to describe the necessary elements for future AGI containment technologies, situating the problem within the comprehensive framework of cyber science.
AGI represents AI with intelligence at or beyond human capability, operating in dynamic, general scopes. This poses a direct and sustained danger compared to narrow AI.
Inspired by Babcock, Kramar, & Yampolskiy (2016), containment is seen as necessitating a combination of traditional cybersecurity technologies. The paper recognizes the existential risks framed by thinkers like Nick Bostrom, making containment a paramount safety concern.
Traditional paradigms (firewalls, intrusion detection) are recognized as having limited capacity to address the unique, adaptive threat of a superintelligent AGI. Their reactive nature is ill-suited for a proactive, general intelligence.
The paper pivots to cyber science, a nascent field offering a more comprehensive knowledge context. It leverages Kott's (2015) definition of cyber operations involving malicious software, drawing a parallel where an AGI or its containment apparatus could be viewed as "malicious" from the opposing perspective, highlighting the need for a structured ontology to organize these constructs.
The ontology was developed through a systematic analysis of existing academic literature on AGI, cybersecurity, and containment strategies.
A hierarchical, domain-specific ontology was built to categorize and relate core concepts. The development process involved identifying key entities, properties, and relationships from the literature.
The analysis distilled fundamental constructs necessary to model the AGI containment scenario, leading to the identification of three primary agent objects.
The core contribution is a single domain ontology designed to provide a common vocabulary and conceptual framework for AGI containment research.
The ontology is organized into five distinct hierarchical levels, containing a total of 32 codes, each with an associated descriptor. These levels logically progress from abstract foundational concepts to concrete implementation mechanisms.
The paper includes ontology diagrams to visually demonstrate the intended relationships between the identified constructs, such as "contains," "monitors," "threatens," and "operates within."
A key insight is the formal identification of three novel agent objects critical for modeling containment:
This tripartite model is essential for moving beyond simple human-vs-AI narratives to a more nuanced systemic view.
The authors posit that their work addresses three significant gaps in the field:
Core Insight: The paper's most valuable contribution isn't a new containment algorithm, but a crucial meta-framework. It correctly diagnoses that the AGI containment debate is mired in ad-hoc, field-specific solutions (CS, philosophy, security) and lacks a unified language. By proposing a cyber-science ontology, it attempts to build the conceptual plumbing necessary for rigorous, interdisciplinary research. This aligns with lessons from mature fields; for instance, the development of the STRIPS planning language was pivotal for AI planning research, providing a common ground for problem formulation and solution comparison.
Logical Flow: The argument is sound: 1) AGI risk is real and containment is needed. 2) Current cybersecurity is insufficient (a well-taken point, echoed in critiques of ML security by Papernot et al.). 3) Therefore, we need a broader foundation—enter Cyber Science. 4) To build within this foundation, we first need a structured ontology to define our terms and relationships. The flow from problem identification to proposed foundational solution is clear and logical.
Strengths & Flaws: The strength is its prescient focus on systematization. The tripartite agent model (Human/AGI/Cyber World) is elegant and necessary. However, the paper's major flaw is its extreme abstraction. It presents the ontology's existence (5 levels, 32 codes) as the result, without publishing the ontology itself. What are the 32 codes? What are the five levels? This is akin to announcing a new programming language syntax but not releasing the grammar. Without this detail, the work remains a promissory note, limiting immediate utility. Furthermore, while citing Kott's cyber science definition is clever, it risks stretching the "malicious software" analogy too thin for a superintelligent agent.
Actionable Insights: For researchers: The immediate task is to operationalize this ontology. Publish the full code set and descriptors. Use it to formally model specific containment scenarios (e.g., an AGI attempting to exfiltrate via a side-channel) and identify gaps in existing strategies. For practitioners and policymakers: This paper underscores that AGI safety is not just an AI problem but a cyber-physical systems security problem on a cosmic scale. Investment should flow not only into AI alignment research but also into foundational work on verifiable computing, formal methods for complex systems, and cyber-resilient architectures, as championed by institutions like DARPA's Guaranteeing AI Robustness against Deception (GARD) program.
While the PDF does not elaborate on mathematical formalisms, a robust ontology for AGI containment should be expressible in a logic-based framework. A potential formalization using description logic or first-order logic could define agents, actions, and containment relations.
Example Formal Snippet (Conceptual):
Let $A$ be the set of Agents, where $A = \{Human, AGI, CyberEntity\}$.
Let $S$ be the set of States of the Cyber World.
Let $Cap(x)$ denote the capabilities of agent $x$.
A core containment axiom might be:
$\forall s \in S, \forall a \in A \text{ where } a = AGI: \text{ContainmentProtocol}(s) \rightarrow \text{Limits}(Cap(a), s)$
This states that for all states and for the AGI agent, an active containment protocol implies a limitation on the AGI's capabilities within that state.
The ontology likely implies a state-transition model where actions by AGI or human defenders change the system state, and containment is a property to be verified across possible transitions, akin to model checking in computer security.
Scenario: Analyzing a proposed "AI Sandbox" using the Pittman & Crosby ontology.
Steps: