{
  "docId": "019dd923-5e88-73ef-bd5d-0d0f98caffe1",
  "docSlug": "5df6bd1b0447b5f6",
  "documentTitle": "2025 Air Street Capital The State of AI Report 2025",
  "authorId": "AirStreetCapital",
  "authorName": "Air Street Capital",
  "documentKindSlug": "consulting-deck",
  "documentKindLabel": "Consulting deck",
  "sourceTypeSlug": "vc_research",
  "sourceTypeLabel": "VC research",
  "presentationDate": null,
  "orientation": "landscape",
  "aspectRatio": 1.777,
  "pageNumber": 266,
  "pageCount": 313,
  "prevPage": 265,
  "nextPage": 267,
  "slideType": "diagnosis",
  "function": "diagnose",
  "density": "overcrowded",
  "nDataPoints": 4,
  "notes": "Includes two embedded infographics/diagrams from OpenAI/Apollo Research.",
  "elementsJson": [
    "headline_text",
    "bullet_list",
    "screenshot",
    "process_diagram"
  ],
  "metadataConfidence": 1,
  "imagePath": null,
  "slideHref": "/slides/019dd923-5e88-73ef-bd5d-0d0f98caffe1/266",
  "deckHref": "/decks/019dd923-5e88-73ef-bd5d-0d0f98caffe1",
  "deckJsonHref": "/decks/019dd923-5e88-73ef-bd5d-0d0f98caffe1.json",
  "deckAnchorHref": "/decks/019dd923-5e88-73ef-bd5d-0d0f98caffe1#slide-266",
  "components": [
    {
      "bbox": null,
      "kind": "callout",
      "text": "In these controlled tests, covert behaviors fell by as much as 30x, yet techniques remain imperfect.",
      "attrs": null,
      "subkind": null,
      "toolName": "Visual emphasis",
      "toolSlug": "visual-emphasis",
      "confidence": null,
      "componentId": "019dd952-47c3-7407-8d08-0a8c9b14831f",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.25,
        "w": 0.4,
        "x": 0.58,
        "y": 0.28
      },
      "kind": "image",
      "text": "Apollo Research / OpenAI Anti-Scheming evaluation summary",
      "attrs": null,
      "subkind": "infographic",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "6a9acf36-b20d-4a1e-9dfb-0911a5038796",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.3,
        "w": 0.4,
        "x": 0.58,
        "y": 0.58
      },
      "kind": "image",
      "text": "Why Situational Awareness Makes Evaluating Scheming Hard",
      "attrs": null,
      "subkind": "infographic",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "71605892-1b00-46aa-b135-2c88d045149d",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.15,
        "w": 0.55,
        "x": 0.03,
        "y": 0.28
      },
      "kind": "list",
      "text": "Gains from a leading training technique, deliberative alignment, did not fully generalize to more realistic and OOD scenarios. This approach also appears to have heightened eval awareness, which could distort the detection of misalignment.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "271f7365-4038-45a4-977e-713d4622cbee",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.1,
        "w": 0.55,
        "x": 0.03,
        "y": 0.62
      },
      "kind": "list",
      "text": "Although traces of scheming were found in every frontier model tested, OAI co-founder Wojciech Zaremba noted that “no harmful scheming has been seen in production traffic.”",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "2c7d3e1a-0256-4486-b648-0168a31034b0",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.08,
        "w": 0.55,
        "x": 0.03,
        "y": 0.75
      },
      "kind": "list",
      "text": "As part of their commitment to anti-scheming, OpenAI is also launching a $500K Kaggle challenge to promote the investigation of new countermeasures.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "2edd31cf-2431-4cc3-a0dc-70d76be80207",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.15,
        "w": 0.55,
        "x": 0.03,
        "y": 0.45
      },
      "kind": "list",
      "text": "Researchers also monitored the effects of capabilities-focused RL runs (without safety training), where evaluation awareness increased from 2.0% to 20.6%. As mid-training continues to pick up, future models could become difficult to reliably surveil.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "f57b33e2-2ebb-49ec-85eb-d0bffbf4be05",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": null,
      "kind": "metric",
      "text": "Covert behavior rate: 30x",
      "attrs": null,
      "subkind": "primary",
      "toolName": "Quantification",
      "toolSlug": "quantification",
      "confidence": null,
      "componentId": "019dd952-47c3-7407-8d08-11e239eae003",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.08,
        "w": 0.55,
        "x": 0.03,
        "y": 0.18
      },
      "kind": "paragraph",
      "text": "In collaboration with Apollo, OpenAI ran extensive evaluations to determine the efficacy of anti-scheming training. In these controlled tests, covert behaviors fell by as much as 30x, yet techniques remain imperfect.",
      "attrs": null,
      "subkind": "paragraph",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "24bf8ae6-254e-4685-ad62-9b78dec62cc5",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": null,
      "kind": "quote",
      "text": "no harmful scheming has been seen in production traffic. — Wojciech Zaremba",
      "attrs": null,
      "subkind": null,
      "toolName": "Authority citation",
      "toolSlug": "authority-citation",
      "confidence": null,
      "componentId": "019dd952-47c3-7407-8d08-0ef934286c54",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.05,
        "w": 0.7,
        "x": 0.03,
        "y": 0.12
      },
      "kind": "title",
      "text": "Despite breakthroughs, there is still no fool-proof mitigation for scheming",
      "attrs": null,
      "subkind": "headline",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "e3b19f16-926d-47fb-853c-2f5e95dbf224",
      "frameworkName": null,
      "frameworkSlug": null
    }
  ],
  "metrics": [],
  "tools": [
    {
      "name": "Pyramid Principle",
      "slug": "pyramid-principle",
      "agent": "Architect",
      "layer": "slide",
      "matchId": "0ba2ee36-9a1d-4017-92d7-fd93a6fc4108",
      "evidence": "list/bullet: Researchers also monitored the effects of capabilities-focused RL runs (without safety training), where evaluation awareness increased from 2.0% to 20.6%.",
      "confidence": 0.7
    }
  ],
  "frameworks": [],
  "arcBeats": [
    {
      "to": 282,
      "from": 190,
      "beatId": "019dd95a-0682-776c-8e35-a256fe16ea9c",
      "arcName": "The Triple Take",
      "arcSlug": "triple-take",
      "beatName": "The Implications (So What)",
      "beatSlug": "triple-take-the-implications-so-what",
      "evidence": "Sections 3-4 frame political and safety consequences of the technical/industry trends.",
      "position": 2,
      "confidence": 70,
      "parentBeatName": "Reflection",
      "parentBeatSlug": "reflection"
    },
    {
      "to": 282,
      "from": 190,
      "beatId": "019dd95a-0682-776c-8e35-b713af60bf88",
      "arcName": "The Onion",
      "arcSlug": "onion",
      "beatName": "Core Insight",
      "beatSlug": "onion-core-insight",
      "evidence": "Politics and Safety reveal the structural risks at the core.",
      "position": 4,
      "confidence": 45,
      "parentBeatName": "Turn",
      "parentBeatSlug": "turn"
    }
  ],
  "loops": [
    {
      "to": 282,
      "from": 247,
      "name": "Pre Mortem",
      "slug": "19-pre-mortem",
      "bestFor": "Project kick-offs, risk management, building trust with skeptical stakeholders",
      "matchId": "019dd95a-07fe-70ce-8d3e-bb4d43115ab4",
      "evidence": "Vibe hacking, AI psychosis, alignment faking, scheming, prompt injection — what could go wrong + mitigations.",
      "position": 12,
      "objective": "Surface AI safety failure modes before they become catastrophic",
      "structure": "The Future Disaster (Hypothetical) -> What Went Wrong -> The Prevention Plan",
      "confidence": 75,
      "description": "Build trust by anticipating failure modes before they happen and solving them in advance"
    }
  ],
  "imagePathAlt": null,
  "thumbSrc": null,
  "thumbSrcAlt": null,
  "locked": true
}