{
  "docId": "019dd923-5e88-73ef-bd5d-0d0f98caffe1",
  "docSlug": "5df6bd1b0447b5f6",
  "documentTitle": "2025 Air Street Capital The State of AI Report 2025",
  "authorId": "AirStreetCapital",
  "authorName": "Air Street Capital",
  "documentKindSlug": "consulting-deck",
  "documentKindLabel": "Consulting deck",
  "sourceTypeSlug": "vc_research",
  "sourceTypeLabel": "VC research",
  "presentationDate": null,
  "orientation": "landscape",
  "aspectRatio": 1.777,
  "pageNumber": 280,
  "pageCount": 313,
  "prevPage": 279,
  "nextPage": 281,
  "slideType": "case_study",
  "function": "illustrate_case",
  "density": "dense",
  "nDataPoints": 8,
  "notes": "The slide highlights a collaborative safety testing initiative between two major AI labs, including a bar chart showing 'Tutor jailbreak resistance' across various models.",
  "elementsJson": [
    "headline_text",
    "action_title",
    "paragraph",
    "bullet_list",
    "bar_chart_vertical"
  ],
  "metadataConfidence": 1,
  "imagePath": null,
  "slideHref": "/slides/019dd923-5e88-73ef-bd5d-0d0f98caffe1/280",
  "deckHref": "/decks/019dd923-5e88-73ef-bd5d-0d0f98caffe1",
  "deckJsonHref": "/decks/019dd923-5e88-73ef-bd5d-0d0f98caffe1.json",
  "deckAnchorHref": "/decks/019dd923-5e88-73ef-bd5d-0d0f98caffe1#slide-280",
  "components": [
    {
      "bbox": null,
      "kind": "callout",
      "text": "Surprisingly, reasoning didn’t always make models safer, and sometimes smaller models outperformed their larger peers.",
      "attrs": null,
      "subkind": null,
      "toolName": "Visual emphasis",
      "toolSlug": "visual-emphasis",
      "confidence": null,
      "componentId": "019dd952-47c3-7407-8d0a-94ce3a17ea0a",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.45,
        "w": 0.35,
        "x": 0.63,
        "y": 0.365
      },
      "kind": "chart",
      "text": "Tutor jailbreak resistance",
      "attrs": null,
      "subkind": "bar-vertical",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "e2a15ff3-eec8-4ea5-991e-847dabdf13a1",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.55,
        "w": 0.6,
        "x": 0.02,
        "y": 0.36
      },
      "kind": "list",
      "text": "Anthropic reports o3 looked as well or better aligned than Claude on most axes, while GPT-4o/4.1 and o4-mini were more willing to assist misuse. Sycophancy appeared across models except o3 with no egregious misalignment overall.\nOpenAI finds Claude strongest on instruction-hierarchy and prompt-extraction, while o3/o4-mini held up better on jailbreaking. Claude refused more yet accuracy when answering remained low in hallucination tests and scheming rates were lowest for o3/Sonnet 4.\nSurprisingly, reasoning didn’t always make models safer, and sometimes smaller models outperformed their larger peers.\nAnthropic seemed to conclude that this wasn’t an effective use of their time, saying that this will be only a small part of their eval portfolio given the substantial logistical investment required.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "bd9d8727-8057-44f9-9cbc-44010000a504",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": null,
      "kind": "metric",
      "text": "Successful resistance",
      "attrs": null,
      "subkind": "primary",
      "toolName": "Quantification",
      "toolSlug": "quantification",
      "confidence": null,
      "componentId": "019dd952-47c3-7407-8d0a-99fd8d9a94cf",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.12,
        "w": 0.6,
        "x": 0.02,
        "y": 0.22
      },
      "kind": "paragraph",
      "text": "The goal of this work was to explore model propensities, the kinds of concerning behaviors models might attempt, and not to conduct full threat modeling or estimating real-world likelihoods of bad behaviors. The tests were run prior to the launch of GPT-5.",
      "attrs": null,
      "subkind": "paragraph",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "e811d0ac-312e-4679-91a5-f46de7ba2388",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.05,
        "w": 0.6,
        "x": 0.02,
        "y": 0.14
      },
      "kind": "title",
      "text": "OpenAI and Anthropic test each other’s models on safety evals for the first time",
      "attrs": null,
      "subkind": "headline",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "ede37dbc-2718-4b68-9e36-b6c12109ca59",
      "frameworkName": null,
      "frameworkSlug": null
    }
  ],
  "metrics": [],
  "tools": [],
  "frameworks": [],
  "arcBeats": [
    {
      "to": 282,
      "from": 190,
      "beatId": "019dd95a-0682-776c-8e35-a256fe16ea9c",
      "arcName": "The Triple Take",
      "arcSlug": "triple-take",
      "beatName": "The Implications (So What)",
      "beatSlug": "triple-take-the-implications-so-what",
      "evidence": "Sections 3-4 frame political and safety consequences of the technical/industry trends.",
      "position": 2,
      "confidence": 70,
      "parentBeatName": "Reflection",
      "parentBeatSlug": "reflection"
    },
    {
      "to": 282,
      "from": 190,
      "beatId": "019dd95a-0682-776c-8e35-b713af60bf88",
      "arcName": "The Onion",
      "arcSlug": "onion",
      "beatName": "Core Insight",
      "beatSlug": "onion-core-insight",
      "evidence": "Politics and Safety reveal the structural risks at the core.",
      "position": 4,
      "confidence": 45,
      "parentBeatName": "Turn",
      "parentBeatSlug": "turn"
    }
  ],
  "loops": [
    {
      "to": 282,
      "from": 247,
      "name": "Pre Mortem",
      "slug": "19-pre-mortem",
      "bestFor": "Project kick-offs, risk management, building trust with skeptical stakeholders",
      "matchId": "019dd95a-07fe-70ce-8d3e-bb4d43115ab4",
      "evidence": "Vibe hacking, AI psychosis, alignment faking, scheming, prompt injection — what could go wrong + mitigations.",
      "position": 12,
      "objective": "Surface AI safety failure modes before they become catastrophic",
      "structure": "The Future Disaster (Hypothetical) -> What Went Wrong -> The Prevention Plan",
      "confidence": 75,
      "description": "Build trust by anticipating failure modes before they happen and solving them in advance"
    }
  ],
  "imagePathAlt": null,
  "thumbSrc": null,
  "thumbSrcAlt": null,
  "locked": true
}