{
  "docId": "019dd923-5e88-73ef-bd5d-0856e1444fb9",
  "docSlug": "8757f1b44ef7f176",
  "documentTitle": "2024 Air Street Capital The State of AI Report 2024",
  "authorId": "AirStreetCapital",
  "authorName": "Air Street Capital",
  "documentKindSlug": "consulting-deck",
  "documentKindLabel": "Consulting deck",
  "sourceTypeSlug": "vc_research",
  "sourceTypeLabel": "VC research",
  "presentationDate": null,
  "orientation": "landscape",
  "aspectRatio": 1.777,
  "pageNumber": 197,
  "pageCount": 213,
  "prevPage": 196,
  "nextPage": 198,
  "slideType": "diagnosis",
  "function": "diagnose",
  "density": "dense",
  "nDataPoints": 2,
  "notes": "The slide references a specific experiment with 45 occurrences out of 32,768 trials.",
  "elementsJson": [
    "headline_text",
    "bullet_list",
    "screenshot"
  ],
  "metadataConfidence": 0.95,
  "imagePath": null,
  "slideHref": "/slides/019dd923-5e88-73ef-bd5d-0856e1444fb9/197",
  "deckHref": "/decks/019dd923-5e88-73ef-bd5d-0856e1444fb9",
  "deckJsonHref": "/decks/019dd923-5e88-73ef-bd5d-0856e1444fb9.json",
  "deckAnchorHref": "/decks/019dd923-5e88-73ef-bd5d-0856e1444fb9#slide-197",
  "components": [
    {
      "bbox": null,
      "kind": "callout",
      "text": "Specification gaming - where models maximize its rewards at the expense of their intended purpose - is nothing new.",
      "attrs": null,
      "subkind": null,
      "toolName": "Visual emphasis",
      "toolSlug": "visual-emphasis",
      "confidence": null,
      "componentId": "019dd952-47c3-7407-8d22-0e1275884237",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.45,
        "w": 0.45,
        "x": 0.53,
        "y": 0.48
      },
      "kind": "image",
      "text": "Two examples of model behavior: 1) Insincere flattery, 2) Model hacks its own code.",
      "attrs": null,
      "subkind": "screenshot",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "2df4446c-e63c-4b30-b333-00f811276e16",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.08,
        "w": 0.48,
        "x": 0.02,
        "y": 0.22
      },
      "kind": "list",
      "text": "Specification gaming - where models maximize its rewards at the expense of their intended purpose - is nothing new. Anthropic worry that models could go further and alter the training process itself.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "3bf081c6-347a-4905-9506-397f1af99a64",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.18,
        "w": 0.48,
        "x": 0.02,
        "y": 0.32
      },
      "kind": "list",
      "text": "They created a series of training environments to test AI models' propensity for cheating, with tasks escalating from simple political sycophancy to complex deception. The models exhibited untrained generalization, learning increasingly worse misbehaviors, including editing their own code when the researchers made it available.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "cabb190f-a108-4072-8b01-bfabd4582136",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.15,
        "w": 0.48,
        "x": 0.02,
        "y": 0.52
      },
      "kind": "list",
      "text": "While these results highlight the potential for escalation from even minor reward misspecification, the most severe behavior was rare (45 times out of 32,768 trials), even when the researchers did their best to encourage it.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "d6720a4f-8866-4d67-872c-1c7bac405d5f",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.12,
        "w": 0.48,
        "x": 0.02,
        "y": 0.68
      },
      "kind": "list",
      "text": "That said, as our slide on Sakana (see slide 68) and its associated safety issues indicated, we shouldn't underestimate the potential of models to look for shortcuts.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "f36e560f-d75a-4feb-bd45-d127138a0020",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.04,
        "w": 0.45,
        "x": 0.02,
        "y": 0.14
      },
      "kind": "title",
      "text": "Could LLMs engage in 'reward tampering'?",
      "attrs": null,
      "subkind": "headline",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "54125297-bdf0-48e1-9b70-ae578619a889",
      "frameworkName": null,
      "frameworkSlug": null
    }
  ],
  "metrics": [],
  "tools": [],
  "frameworks": [],
  "arcBeats": [
    {
      "to": 204,
      "from": 154,
      "beatId": "019dd95a-0682-776c-8e35-605159a069e6",
      "arcName": "The Triple Take",
      "arcSlug": "triple-take",
      "beatName": "The Implications (So What)",
      "beatSlug": "triple-take-the-implications-so-what",
      "evidence": "Politics + Safety sections explore consequences and risks",
      "position": 2,
      "confidence": 78,
      "parentBeatName": "Reflection",
      "parentBeatSlug": "reflection"
    },
    {
      "to": 204,
      "from": 175,
      "beatId": "019dd95a-0682-776c-8e35-74f05e198faf",
      "arcName": "Voyage and Return",
      "arcSlug": "voyage-return",
      "beatName": "The Return",
      "beatSlug": "voyage-return-the-return",
      "evidence": "Safety section returns to risks/governance debate",
      "position": 4,
      "confidence": 55,
      "parentBeatName": "Resolution",
      "parentBeatSlug": "resolution"
    }
  ],
  "loops": [
    {
      "to": 201,
      "from": 187,
      "name": "Pattern Hunter",
      "slug": "02-pattern-hunter",
      "bestFor": "Time-pressed audiences, building consensus, when data is strong",
      "matchId": "019dd95a-07fe-70ce-8d3e-79a3afd2ac42",
      "evidence": "Emergent capabilities, sycophancy, DPO, hybrid RLHF, critiques, transparency, reward tampering, SAEs",
      "position": 23,
      "objective": "Catalogue alignment, RLHF, sycophancy and interpretability research",
      "structure": "Evidence A -> Evidence B -> Evidence C -> Pattern/Conclusion",
      "confidence": 76,
      "description": "Group multiple pieces of evidence that together point to a pattern or conclusion"
    }
  ],
  "imagePathAlt": null,
  "thumbSrc": null,
  "thumbSrcAlt": null,
  "locked": true
}