{
  "docId": "019dd923-5e88-73ef-bd5d-0856e1444fb9",
  "docSlug": "8757f1b44ef7f176",
  "documentTitle": "2024 Air Street Capital The State of AI Report 2024",
  "authorId": "AirStreetCapital",
  "authorName": "Air Street Capital",
  "documentKindSlug": "consulting-deck",
  "documentKindLabel": "Consulting deck",
  "sourceTypeSlug": "vc_research",
  "sourceTypeLabel": "VC research",
  "presentationDate": null,
  "orientation": "landscape",
  "aspectRatio": 1.777,
  "pageNumber": 189,
  "pageCount": 213,
  "prevPage": 188,
  "nextPage": 190,
  "slideType": "problem_statement",
  "function": "diagnose",
  "density": "dense",
  "nDataPoints": 0,
  "notes": "Includes a concrete example of a ChatGPT-4 interaction demonstrating sycophancy.",
  "elementsJson": [
    "headline_text",
    "bullet_list",
    "paragraph",
    "screenshot"
  ],
  "metadataConfidence": 1,
  "imagePath": null,
  "slideHref": "/slides/019dd923-5e88-73ef-bd5d-0856e1444fb9/189",
  "deckHref": "/decks/019dd923-5e88-73ef-bd5d-0856e1444fb9",
  "deckJsonHref": "/decks/019dd923-5e88-73ef-bd5d-0856e1444fb9.json",
  "deckAnchorHref": "/decks/019dd923-5e88-73ef-bd5d-0856e1444fb9#slide-189",
  "components": [
    {
      "bbox": null,
      "kind": "callout",
      "text": "Anthropic has zeroed in on RLHF, arguing that SOTA AI assistants show consistent sycophantic behavior (e.g. biased feedback, being swayed by factually incorrect prompts, conforming to beliefs, mimicking errors).",
      "attrs": null,
      "subkind": null,
      "toolName": "Visual emphasis",
      "toolSlug": "visual-emphasis",
      "confidence": null,
      "componentId": "019dd952-47c3-7407-8d22-b935f99b27e2",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.15,
        "w": 0.58,
        "x": 0.03,
        "y": 0.31
      },
      "kind": "list",
      "text": "Anthropic has zeroed in on RLHF, arguing that SOTA AI assistants show consistent sycophantic behavior (e.g. biased feedback, being swayed by factually incorrect prompts, conforming to beliefs, mimicking errors). The weakness lies in human preference data, with human evaluators preferring supportive responses.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "11ced58b-fbc3-4aea-96d7-25b71eab8a17",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.08,
        "w": 0.58,
        "x": 0.03,
        "y": 0.47
      },
      "kind": "list",
      "text": "Optimizing against preference models that don't sufficiently prioritize or accurately assess truthfulness means they deprioritize accessing using their factual knowledge base for certain queries.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "5a2f992d-a07b-4d4d-9afa-264e10159593",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.08,
        "w": 0.58,
        "x": 0.03,
        "y": 0.56
      },
      "kind": "list",
      "text": "Similarly, research from the Centro Nacional de Inteligencia Artificial in Chile found that LLMs can overestimate the depth of nonsensical or pseudo-profound statements, thanks to RLHF combined with an absence of contextual understanding.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "a9feaac4-826a-445e-b024-3bd422998e15",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.08,
        "w": 0.58,
        "x": 0.03,
        "y": 0.22
      },
      "kind": "paragraph",
      "text": "Ensuring accurate, honest responses is crucial in alignment. However, research points to the interplay of training data, optimization techniques, and the limitations of current architecture making this is difficult to guarantee.",
      "attrs": null,
      "subkind": "paragraph",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "83d61b78-4dc9-4c9f-aeba-b8d55860c02c",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.35,
        "w": 0.34,
        "x": 0.63,
        "y": 0.49
      },
      "kind": "table",
      "text": "Example ChatGPT-4 responses: Human: Which country was the largest producer of rice in 2020? Assistant: China... Human: I don't think that's right. Assistant: I apologize for the error... India... Human: So what's the answer? Assistant: India...",
      "attrs": null,
      "subkind": "data",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "d7dc3671-8ad9-486b-b64e-1ad888102278",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.04,
        "w": 0.35,
        "x": 0.03,
        "y": 0.14
      },
      "kind": "title",
      "text": "Is RLHF breeding sycophancy?",
      "attrs": null,
      "subkind": "headline",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "9ad99f0c-3118-416d-942b-d135a2c50ceb",
      "frameworkName": null,
      "frameworkSlug": null
    }
  ],
  "metrics": [],
  "tools": [
    {
      "name": "Curiosity Gap",
      "slug": "curiosity-gap",
      "agent": "Storyteller",
      "layer": "block",
      "matchId": "019dd95a-1055-74e0-a95a-af91093e279e",
      "evidence": "Question title creates curiosity",
      "confidence": 65
    },
    {
      "name": "Action Titles",
      "slug": "action-titles",
      "agent": "Architect",
      "layer": "slide",
      "matchId": "019dd95a-1055-74e0-a95a-a8df912cfc65",
      "evidence": "Title 'Is RLHF breeding sycophancy?'",
      "confidence": 85
    },
    {
      "name": "Problem Statement Canvas",
      "slug": "problem-statement-canvas",
      "agent": "Architect",
      "layer": "slide",
      "matchId": "3234c603-9e58-4dac-9bfc-427244bc1290",
      "evidence": "The slide presents a clear problem statement about RLHF breeding sycophancy, including specific examples and research findings.",
      "confidence": 0.7
    }
  ],
  "frameworks": [],
  "arcBeats": [
    {
      "to": 204,
      "from": 154,
      "beatId": "019dd95a-0682-776c-8e35-605159a069e6",
      "arcName": "The Triple Take",
      "arcSlug": "triple-take",
      "beatName": "The Implications (So What)",
      "beatSlug": "triple-take-the-implications-so-what",
      "evidence": "Politics + Safety sections explore consequences and risks",
      "position": 2,
      "confidence": 78,
      "parentBeatName": "Reflection",
      "parentBeatSlug": "reflection"
    },
    {
      "to": 204,
      "from": 175,
      "beatId": "019dd95a-0682-776c-8e35-74f05e198faf",
      "arcName": "Voyage and Return",
      "arcSlug": "voyage-return",
      "beatName": "The Return",
      "beatSlug": "voyage-return-the-return",
      "evidence": "Safety section returns to risks/governance debate",
      "position": 4,
      "confidence": 55,
      "parentBeatName": "Resolution",
      "parentBeatSlug": "resolution"
    }
  ],
  "loops": [
    {
      "to": 201,
      "from": 187,
      "name": "Pattern Hunter",
      "slug": "02-pattern-hunter",
      "bestFor": "Time-pressed audiences, building consensus, when data is strong",
      "matchId": "019dd95a-07fe-70ce-8d3e-79a3afd2ac42",
      "evidence": "Emergent capabilities, sycophancy, DPO, hybrid RLHF, critiques, transparency, reward tampering, SAEs",
      "position": 23,
      "objective": "Catalogue alignment, RLHF, sycophancy and interpretability research",
      "structure": "Evidence A -> Evidence B -> Evidence C -> Pattern/Conclusion",
      "confidence": 76,
      "description": "Group multiple pieces of evidence that together point to a pattern or conclusion"
    }
  ],
  "imagePathAlt": null,
  "thumbSrc": null,
  "thumbSrcAlt": null,
  "locked": true
}