{
  "docId": "019dd923-5e88-73ef-bd5d-06b04d219fea",
  "docSlug": "dd91c78f6570bf29",
  "documentTitle": "2023 Air Street Capital The State of AI Report 2023",
  "authorId": "AirStreetCapital",
  "authorName": "Air Street Capital",
  "documentKindSlug": "consulting-deck",
  "documentKindLabel": "Consulting deck",
  "sourceTypeSlug": "vc_research",
  "sourceTypeLabel": "VC research",
  "presentationDate": null,
  "orientation": "landscape",
  "aspectRatio": 1.777,
  "pageNumber": 151,
  "pageCount": 163,
  "prevPage": 150,
  "nextPage": 152,
  "slideType": "industry_trends",
  "function": "present_framework",
  "density": "dense",
  "nDataPoints": 6,
  "notes": "Includes a process diagram for the 'Self-Align' technique.",
  "elementsJson": [
    "headline_text",
    "bullet_list",
    "process_diagram",
    "paragraph"
  ],
  "metadataConfidence": 1,
  "imagePath": null,
  "slideHref": "/slides/019dd923-5e88-73ef-bd5d-06b04d219fea/151",
  "deckHref": "/decks/019dd923-5e88-73ef-bd5d-06b04d219fea",
  "deckJsonHref": "/decks/019dd923-5e88-73ef-bd5d-06b04d219fea.json",
  "deckAnchorHref": "/decks/019dd923-5e88-73ef-bd5d-06b04d219fea#slide-151",
  "components": [
    {
      "bbox": null,
      "kind": "callout",
      "text": "Models can become more capable (both helpful and safe) with minimal human supervision by bootstrapping.",
      "attrs": null,
      "subkind": null,
      "toolName": "Visual emphasis",
      "toolSlug": "visual-emphasis",
      "confidence": null,
      "componentId": "019dd952-47c4-7719-a22f-e5155c598366",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.65,
        "w": 0.25,
        "x": 0.72,
        "y": 0.28
      },
      "kind": "diagram",
      "text": "Self-Align process flow showing steps from Topic-Guided Red-Teaming to Verbose Cloning.",
      "attrs": null,
      "subkind": "process",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "a9c4c86d-1dae-4e75-bc1d-af12268753fb",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.1,
        "w": 0.65,
        "x": 0.02,
        "y": 0.83
      },
      "kind": "list",
      "text": "One way in which these techniques are potentially better than RLHF is it explicitly directs the model into satisfying some constraints as opposed to possible reward hacking.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "1faff060-89fa-42db-94fa-e62e352a68a2",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.25,
        "w": 0.65,
        "x": 0.02,
        "y": 0.58
      },
      "kind": "list",
      "text": "Self-Align is a similar technique that uses a small set of guiding principles. It gets the model to generate synthetic prompts and guides it using in-context learning from the set of principles to explain why some of them are harmful. The newer aligned responses are used for finetuning the original model. The resulting model generates desirable responses according to the desired principles but without using them directly.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "301e965f-1885-4b0c-8c52-548dc0b97688",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.3,
        "w": 0.65,
        "x": 0.02,
        "y": 0.28
      },
      "kind": "list",
      "text": "Constitutional AI is based on the idea that supervision will come from a set of principles that govern AI behaviour and very few feedback labels. First, the model itself generates self-critiques and revisions in line with the set of principles which it uses for finetuning. Second, the model generates samples for a preference model to choose from. Using the preference model for re-training the original model is referred to as RL from AI Feedback (RLAIF).",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "64f6206e-3a50-4fd8-b644-463225ccc3cd",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.05,
        "w": 0.4,
        "x": 0.02,
        "y": 0.14
      },
      "kind": "title",
      "text": "Constitutional AI and Self-Alignment",
      "attrs": null,
      "subkind": "headline",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "29151824-ae4e-4ed5-8113-fcaa472102d8",
      "frameworkName": null,
      "frameworkSlug": null
    }
  ],
  "metrics": [],
  "tools": [
    {
      "name": "Process flow",
      "slug": "process-flow",
      "agent": null,
      "layer": "slide",
      "matchId": "66613c38-b090-468b-a42b-5bb6e439c793",
      "evidence": "diagram/process: Self-Align process flow showing steps from Topic-Guided Red-Teaming to Verbose Cloning.",
      "confidence": 0.7
    }
  ],
  "frameworks": [
    {
      "name": "Constitutional AI",
      "slug": null,
      "matchId": "019dd95a-1ca5-70bb-bba0-a936b3951940",
      "evidence": "Slide title is 'Constitutional AI and Self-Alignment'.",
      "confidence": 85
    },
    {
      "name": "process",
      "slug": null,
      "matchId": "a137a0c7-403e-4456-94e2-6ff60490f136",
      "evidence": "The Self-Align diagram illustrates a sequential workflow.",
      "confidence": 0.8
    }
  ],
  "arcBeats": [
    {
      "to": 155,
      "from": 121,
      "beatId": "019dd95a-0682-776c-8e35-478c8c58938c",
      "arcName": "The Triple Take",
      "arcSlug": "triple-take",
      "beatName": "The Implications (So What)",
      "beatSlug": "triple-take-the-implications-so-what",
      "evidence": "Politics + Safety sections explore regulatory, geopolitical and risk implications.",
      "position": 2,
      "confidence": 78,
      "parentBeatName": "Reflection",
      "parentBeatSlug": "reflection"
    },
    {
      "to": 155,
      "from": 121,
      "beatId": "019dd95a-0682-776c-8e35-554ec1797b3a",
      "arcName": "The Mountain",
      "arcSlug": "mountain",
      "beatName": "Climax",
      "beatSlug": "mountain-climax",
      "evidence": "Regulatory divergence and the x-risk debate erupt into the mainstream.",
      "position": 3,
      "confidence": 45,
      "parentBeatName": "Turn",
      "parentBeatSlug": "turn"
    }
  ],
  "loops": [
    {
      "to": 155,
      "from": 149,
      "name": "Pattern Hunter",
      "slug": "02-pattern-hunter",
      "bestFor": "Time-pressed audiences, building consensus, when data is strong",
      "matchId": "019dd95a-07fe-70ce-8d3e-1a62463741ad",
      "evidence": "RLHF survey, conditional pretraining, Constitutional AI, scalable supervision, process eval, interpretability, benchmark drift.",
      "position": 15,
      "objective": "Show alignment research progress on RLHF problems",
      "structure": "Evidence A -> Evidence B -> Evidence C -> Pattern/Conclusion",
      "confidence": 70,
      "description": "Group multiple pieces of evidence that together point to a pattern or conclusion"
    }
  ],
  "imagePathAlt": null,
  "thumbSrc": null,
  "thumbSrcAlt": null,
  "locked": true
}