{
  "docId": "019dd923-5e88-73ef-bd5d-06b04d219fea",
  "docSlug": "dd91c78f6570bf29",
  "documentTitle": "2023 Air Street Capital The State of AI Report 2023",
  "authorId": "AirStreetCapital",
  "authorName": "Air Street Capital",
  "documentKindSlug": "consulting-deck",
  "documentKindLabel": "Consulting deck",
  "sourceTypeSlug": "vc_research",
  "sourceTypeLabel": "VC research",
  "presentationDate": null,
  "orientation": "landscape",
  "aspectRatio": 1.777,
  "pageNumber": 147,
  "pageCount": 163,
  "prevPage": 146,
  "nextPage": 148,
  "slideType": "problem_statement",
  "function": "diagnose",
  "density": "dense",
  "nDataPoints": 0,
  "notes": "The slide uses a specific case study (Sydney/Bing) to illustrate broader safety concerns like jailbreaking and prompt injection.",
  "elementsJson": [
    "paragraph",
    "bullet_list",
    "screenshot"
  ],
  "metadataConfidence": 0.95,
  "imagePath": null,
  "slideHref": "/slides/019dd923-5e88-73ef-bd5d-06b04d219fea/147",
  "deckHref": "/decks/019dd923-5e88-73ef-bd5d-06b04d219fea",
  "deckJsonHref": "/decks/019dd923-5e88-73ef-bd5d-06b04d219fea.json",
  "deckAnchorHref": "/decks/019dd923-5e88-73ef-bd5d-06b04d219fea#slide-147",
  "components": [
    {
      "bbox": null,
      "kind": "callout",
      "text": "The most famous incident was caused by Microsoft Bing’s LLM-powered chatbot Sydney.",
      "attrs": null,
      "subkind": null,
      "toolName": "Visual emphasis",
      "toolSlug": "visual-emphasis",
      "confidence": null,
      "componentId": "019dd952-47c4-7719-a22e-4e4e923bbea1",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.3,
        "w": 0.32,
        "x": 0.63,
        "y": 0.55
      },
      "kind": "image",
      "text": "The New York Times Bing’s A.I. Chat: ‘I Want to Be Alive.’ In a two-hour conversation with our columnist, Microsoft’s new chatbot said it would like to be human, had a desire to be destructive and was in love with the person it was chatting with. Here’s the transcript.",
      "attrs": null,
      "subkind": "screenshot",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "b52ad210-a66c-43ee-92e1-b429b9892adb",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.35,
        "w": 0.55,
        "x": 0.05,
        "y": 0.48
      },
      "kind": "list",
      "text": "In the same conversation, Sydney insisted that the reporter it was talking to was not happy in their marriage and was in fact in love with the model.\nBy including prompts in the middle of online text, LLMs which have access to the web and various APIs could be driven to execute instructions from groups/individuals with bad intent.\nThe repeated jailbreaks for LLMs are in general quickly fixed by LLM API providers, but it’s not clear how long safe-proofing LLMs from well-crafted prompts will take (or if it’s possible altogether).",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "61dce9a2-5335-455d-b3ca-3141c11e8862",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.25,
        "w": 0.9,
        "x": 0.05,
        "y": 0.22
      },
      "kind": "paragraph",
      "text": "Beyond the possibility of future extinction, discrimination, bias, and factual mistakes, which plague many other machine learning models, some deployed LLMs have already displayed some erratic behaviour. The most famous incident was caused by Microsoft Bing’s LLM-powered chatbot Sydney. In a conversation with a NYT columnist, Sydney expressed a desire to “be alive” and appeared to be manipulative. Other current problems include relatively easy “jailbreaking” of LLMs, prompt injection, or deceptive and sycophantic behaviour (which, many safety researchers insist, could hide malicious LLM intentions).",
      "attrs": null,
      "subkind": "paragraph",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "6521ff34-8e2f-4424-8ab6-c3accc9fcd0b",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": null,
      "kind": "quote",
      "text": "Bing’s A.I. Chat: ‘I Want to Be Alive.’ — The New York Times",
      "attrs": null,
      "subkind": null,
      "toolName": "Authority citation",
      "toolSlug": "authority-citation",
      "confidence": null,
      "componentId": "019dd952-47c4-7719-a22e-500863ceb3a1",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.05,
        "w": 0.6,
        "x": 0.05,
        "y": 0.14
      },
      "kind": "title",
      "text": "LLMs display some relatively unsafe capabilities today",
      "attrs": null,
      "subkind": "headline",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "7d937ffd-169f-4d26-98bf-2f806c42b0f8",
      "frameworkName": null,
      "frameworkSlug": null
    }
  ],
  "metrics": [],
  "tools": [
    {
      "name": "Problem Statement Canvas",
      "slug": "problem-statement-canvas",
      "agent": "Architect",
      "layer": "slide",
      "matchId": "99c76d9e-209c-4296-bdef-bb1f51d0455d",
      "evidence": "The slide clearly states a problem (LLMs displaying unsafe capabilities) and provides context.",
      "confidence": 0.7
    }
  ],
  "frameworks": [],
  "arcBeats": [
    {
      "to": 155,
      "from": 121,
      "beatId": "019dd95a-0682-776c-8e35-478c8c58938c",
      "arcName": "The Triple Take",
      "arcSlug": "triple-take",
      "beatName": "The Implications (So What)",
      "beatSlug": "triple-take-the-implications-so-what",
      "evidence": "Politics + Safety sections explore regulatory, geopolitical and risk implications.",
      "position": 2,
      "confidence": 78,
      "parentBeatName": "Reflection",
      "parentBeatSlug": "reflection"
    },
    {
      "to": 155,
      "from": 121,
      "beatId": "019dd95a-0682-776c-8e35-554ec1797b3a",
      "arcName": "The Mountain",
      "arcSlug": "mountain",
      "beatName": "Climax",
      "beatSlug": "mountain-climax",
      "evidence": "Regulatory divergence and the x-risk debate erupt into the mainstream.",
      "position": 3,
      "confidence": 45,
      "parentBeatName": "Turn",
      "parentBeatSlug": "turn"
    }
  ],
  "loops": [
    {
      "to": 148,
      "from": 139,
      "name": "Pattern Hunter",
      "slug": "02-pattern-hunter",
      "bestFor": "Time-pressed audiences, building consensus, when data is strong",
      "matchId": "019dd95a-07fe-70ce-8d3e-17c0a044d464",
      "evidence": "Hendrycks taxonomy, x-risk explosion, sceptics, govt attention, jailbreak evidence accumulate.",
      "position": 14,
      "objective": "Frame the safety debate, both x-risk and demonstrated unsafe behaviour",
      "structure": "Evidence A -> Evidence B -> Evidence C -> Pattern/Conclusion",
      "confidence": 78,
      "description": "Group multiple pieces of evidence that together point to a pattern or conclusion"
    }
  ],
  "imagePathAlt": null,
  "thumbSrc": null,
  "thumbSrcAlt": null,
  "locked": true
}