{
  "docId": "019dd923-5e88-73ef-bd5d-06b04d219fea",
  "docSlug": "dd91c78f6570bf29",
  "documentTitle": "2023 Air Street Capital The State of AI Report 2023",
  "authorId": "AirStreetCapital",
  "authorName": "Air Street Capital",
  "documentKindSlug": "consulting-deck",
  "documentKindLabel": "Consulting deck",
  "sourceTypeSlug": "vc_research",
  "sourceTypeLabel": "VC research",
  "presentationDate": null,
  "orientation": "landscape",
  "aspectRatio": 1.777,
  "pageNumber": 17,
  "pageCount": 163,
  "prevPage": 16,
  "nextPage": 18,
  "slideType": "case_study",
  "function": "illustrate_case",
  "density": "dense",
  "nDataPoints": 24,
  "notes": "The slide discusses the release of LLaMa, its performance metrics (RACE benchmark), and the controversy regarding its 'open-source' status.",
  "elementsJson": [
    "paragraph",
    "bullet_list",
    "data_table"
  ],
  "metadataConfidence": 1,
  "imagePath": null,
  "slideHref": "/slides/019dd923-5e88-73ef-bd5d-06b04d219fea/17",
  "deckHref": "/decks/019dd923-5e88-73ef-bd5d-06b04d219fea",
  "deckJsonHref": "/decks/019dd923-5e88-73ef-bd5d-06b04d219fea.json",
  "deckAnchorHref": "/decks/019dd923-5e88-73ef-bd5d-06b04d219fea#slide-17",
  "components": [
    {
      "bbox": null,
      "kind": "callout",
      "text": "The LLaMa-1 models outperform GPT-3 (the original one, not the InstructGPT variants) and are competitive with DeepMind's Chinchilla and Google's PaLM.",
      "attrs": null,
      "subkind": null,
      "toolName": "Visual emphasis",
      "toolSlug": "visual-emphasis",
      "confidence": null,
      "componentId": "019dd952-47c4-7719-a221-b8e3eed9ba62",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.45,
        "w": 0.68,
        "x": 0.02,
        "y": 0.38
      },
      "kind": "list",
      "text": "The LLaMa-1 models use regular transformers, with slight changes to the architecture. The authors also made a few changes to the optimizer and to the implementation of attention. As a result, “when training a 65B-parameter model, [their] code processes around 380 tokens/sec/GPU on 2048 A100 GPU with 80GB of RAM. This means that training over [their] dataset containing 1.4T tokens takes approximately 21 days.”\nThe LLaMa-1 models outperform GPT-3 (the original one, not the InstructGPT variants) and are competitive with DeepMind’s Chinchilla and Google’s PaLM.\nLLaMa-1 didn’t allow commercial use, prompting heavy criticism around the term “open-source” that Meta used to describe the model release. But a second LLaMa iteration appeased most of the open source community.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "2f280226-b4c8-4c42-bbbd-b3dbf71e53c5",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": null,
      "kind": "metric",
      "text": "Zero-shot accuracy: 51.6",
      "attrs": null,
      "subkind": "primary",
      "toolName": "Quantification",
      "toolSlug": "quantification",
      "confidence": null,
      "componentId": "019dd952-47c4-7719-a221-bee60167dd97",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.15,
        "w": 0.68,
        "x": 0.02,
        "y": 0.21
      },
      "kind": "paragraph",
      "text": "In February '23, Meta released a series of models called LLaMa. At their release, they stood out as being the most capable models trained exclusively on publicly available datasets. Meta initially granted access to the LLaMa model weights on demand only to researchers, but the weights were quickly leaked and published online.",
      "attrs": null,
      "subkind": "paragraph",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "2795c48b-b931-4dbf-afd4-73e5b6af58fb",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.35,
        "w": 0.26,
        "x": 0.72,
        "y": 0.45
      },
      "kind": "table",
      "text": "Table 6: Reading Comprehension. Zero-shot accuracy.",
      "attrs": null,
      "subkind": "data",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "cb52647c-a942-49da-aff9-586236d97e96",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.04,
        "w": 0.35,
        "x": 0.02,
        "y": 0.14
      },
      "kind": "title",
      "text": "...unless LLaMas reverse the trend",
      "attrs": null,
      "subkind": "headline",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "b20e1dcb-b16c-4534-b28e-2dca628db0d4",
      "frameworkName": null,
      "frameworkSlug": null
    }
  ],
  "metrics": [],
  "tools": [
    {
      "name": "Anadiplosis",
      "slug": "anadiplosis",
      "agent": "Storyteller",
      "layer": "loop",
      "matchId": "019dd95a-1055-74e0-a957-1afc60409f92",
      "evidence": "Title chains directly to prior 'nail in the coffin' slide.",
      "confidence": 60
    },
    {
      "name": "Action Titles",
      "slug": "action-titles",
      "agent": "Architect",
      "layer": "slide",
      "matchId": "019dd95a-1055-74e0-a957-151157984b9c",
      "evidence": "'...unless LLaMas reverse the trend' picks up prior slide via narrative continuation.",
      "confidence": 85
    },
    {
      "name": "Table data",
      "slug": "table-data",
      "agent": null,
      "layer": "slide",
      "matchId": "9bba61b9-ab85-4a99-90c0-20fe089fac30",
      "evidence": "The slide contains a table with data on reading comprehension, specifically 'Table 6: Reading Comprehension. Zero-shot accuracy.'",
      "confidence": 0.9
    }
  ],
  "frameworks": [],
  "arcBeats": [
    {
      "to": 120,
      "from": 11,
      "beatId": "019dd95a-0682-776c-8e35-41afd44ef59f",
      "arcName": "The Triple Take",
      "arcSlug": "triple-take",
      "beatName": "The Facts (What)",
      "beatSlug": "triple-take-the-facts-what",
      "evidence": "Research + Industry sections inventory model, compute, funding facts.",
      "position": 1,
      "confidence": 78,
      "parentBeatName": "Setup",
      "parentBeatSlug": "setup"
    },
    {
      "to": 120,
      "from": 11,
      "beatId": "019dd95a-0682-776c-8e35-523bfb7f96e6",
      "arcName": "The Mountain",
      "arcSlug": "mountain",
      "beatName": "Rising Action",
      "beatSlug": "mountain-rising-action",
      "evidence": "Escalating capabilities, compute concentration and capital flows.",
      "position": 2,
      "confidence": 45,
      "parentBeatName": "Development",
      "parentBeatSlug": "development"
    }
  ],
  "loops": [
    {
      "to": 21,
      "from": 12,
      "name": "Pattern Hunter",
      "slug": "02-pattern-hunter",
      "bestFor": "Time-pressed audiences, building consensus, when data is strong",
      "matchId": "019dd95a-07fe-70ce-8d3d-e61d00ffc6e4",
      "evidence": "Sequential GPT-4, RLHF, LLaMa, popularity-mention slides converge on a single conclusion.",
      "position": 2,
      "objective": "Pile up evidence that GPT-4 and RLHF dominate the LLM landscape",
      "structure": "Evidence A -> Evidence B -> Evidence C -> Pattern/Conclusion",
      "confidence": 82,
      "description": "Group multiple pieces of evidence that together point to a pattern or conclusion"
    }
  ],
  "imagePathAlt": null,
  "thumbSrc": null,
  "thumbSrcAlt": null,
  "locked": true
}