{
  "docId": "019dd923-5e88-73ef-bd5c-fd5012384ae3",
  "docSlug": "bf350dd574c19997",
  "documentTitle": "2021 Air Street Capital The State of AI Report 2021",
  "authorId": "AirStreetCapital",
  "authorName": "Air Street Capital",
  "documentKindSlug": "consulting-deck",
  "documentKindLabel": "Consulting deck",
  "sourceTypeSlug": "vc_research",
  "sourceTypeLabel": "VC research",
  "presentationDate": null,
  "orientation": "landscape",
  "aspectRatio": 1.777,
  "pageNumber": 45,
  "pageCount": 188,
  "prevPage": 44,
  "nextPage": 46,
  "slideType": "industry_trends",
  "function": "present_framework",
  "density": "dense",
  "nDataPoints": 0,
  "notes": "The slide references a CMU survey of 60+ papers. It includes a table comparing four paradigms: Fully Supervised (Non-Neural), Fully Supervised (Neural), Pre-train/Fine-tune, and Pre-train/Prompt/Predict.",
  "elementsJson": [
    "headline_text",
    "bullet_list",
    "comparison_table",
    "footnote"
  ],
  "metadataConfidence": 0.95,
  "imagePath": null,
  "slideHref": "/slides/019dd923-5e88-73ef-bd5c-fd5012384ae3/45",
  "deckHref": "/decks/019dd923-5e88-73ef-bd5c-fd5012384ae3",
  "deckJsonHref": "/decks/019dd923-5e88-73ef-bd5c-fd5012384ae3.json",
  "deckAnchorHref": "/decks/019dd923-5e88-73ef-bd5c-fd5012384ae3#slide-45",
  "components": [
    {
      "bbox": null,
      "kind": "callout",
      "text": "The price tag for this model flexibility is prompt engineering: how to choose the best prompt for the task at hand?",
      "attrs": null,
      "subkind": null,
      "toolName": "Visual emphasis",
      "toolSlug": "visual-emphasis",
      "confidence": null,
      "componentId": "019dd952-47c3-7407-8d27-4ade367b1fda",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.15,
        "w": 0.45,
        "x": 0.05,
        "y": 0.22
      },
      "kind": "list",
      "text": "Researchers at CMU surveyed more than 60 papers to make sense of the ongoing progress in prompting research in NLP. They thoroughly document the shift from the \"pre-train, fine-tune\" procedure to the \"pre-train, prompt and predict\" one, which is especially relevant for zero-shot learning.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "324bc883-7932-4634-9c97-bcf8235634ce",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.1,
        "w": 0.45,
        "x": 0.05,
        "y": 0.38
      },
      "kind": "list",
      "text": "To use a pre-trained language model (LM) on a new task, the dominant method was to fine-tune it by adapting the objective of the LM via a textual prompt.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "61271bed-03bc-4520-9de4-79ef3733bb09",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.08,
        "w": 0.45,
        "x": 0.05,
        "y": 0.7
      },
      "kind": "list",
      "text": "The price tag for this model flexibility is prompt engineering: how to choose the best prompt for the task at hand?",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "baba5b43-c022-4360-92ac-a1ff0d51efe4",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.2,
        "w": 0.45,
        "x": 0.05,
        "y": 0.49
      },
      "kind": "list",
      "text": "In prompting, we do the inverse: We adapt the new tasks to LMs. For example: given a model pre-trained on a multilingual dataset, \"if we choose the prompt \"English: I missed the bus today. French: __\"), an LM may be able to fill in the blank with a French translation\" without specifically training the model on a translation task.",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "e07d16dc-a2fa-4300-9e62-9b7f4cba6a64",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.45,
        "w": 0.4,
        "x": 0.55,
        "y": 0.38
      },
      "kind": "table",
      "text": "Table 1: Four paradigms in NLP. The \"engineering\" column represents the type of engineering to be done to build strong systems. The \"task relation\" column, shows the relationship between language models (LM) and other NLP tasks.",
      "attrs": null,
      "subkind": "data",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "713ce499-8d91-48b3-adb4-81b291d17a05",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.05,
        "w": 0.5,
        "x": 0.05,
        "y": 0.14
      },
      "kind": "title",
      "text": "Pre-train, prompt, predict: a new paradigm for NLP models",
      "attrs": null,
      "subkind": "headline",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "be8f2d53-f7f0-4f4d-9441-60797bdcc074",
      "frameworkName": null,
      "frameworkSlug": null
    }
  ],
  "metrics": [],
  "tools": [],
  "frameworks": [
    {
      "name": "paradigm-shift",
      "slug": null,
      "matchId": "c56b33f0-c4c9-4a32-854a-9f4d8b63e13a",
      "evidence": "Shift from pre-train/fine-tune to pre-train/prompt/predict",
      "confidence": 0.8
    }
  ],
  "arcBeats": [
    {
      "to": 152,
      "from": 5,
      "beatId": "019dd95a-0682-776c-8e35-0affbadec38e",
      "arcName": "The Triple Take",
      "arcSlug": "triple-take",
      "beatName": "The Facts (What)",
      "beatSlug": "triple-take-the-facts-what",
      "evidence": "Definitions p.5-6, Exec Summary p.7, then Sections 1-3 catalog evidence.",
      "position": 1,
      "confidence": 78,
      "parentBeatName": "Setup",
      "parentBeatSlug": "setup"
    },
    {
      "to": 152,
      "from": 10,
      "beatId": "019dd95a-0682-776c-8e35-1b3ee3a1c012",
      "arcName": "The Mountain",
      "arcSlug": "mountain",
      "beatName": "Rising Action",
      "beatSlug": "mountain-rising-action",
      "evidence": "Three sections accumulating evidence of accelerating progress.",
      "position": 2,
      "confidence": 55,
      "parentBeatName": "Development",
      "parentBeatSlug": "development"
    }
  ],
  "loops": [
    {
      "to": 51,
      "from": 41,
      "name": "Pattern Hunter",
      "slug": "02-pattern-hunter",
      "bestFor": "Time-pressed audiences, building consensus, when data is strong",
      "matchId": "019dd95a-07fe-70ce-8d3d-576c8c35c986",
      "evidence": "Codex, math limits, truthfulness, prompt engineering, SuperGLUE, multilingual LLMs.",
      "position": 6,
      "objective": "Trace LLM frontier and the prompting paradigm",
      "structure": "Evidence A -> Evidence B -> Evidence C -> Pattern/Conclusion",
      "confidence": 78,
      "description": "Group multiple pieces of evidence that together point to a pattern or conclusion"
    }
  ],
  "imagePathAlt": null,
  "thumbSrc": null,
  "thumbSrcAlt": null,
  "locked": true
}