{
  "docId": "019dd923-5e88-73ef-bd5d-0856e1444fb9",
  "docSlug": "8757f1b44ef7f176",
  "documentTitle": "2024 Air Street Capital The State of AI Report 2024",
  "authorId": "AirStreetCapital",
  "authorName": "Air Street Capital",
  "documentKindSlug": "consulting-deck",
  "documentKindLabel": "Consulting deck",
  "sourceTypeSlug": "vc_research",
  "sourceTypeLabel": "VC research",
  "presentationDate": null,
  "orientation": "landscape",
  "aspectRatio": 1.777,
  "pageNumber": 49,
  "pageCount": 213,
  "prevPage": 48,
  "nextPage": 50,
  "slideType": "industry_trends",
  "function": "summarize",
  "density": "dense",
  "nDataPoints": 0,
  "notes": "Discusses the transition from specialized equivariant models to domain-agnostic diffusion models in molecular modeling.",
  "elementsJson": [
    "headline_text",
    "bullet_list",
    "paragraph",
    "screenshot",
    "other"
  ],
  "metadataConfidence": 0.95,
  "imagePath": null,
  "slideHref": "/slides/019dd923-5e88-73ef-bd5d-0856e1444fb9/49",
  "deckHref": "/decks/019dd923-5e88-73ef-bd5d-0856e1444fb9",
  "deckJsonHref": "/decks/019dd923-5e88-73ef-bd5d-0856e1444fb9.json",
  "deckAnchorHref": "/decks/019dd923-5e88-73ef-bd5d-0856e1444fb9#slide-49",
  "components": [
    {
      "bbox": null,
      "kind": "callout",
      "text": "We [...] empirically show that explicitly enforcing roto-translation equivariance is not a strong requirement for generalization.",
      "attrs": null,
      "subkind": null,
      "toolName": "Visual emphasis",
      "toolSlug": "visual-emphasis",
      "confidence": null,
      "componentId": "019dd952-47c3-7407-8d14-b3b74da7cce4",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.25,
        "w": 0.3,
        "x": 0.65,
        "y": 0.65
      },
      "kind": "image",
      "text": "Generating Molecular Conformer Fields diagram",
      "attrs": null,
      "subkind": "illustration",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "48a49797-401d-43a0-a9bc-08f5d25d3e24",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.55,
        "w": 0.6,
        "x": 0.02,
        "y": 0.35
      },
      "kind": "list",
      "text": "The first shots were fired by Apple, with a paper that obtained SOTA results on predicting the 3D structures of small molecules using a non-equivariant diffusion model with a transformer encoder.\nRemarkably, the authors showed that using the domain-agnostic model did not deleteriously impact generalization and was consistently able to outperform specialist models (assuming sufficient scale was used).\nNext was AlphaFold 3, which infamously dropped all the equivariance and frames constraints from the previous model in favour of another diffusion process coupled with augmentations and, of course, scale.\nRegardless, the greatly improved training efficiency of equivariant models means the practice is likely to stay for a while (at least for academic groups working on large systems such as proteins).",
      "attrs": null,
      "subkind": "bullet",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "f7b88ec2-68da-44a1-a366-e2b00be93d0f",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.12,
        "w": 0.95,
        "x": 0.02,
        "y": 0.21
      },
      "kind": "paragraph",
      "text": "Equivariance is the idea of giving a model the inductive biases to natively handle rotations, translations and (sometimes) reflections. It has been at the core of Geometric Deep Learning and biomolecular modelling research since AlphaFold 2. However, recent works by top labs have questioned the existing mantra.",
      "attrs": null,
      "subkind": "paragraph",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "4d2430d3-ea6d-4515-b1ce-fe84355849a1",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.2,
        "w": 0.3,
        "x": 0.65,
        "y": 0.04
      },
      "kind": "quote",
      "text": "We [...] empirically show that explicitly enforcing roto-translation equivariance is not a strong requirement for generalization. Furthermore, we also show that approaches that do not explicitly enforce roto-translation equivariance (like ours) can match or outperform approaches that do.",
      "attrs": null,
      "subkind": "testimonial",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "37eebef6-521c-4453-93f5-568e094538f4",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": null,
      "kind": "quote",
      "text": "We [...] empirically show that explicitly enforcing roto-translation equivariance is not a strong requirement for generalization. Furthermore, we also show that approaches that do not explicitly enforce roto-translation equivariance (like ours) can match or outperform approaches that do. — Thomas Kipf (@tkipf)",
      "attrs": null,
      "subkind": null,
      "toolName": "Authority citation",
      "toolSlug": "authority-citation",
      "confidence": null,
      "componentId": "019dd952-47c3-7407-8d14-b771c2bac0bf",
      "frameworkName": null,
      "frameworkSlug": null
    },
    {
      "bbox": {
        "h": 0.05,
        "w": 0.5,
        "x": 0.02,
        "y": 0.13
      },
      "kind": "title",
      "text": "The Bitter Lesson: Equivariance is dead...long live equivariance!",
      "attrs": null,
      "subkind": "headline",
      "toolName": null,
      "toolSlug": null,
      "confidence": null,
      "componentId": "ad9e00da-8311-4241-9300-5413ba799b25",
      "frameworkName": null,
      "frameworkSlug": null
    }
  ],
  "metrics": [],
  "tools": [
    {
      "name": "Analytical method",
      "slug": "analytical-method",
      "agent": null,
      "layer": "slide",
      "matchId": "bc88cc5d-a8c4-44b8-bd5c-9318f335c6e6",
      "evidence": "Equivariance is the idea of giving a model the inductive biases to natively handle rotations, translations and (sometimes) reflections",
      "confidence": 0.6
    }
  ],
  "frameworks": [],
  "arcBeats": [
    {
      "to": 153,
      "from": 9,
      "beatId": "019dd95a-0682-776c-8e35-5f2398b8d1d0",
      "arcName": "The Triple Take",
      "arcSlug": "triple-take",
      "beatName": "The Facts (What)",
      "beatSlug": "triple-take-the-facts-what",
      "evidence": "Exec summary + Research + Industry sections inventory the year",
      "position": 1,
      "confidence": 78,
      "parentBeatName": "Setup",
      "parentBeatSlug": "setup"
    },
    {
      "to": 85,
      "from": 12,
      "beatId": "019dd95a-0682-776c-8e35-6d9d8ee06d0a",
      "arcName": "Voyage and Return",
      "arcSlug": "voyage-return",
      "beatName": "The Unknown",
      "beatSlug": "voyage-return-the-unknown",
      "evidence": "Research section explores frontier model uncharted territory",
      "position": 2,
      "confidence": 55,
      "parentBeatName": "Complication",
      "parentBeatSlug": "complication"
    }
  ],
  "loops": [
    {
      "to": 58,
      "from": 45,
      "name": "Zoom In",
      "slug": "06-zoom-in",
      "bestFor": "Technical deep-dives, case studies, detailed analysis",
      "matchId": "019dd95a-07fe-70ce-8d3e-3bc439910222",
      "evidence": "Nobels, AlphaFold3, AlphaProteo, ESM3, gene editors, materials, brain fMRI, speech-from-cortex case studies",
      "position": 7,
      "objective": "Zoom into AI for science domain by domain",
      "structure": "The Big Picture -> Key Area of Focus -> Specific Detail -> Implication",
      "confidence": 80,
      "description": "Start broad, then progressively focus on specific details that prove your point"
    }
  ],
  "imagePathAlt": null,
  "thumbSrc": null,
  "thumbSrcAlt": null,
  "locked": true
}