OpenClawSkills
GitHub
Nodes & Media • TutorialHeader.readTime

Media Understanding

ReferenceNodesMediaUnderstandingPage.header.description

ReferenceNodesMediaUnderstandingPage.intro.p1

Tutorial.step

Goals

ReferenceNodesMediaUnderstandingPage.steps.01.item1

ReferenceNodesMediaUnderstandingPage.steps.01.item2

ReferenceNodesMediaUnderstandingPage.steps.01.item3

ReferenceNodesMediaUnderstandingPage.steps.01.item4

Tutorial.step

ReferenceNodesMediaUnderstandingPage.steps.02.title

ReferenceNodesMediaUnderstandingPage.steps.02.item1

ReferenceNodesMediaUnderstandingPage.steps.02.item2

ReferenceNodesMediaUnderstandingPage.steps.02.item3

ReferenceNodesMediaUnderstandingPage.steps.02.item4

5. On success:

ReferenceNodesMediaUnderstandingPage.steps.02.item5a

ReferenceNodesMediaUnderstandingPage.steps.02.item5b

ReferenceNodesMediaUnderstandingPage.steps.02.item5c

ReferenceNodesMediaUnderstandingPage.steps.02.item5d

ReferenceNodesMediaUnderstandingPage.steps.02.item6

Tutorial.step

Config overview

ReferenceNodesMediaUnderstandingPage.steps.03.p1

ReferenceNodesMediaUnderstandingPage.steps.03.item1

- tools.media.image / tools.media.audio / tools.media.video:

- defaults (prompt, maxChars, maxBytes, timeoutSeconds, language)

- Provider overrides (baseUrl, headers, providerOptions)

ReferenceNodesMediaUnderstandingPage.steps.03.item2c

ReferenceNodesMediaUnderstandingPage.steps.03.item2d

- attachments policy (mode, maxAttachments, prefer)

ReferenceNodesMediaUnderstandingPage.steps.03.item4

- tools.media.concurrency: max concurrent function executions (default 2).

Json5
{
  tools: {
    media: {
      models: [
        /* shared list */
      ],
      image: {
        /* optional overrides */
      },
      audio: {
        /* optional overrides */
      },
      video: {
        /* optional overrides */
      },
    },
  },
}

#

Tutorial.step

ReferenceNodesMediaUnderstandingPage.steps.04.title

ReferenceNodesMediaUnderstandingPage.steps.04.p1

Json5
{
  type: "provider", // default if omitted
  provider: "openai",
  model: "gpt-5.2",
  prompt: "Describe the image in <= 500 chars.",
  maxChars: 500,
  maxBytes: 10485760,
  timeoutSeconds: 60,
  capabilities: ["image"], // optional, used for multi‑modal entries
  profile: "vision-profile",
  preferredProfile: "vision-fallback",
}
Json5
{
  type: "cli",
  command: "gemini",
  args: [
    "-m",
    "gemini-3-flash",
    "--allowed-tools",
    "read_file",
    "Read the media at {{MediaPath}} and describe it in <= {{MaxChars}} characters.",
  ],
  maxChars: 500,
  maxBytes: 52428800,
  timeoutSeconds: 120,
  capabilities: ["video", "image"],
}

CLI templates can also use:

- {{MediaDir}} (directory containing the media file)

- {{OutputDir}} (scratch dir created for this run)

- {{OutputBase}} (scratch file base path, no extension)

Tutorial.step

Defaults and limits

Recommended defaults:

ReferenceNodesMediaUnderstandingPage.steps.05.item1

ReferenceNodesMediaUnderstandingPage.steps.05.item2

- maxBytes:

- Image: 10MB

- Audio: 20MB

- Video: 50MB

Rules:

ReferenceNodesMediaUnderstandingPage.steps.05.rule1

ReferenceNodesMediaUnderstandingPage.steps.05.rule2

ReferenceNodesMediaUnderstandingPage.steps.05.rule3

ReferenceNodesMediaUnderstandingPage.steps.05.rule4

#

Tutorial.step

ReferenceNodesMediaUnderstandingPage.steps.06.title

ReferenceNodesMediaUnderstandingPage.steps.06.p1

ReferenceNodesMediaUnderstandingPage.steps.06.p2

ReferenceNodesMediaUnderstandingPage.steps.06.p3

ReferenceNodesMediaUnderstandingPage.steps.06.item1

ReferenceNodesMediaUnderstandingPage.steps.06.sherpa

ReferenceNodesMediaUnderstandingPage.steps.06.whisperCli

ReferenceNodesMediaUnderstandingPage.steps.06.whisper

ReferenceNodesMediaUnderstandingPage.steps.06.item2

3. Provider keys

- Audio: OpenAI → Groq → Deepgram → Google

- Image: OpenAI → Anthropic → Google → MiniMax

- Video: Google

ReferenceNodesMediaUnderstandingPage.steps.06.disable

Json5
{
  tools: {
    media: {
      audio: {
        enabled: false,
      },
    },
  },
}

ReferenceNodesMediaUnderstandingPage.steps.06.note

Tutorial.step

Capabilities (Optional)

ReferenceNodesMediaUnderstandingPage.steps.07.p1

ReferenceNodesMediaUnderstandingPage.steps.07.p2

- openai, anthropic, minimax: Image

- google (Gemini API): Image + Audio + Video

- groq: Audio

- deepgram: Audio

ReferenceNodesMediaUnderstandingPage.steps.07.p3

ReferenceNodesMediaUnderstandingPage.steps.07.p4

Tutorial.step

ReferenceNodesMediaUnderstandingPage.steps.08.title

| Capability | Provider Integration | Notes |

| ---- |

ReferenceNodesMediaUnderstandingPage.steps.08.p3

| Audio | OpenAI, Groq, Deepgram, Google | Provider transcription (Whisper/Deepgram/Gemini). |

| Video | Google (Gemini API) | Provider video understanding. |

Tutorial.step

Recommended Providers

Image

ReferenceNodesMediaUnderstandingPage.steps.09.imageItem1

ReferenceNodesMediaUnderstandingPage.steps.09.imageItem2

Audio

- ''openai/gpt-4o-mini-transcribe'', ''groq/whisper-large-v3-turbo'', or ''deepgram/nova-3''.

- CLI Fallback: ''whisper-cli'' (whisper-cpp) or ''whisper''.

- Deepgram setup: ''Deepgram (audio transcription)''.

Video

ReferenceNodesMediaUnderstandingPage.steps.09.videoItem1

ReferenceNodesMediaUnderstandingPage.steps.09.videoItem2

Tutorial.step

Attachment file policy

ReferenceNodesMediaUnderstandingPage.steps.10.p1

- ''mode'': ''first'' (default) or ''all''

ReferenceNodesMediaUnderstandingPage.steps.10.item2

- ''prefer'': ''first'', ''last'', ''path'', ''url''

ReferenceNodesMediaUnderstandingPage.steps.10.p2

Tutorial.step

Config examples

#

Tutorial.step

1) Shared Models list + override

Json5
{
  tools: {
    media: {
      models: [
        { provider: "openai", model: "gpt-5.2", capabilities: ["image"] },
        {
          provider: "google",
          model: "gemini-3-flash-preview",
          capabilities: ["image", "audio", "video"],
        },
        {
          type: "cli",
          command: "gemini",
          args: [
            "-m",
            "gemini-3-flash",
            "--allowed-tools",
            "read_file",
            "Read the media at {{MediaPath}} and describe it in <= {{MaxChars}} characters.",
          ],
          capabilities: ["image", "video"],
        },
      ],
      audio: {
        attachments: { mode: "all", maxAttachments: 2 },
      },
      video: {
        maxChars: 500,
      },
    },
  },
}

#

Tutorial.step

ReferenceNodesMediaUnderstandingPage.steps.13.title

Json5
{
  tools: {
    media: {
      audio: {
        enabled: true,
        models: [
          { provider: "openai", model: "gpt-4o-mini-transcribe" },
          {
            type: "cli",
            command: "whisper",
            args: ["--model", "base", "{{MediaPath}}"],
          },
        ],
      },
      video: {
        enabled: true,
        maxChars: 500,
        models: [
          { provider: "google", model: "gemini-3-flash-preview" },
          {
            type: "cli",
            command: "gemini",
            args: [
              "-m",
              "gemini-3-flash",
              "--allowed-tools",
              "read_file",
              "Read the media at {{MediaPath}} and describe it in <= {{MaxChars}} characters.",
            ],
          },
        ],
      },
    },
  },
}

#

Tutorial.step

ReferenceNodesMediaUnderstandingPage.steps.14.title

Json5
{
  tools: {
    media: {
      image: {
        enabled: true,
        maxBytes: 10485760,
        maxChars: 500,
        models: [
          { provider: "openai", model: "gpt-5.2" },
          { provider: "anthropic", model: "claude-opus-4-5" },
          {
            type: "cli",
            command: "gemini",
            args: [
              "-m",
              "gemini-3-flash",
              "--allowed-tools",
              "read_file",
              "Read the media at {{MediaPath}} and describe it in <= {{MaxChars}} characters.",
            ],
          },
        ],
      },
    },
  },
}

#

Tutorial.step

ReferenceNodesMediaUnderstandingPage.steps.15.title

Json5
{
  tools: {
    media: {
      image: {
        models: [
          {
            provider: "google",
            model: "gemini-3-pro-preview",
            capabilities: ["image", "video", "audio"],
          },
        ],
      },
      audio: {
        models: [
          {
            provider: "google",
            model: "gemini-3-pro-preview",
            capabilities: ["image", "video", "audio"],
          },
        ],
      },
      video: {
        models: [
          {
            provider: "google",
            model: "gemini-3-pro-preview",
            capabilities: ["image", "video", "audio"],
          },
        ],
      },
    },
  },
}
Tutorial.step

Status output

ReferenceNodesMediaUnderstandingPage.steps.16.p1

Terminal
📎 Media: image ok (openai/gpt-5.2) · audio skipped (maxBytes)

ReferenceNodesMediaUnderstandingPage.steps.16.p2

Tutorial.step

Notes

ReferenceNodesMediaUnderstandingPage.steps.17.item1

ReferenceNodesMediaUnderstandingPage.steps.17.item2

ReferenceNodesMediaUnderstandingPage.steps.17.item3

Tutorial.step

Further reading

- ''Configuration''

ReferenceNodesMediaUnderstandingPage.steps.18.item2