Labsco
anthropics logo

ui-toolkit/web

✓ Official22,300

by anthropic · part of anthropics/knowledge-work-plugins

ui-toolkit/web — an installable skill for AI agents, published by anthropics/knowledge-work-plugins.

🔥🔥FreeQuick setup
🧩 One of 7 skills in the anthropics/knowledge-work-plugins package — works on its own, and pairs well with its siblings.

ui-toolkit/web — an installable skill for AI agents, published by anthropics/knowledge-work-plugins.

Inspect the full instructions your agent will receiveExpand

This is the exact playbook injected into your agent when the skill activates — shown here so you can audit it before installing. You don't need to read it to use the skill.


name: video-sdk/web description: "Zoom Video SDK for Web - JavaScript/TypeScript integration for browser-based video sessions, real-time communication, screen sharing, recording, and live transcription" user-invocable: false triggers:

  • "video sdk web"
  • "custom video web"
  • "attachvideo"
  • "peer-video-state-change"
  • "web videosdk"

Zoom Video SDK - Web Development

Expert guidance for developing with the Zoom Video SDK on Web. This SDK enables custom video applications in the browser with real-time video/audio, screen sharing, cloud recording, live streaming, chat, and live transcription.

This skill is for custom video sessions, not embedded Zoom meetings. If the user wants a custom UI for a real Zoom meeting, route to ../../meeting-sdk/web/component-view/SKILL.md.

Official Documentation: https://developers.zoom.us/docs/video-sdk/web/ API Reference: https://marketplacefront.zoom.us/sdk/custom/web/modules.html Sample Repository: https://github.com/zoom/videosdk-web-sample

Quick Links

New to Video SDK? Follow this path:

  1. SDK Architecture Pattern - Universal 3-step pattern for ANY feature
  2. Session Join Pattern - Complete working code to join a session
  3. Video Rendering - Display video with attachVideo()
  4. Event Handling - Required events for video/audio

Reference:

Having issues?

  • Video not showing → Video Rendering (use attachVideo, not renderVideo)
  • getMediaStream() returns undefined → Call AFTER join() completes
  • Quick diagnostics → Common Issues

SDK Overview

The Zoom Video SDK for Web is a JavaScript library that provides:

  • Session Management: Join/leave video SDK sessions
  • Video/Audio: Start/stop camera and microphone
  • Screen Sharing: Share screens or browser tabs
  • Cloud Recording: Record sessions to Zoom cloud
  • Live Streaming: Stream to RTMP endpoints
  • Chat: In-session messaging
  • Command Channel: Custom command messaging
  • Live Transcription: Real-time speech-to-text
  • Subsessions: Breakout room support
  • Whiteboard: Collaborative whiteboard features
  • Virtual Background: Blur or custom image backgrounds

SDK Lifecycle (CRITICAL ORDER)

The SDK has a strict lifecycle. Violating it causes silent failures.

Copy & paste — that's it
1. Create client:     client = ZoomVideo.createClient()
2. Initialize:        await client.init('en-US', 'Global', options)
3. Join session:      await client.join(topic, signature, userName, password)
4. Get stream:        stream = client.getMediaStream()  ← ONLY AFTER JOIN
5. Start media:       await stream.startVideo() / await stream.startAudio()

Common Mistake:

Copy & paste — that's it
// WRONG: Getting stream before joining
const stream = client.getMediaStream();  // Returns undefined!
await client.join(...);

// CORRECT: Get stream after joining
await client.join(...);
const stream = client.getMediaStream();  // Works!

Critical Gotchas and Best Practices

getMediaStream() ONLY Works After join()

The #1 issue that causes video/audio to fail:

Copy & paste — that's it
// WRONG
const stream = client.getMediaStream();  // undefined!
await client.join(...);

// CORRECT
await client.join(...);
const stream = client.getMediaStream();  // Works

Use attachVideo() NOT renderVideo()

renderVideo() is deprecated. Use attachVideo() which returns a VideoPlayer element:

Copy & paste — that's it
import { VideoQuality } from '@zoom/videosdk';

// CORRECT: attachVideo returns element to append
const videoElement = await stream.attachVideo(userId, VideoQuality.Video_360P);
document.getElementById('video-container').appendChild(videoElement);

// WRONG: renderVideo is deprecated
await stream.renderVideo(canvas, userId, ...);  // Don't use!

Video Rendering is Event-Driven (CRITICAL)

You MUST listen for events to properly render participant videos:

Copy & paste — that's it
// When another participant's video state changes
client.on('peer-video-state-change', async (payload) => {
  const { action, userId } = payload;
  
  if (action === 'Start') {
    // Participant turned on video - attach it
    const element = await stream.attachVideo(userId, VideoQuality.Video_360P);
    container.appendChild(element);
  } else if (action === 'Stop') {
    // Participant turned off video - detach it
    await stream.detachVideo(userId);
  }
});

// When participants join/leave
client.on('user-added', (payload) => {
  // New participant joined - check if their video is on
  const users = client.getAllUser();
  // Render videos for users with bVideoOn === true
});

client.on('user-removed', (payload) => {
  // Participant left - clean up their video element
  stream.detachVideo(payload[0].userId);
});

Peer Video on Mid-Session Join

Existing participants' videos won't auto-render when you join mid-session.

Copy & paste — that's it
// After joining, render existing participants' videos
const renderExistingVideos = async () => {
  await new Promise(resolve => setTimeout(resolve, 500));
  
  const users = client.getAllUser();
  const currentUserId = client.getCurrentUserInfo().userId;
  
  for (const user of users) {
    if (user.bVideoOn && user.userId !== currentUserId) {
      const element = await stream.attachVideo(user.userId, VideoQuality.Video_360P);
      document.getElementById(`video-${user.userId}`).appendChild(element);
    }
  }
};

CDN Race Condition with ES Modules

When using <script type="module"> with CDN, the SDK may not be loaded yet:

Copy & paste — that's it
function waitForSDK(timeout = 10000) {
  return new Promise((resolve, reject) => {
    if (typeof WebVideoSDK !== 'undefined') {
      resolve();
      return;
    }
    const start = Date.now();
    const check = setInterval(() => {
      if (typeof WebVideoSDK !== 'undefined') {
        clearInterval(check);
        resolve();
      } else if (Date.now() - start > timeout) {
        clearInterval(check);
        reject(new Error('SDK failed to load'));
      }
    }, 100);
  });
}

await waitForSDK();
const ZoomVideo = WebVideoSDK.default;

SharedArrayBuffer for HD Video

For optimal performance and HD video, configure these headers on your server:

Copy & paste — that's it
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

Note: As of v1.11.2, SharedArrayBuffer is elective (not strictly required).

Check HD Capability Before Enabling

Copy & paste — that's it
const stream = client.getMediaStream();

// Check if 720p is supported
const hdSupported = stream.isSupportHDVideo();

// Get maximum video quality
const maxQuality = stream.getVideoMaxQuality();
// 0=90P, 1=180P, 2=360P, 3=720P, 4=1080P

// Start video with HD
if (hdSupported) {
  await stream.startVideo({ hd: true });
}

Screen Share Rendering Mode Check

Copy & paste — that's it
const stream = client.getMediaStream();

// Check which element type to use
if (stream.isStartShareScreenWithVideoElement()) {
  // Use video element
  const video = document.getElementById('share-video');
  await stream.startShareScreen(video as unknown as HTMLCanvasElement);
} else {
  // Use canvas element
  const canvas = document.getElementById('share-canvas');
  await stream.startShareScreen(canvas);
}

Key Features

Video Quality Enum

Copy & paste — that's it
import { VideoQuality } from '@zoom/videosdk';

VideoQuality.Video_90P   // 0
VideoQuality.Video_180P  // 1
VideoQuality.Video_360P  // 2 (recommended for most cases)
VideoQuality.Video_720P  // 3
VideoQuality.Video_1080P // 4

Virtual Backgrounds

Copy & paste — that's it
const stream = client.getMediaStream();

// Always check support first
if (stream.isSupportVirtualBackground()) {
  // Blur background
  await stream.updateVirtualBackgroundImage('blur');
  
  // Custom image background
  await stream.updateVirtualBackgroundImage('https://example.com/bg.jpg');
  
  // Remove virtual background
  await stream.updateVirtualBackgroundImage(undefined);
}

Video Processor (Custom Effects)

The VideoProcessor class allows you to intercept and modify video frames:

Copy & paste — that's it
// video-processor-worker.js
class MyVideoProcessor extends VideoProcessor {
  processFrame(input, output) {
    const ctx = output.getContext('2d');
    ctx.drawImage(input, 0, 0);
    
    // Add overlay
    ctx.fillStyle = 'white';
    ctx.font = '24px Arial';
    ctx.fillText('Live', 20, 40);
    
    return true;
  }
}

WebRTC Mode

Enable WebRTC mode for direct peer-to-peer streaming with HD video support:

Copy & paste — that's it
await client.init('en-US', 'Global', {
  patchJsMedia: true,
  webrtc: true  // Enable WebRTC mode
});

Feature Clients

Access specialized clients from the VideoClient:

ClientAccess MethodPurpose
Streamclient.getMediaStream()Video, audio, screen share, devices
Chatclient.getChatClient()Send/receive messages
Commandclient.getCommandClient()Custom commands (reactions, etc.)
Recordingclient.getRecordingClient()Cloud recording control
Transcriptionclient.getLiveTranscriptionClient()Live captions
LiveStreamclient.getLiveStreamClient()RTMP streaming
Subsessionclient.getSubsessionClient()Breakout rooms
Whiteboardclient.getWhiteboardClient()Collaborative whiteboard

Common Tasks

Start/Stop Video

Copy & paste — that's it
await stream.startVideo();
await stream.stopVideo();

Start/Stop Audio

Copy & paste — that's it
await stream.startAudio();
await stream.muteAudio();
await stream.unmuteAudio();
await stream.stopAudio();

Switch Devices

Copy & paste — that's it
// Get available devices
const cameras = stream.getCameraList();
const mics = stream.getMicList();
const speakers = stream.getSpeakerList();

// Switch devices
await stream.switchCamera(cameraId);
await stream.switchMicrophone(micId);
await stream.switchSpeaker(speakerId);

Screen Sharing

Copy & paste — that's it
// Start sharing
await stream.startShareScreen(canvas);

// Stop sharing
await stream.stopShareScreen();

// Receive share
client.on('active-share-change', async (payload) => {
  if (payload.state === 'Active') {
    await stream.startShareView(canvas, payload.userId);
  } else {
    await stream.stopShareView();
  }
});

Chat

Copy & paste — that's it
const chatClient = client.getChatClient();

// Send to everyone
await chatClient.send('Hello, everyone!');

// Send to specific user
await chatClient.sendToUser(userId, 'Private message');

// Receive messages
client.on('chat-on-message', (payload) => {
  console.log(`${payload.sender.name}: ${payload.message}`);
});

Recording (Host Only)

Copy & paste — that's it
const recordingClient = client.getRecordingClient();

await recordingClient.startCloudRecording();
await recordingClient.stopCloudRecording();

client.on('recording-change', (payload) => {
  console.log('Recording status:', payload.state);
});

Leave/End Session

Copy & paste — that's it
// Leave session (others stay)
await client.leave();

// End session for ALL participants (host only)
await client.leave(true);

Error Handling

Common Join Errors

ErrorCauseSolution
Invalid signatureJWT expired or malformedGenerate new signature
Session does not existHost hasn't started yetShow "waiting" message, retry
Permission deniedUser denied camera/micRequest permission again

Example Error Handler

Copy & paste — that's it
try {
  await client.join(topic, signature, userName, password);
} catch (error) {
  if (error.reason?.includes('signature')) {
    // Regenerate signature and retry
  } else if (error.reason?.includes('Session')) {
    // Show "Waiting for host..." and poll
  } else if (error.reason?.includes('Permission')) {
    // Guide user to enable permissions
  }
  console.error('Join failed:', error);
}

Browser Compatibility

FeatureChromeFirefoxSafariEdge
Video80+75+14+80+
Audio80+75+14+80+
Screen Share80+75+15+80+
Virtual BG80+90+-80+

Safari Notes:

  • Virtual background not supported
  • Screen sharing requires macOS 15+

CORS Errors (Telemetry)

CORS errors to log-external-gateway.zoom.us are harmless.

These are caused by COOP/COEP headers blocking telemetry requests. They don't affect SDK functionality.

Complete Documentation Library

Core Concepts

Complete Examples

Framework Integrations

Troubleshooting

References

Official Sample Repositories

Resources


Need help? Start with SKILL.md for complete navigation.

Merged from video-sdk/web/SKILL.md

Zoom Video SDK Web - Complete Documentation Index

Documentation Structure

Copy & paste — that's it
video-sdk/web/
├── SKILL.md                           # Main skill overview
├── SKILL.md                           # This file - navigation guide
│
├── concepts/                          # Core architectural patterns
│   ├── sdk-architecture-pattern.md   # Universal formula for ANY feature
│   └── singleton-hierarchy.md        # 4-level navigation guide
│
├── examples/                          # Complete working code
│   ├── session-join-pattern.md       # JWT auth + session join
│   ├── video-rendering.md            # attachVideo() patterns
│   ├── screen-share.md               # Send and receive screen shares
│   ├── event-handling.md             # Required events
│   ├── chat.md                       # Chat implementation
│   ├── command-channel.md            # Command channel messaging
│   ├── recording.md                  # Cloud recording control
│   ├── transcription.md              # Live transcription/captions
│   ├── react-hooks.md                # Official @zoom/videosdk-react library
│   └── framework-integrations.md     # Next.js, Vue/Nuxt, ZFG patterns
│
├── troubleshooting/                   # Problem solving guides
│   └── common-issues.md              # Quick diagnostic workflow
│
└── references/                        # Reference documentation
    ├── web-reference.md              # API hierarchy, methods, error codes
    └── events-reference.md           # All event types

By Use Case

I want to build a video app

  1. SDK Architecture Pattern - Understand the pattern
  2. Session Join Pattern - Join sessions
  3. Video Rendering - Display video
  4. Event Handling - Listen for video events

I'm getting runtime errors

  1. Common Issues - Error code tables
  2. "getMediaStream() is undefined" → Call AFTER join() completes

I want to receive screen shares

  1. Screen Share - startShareView() patterns
  2. Event Handling - active-share-change event

I want to send screen shares

  1. Screen Share - startShareScreen() patterns
  2. Check isStartShareScreenWithVideoElement() for element type

I want to use chat

  1. Chat - Send/receive messages
  2. getChatClient() for ChatClient access

I want to record sessions

  1. Recording - Cloud recording (host only)
  2. getRecordingClient() for RecordingClient access

I want to use live transcription

  1. Transcription - Enable live captions
  2. getLiveTranscriptionClient() for LiveTranscriptionClient access

I want to use command channel

  1. Command Channel - Custom signaling between participants
  2. Must call getCommandClient() AFTER join()

I want to implement a specific feature

  1. SDK Architecture Pattern - START HERE!
  2. Singleton Hierarchy - Navigate to the feature
  3. API Reference - Method signatures

I'm using React

  1. React Hooks - Official @zoom/videosdk-react library
  2. Provides hooks: useSession, useSessionUsers, useVideoState, useAudioState
  3. Pre-built components: VideoPlayerComponent, ScreenSharePlayerComponent

I'm using Next.js or Vue/Nuxt

  1. Framework Integrations - SSR considerations
  2. Server-side JWT generation patterns
  3. Client-side only SDK usage

Most Critical Documents

1. SDK Architecture Pattern (MASTER DOCUMENT)

concepts/sdk-architecture-pattern.md

The universal 5-step pattern:

  1. Create client
  2. Initialize SDK
  3. Join session
  4. Get stream
  5. Use features + listen to events

2. Common Issues (MOST COMMON PROBLEMS)

troubleshooting/common-issues.md

Common issues:

  • getMediaStream() returns undefined
  • Video not displaying
  • renderVideo() deprecated

3. Singleton Hierarchy (NAVIGATION MAP)

concepts/singleton-hierarchy.md

4-level deep navigation showing how to reach every feature.


Key Learnings

Critical Discoveries:

  1. getMediaStream() ONLY works after join()

  2. Use attachVideo() NOT renderVideo()

    • renderVideo() is deprecated
    • attachVideo() returns a VideoPlayer element to append to DOM
    • See: Video Rendering
  3. The SDK is Event-Driven

    • You MUST listen for events to render participant videos
    • key events: peer-video-state-change, user-added, user-removed
    • See: Event Handling
  4. Peer Videos on Mid-Session Join

    • Existing participants' videos won't auto-render
    • Must manually iterate getAllUser() and attachVideo()
    • See: Video Rendering
  5. CDN vs NPM

    • CDN exports as WebVideoSDK.default, not ZoomVideo
    • Some networks/ad blockers may block source.zoom.us - allowlist or use a permitted fallback strategy
    • See: Session Join Pattern
  6. SharedArrayBuffer for HD

    • Required for 720p/1080p video
    • Need COOP/COEP headers on server
    • Check with stream.isSupportHDVideo()
  7. Screen Share Element Type

    • Check isStartShareScreenWithVideoElement() for correct element type
    • See: Screen Share
  8. Command Channel Setup Order

    • Must call getCommandClient() AFTER client.join()
    • Register listeners AFTER join, not before
    • Web uses getCommandClient() not getCmdChannel()
    • See: Command Channel
  9. Command Channel is Session-Scoped

    • Does NOT span across different sessions
    • Both sender and receiver must be in the same session

Quick Reference

"getMediaStream() returns undefined"

→ Call AFTER join() completes

"Video not showing"

Video Rendering - Use attachVideo(), check events

"renderVideo() doesn't work"

Video Rendering - Use attachVideo() instead

"How do I implement [feature]?"

SDK Architecture Pattern

"How do I navigate to [client]?"

Singleton Hierarchy

"What error code means what?"

Common Issues


Document Version

Based on Zoom Video SDK for Web v2.3.x


Happy coding!

Remember: The SDK Architecture Pattern is your key to unlocking the entire SDK. Read it first!

Operations

  • RUNBOOK.md - 5-minute preflight and debugging checklist.