drio
Open app

Multimodal Model Context Protocal Server

Source

A multimodal mcp server

Catalog onlyCatalog onlySTDIO

Overview

The Multimodal Model Context Protocol Server is a server implementation designed to handle multimodal data indexing and querying, including audio, video, images, and documents.

To use the server, clone the repository, install the required packages, and run the services using Docker. Each service can be accessed through designated endpoints for audio, video, image, and document indexing.

  • Audio file indexing with transcription capabilities - Video file indexing with frame extraction - Image indexing with object detection - Document indexing with text extraction and Retrieval-Augmented Generation (RAG) support - Multi-index support for various data types
  1. Indexing and searching audio files for content-based retrieval.
  2. Extracting frames from videos for analysis and search.
  3. Performing similarity searches on images.
  4. Extracting text from documents for enhanced search capabilities.

Add to your AI client

Use these steps to connect Multimodal Model Context Protocal Server in Cursor, Claude, VS Code, and other MCP-compatible apps. The same JSON appears in the Use with menu above for one-click copy.

Cursor

Add this to your .cursor/mcp.json file in your project root, then restart Cursor.

.cursor/mcp.json

{
  "mcpServers": {
    "mcp-server-pixeltable-pixeltable": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-mcp-server-pixeltable-pixeltable"
      ]
    }
  }
}

Claude Desktop

Add this server entry to the mcpServers object in your Claude Desktop config, then restart the app.

~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows)

{
  "mcpServers": {
    "mcp-server-pixeltable-pixeltable": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-mcp-server-pixeltable-pixeltable"
      ]
    }
  }
}

Claude Code

Add this to your project's .mcp.json file. Claude Code will detect it automatically.

.mcp.json (project root)

{
  "mcpServers": {
    "mcp-server-pixeltable-pixeltable": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-mcp-server-pixeltable-pixeltable"
      ]
    }
  }
}

VS Code (Copilot)

Add this to your .vscode/mcp.json file. Requires the GitHub Copilot extension with MCP support enabled.

.vscode/mcp.json

{
  "servers": {
    "mcp-server-pixeltable-pixeltable": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-mcp-server-pixeltable-pixeltable"
      ]
    }
  }
}

Windsurf

Add this to your Windsurf MCP config file, then restart Windsurf.

~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "mcp-server-pixeltable-pixeltable": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-mcp-server-pixeltable-pixeltable"
      ]
    }
  }
}

Cline

Open Cline settings, navigate to MCP Servers, and add this server configuration.

Cline MCP Settings (via UI)

{
  "mcpServers": {
    "mcp-server-pixeltable-pixeltable": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-mcp-server-pixeltable-pixeltable"
      ]
    }
  }
}

FAQ

What types of data can be indexed?

The server can index audio, video, images, and documents.

How do I run the server locally?

You can run the server locally using Docker by following the installation instructions provided in the repository.

Is there support for community engagement?

Yes! You can join the Pixeltable community on Discord for support and discussions.7:["$","div",null,{"className":"container mx-auto flex flex-col gap-4","children":["$L26","$L27",["$","$L28",null,{"currentProject":{"id":2745,"uuid":"88c911bb-1721-43dc-906f-85b43fb3758e","name":"mcp-server-pixeltable","title":"Multimodal Model Context Protocal Server","description":"A multimodal mcp server","avatar_url":"https://avatars.githubusercontent.com/u/160283145?v=4","created_at":"2025-03-14T00:57:35.018Z","updated_at":"2025-03-14T01:12:24.007Z","status":"created","author_name":"pixeltable","author_avatar_url":"https://avatars.githubusercontent.com/u/160283145?v=4","tags":"mcp-server,multimodal,data-indexing","category":"research-and-data","is_featured":false,"sort":1,"url":"https://github.com/pixeltable/mcp-server-pixeltable","target":"_self","content":"$29","summary":"$2a","img_url":"https://camo.githubusercontent.com/f50916cb35299eec933c47d8f5086933c5b9252b2b7831239062d9236eca8f78/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f507974686f6e2d332e392b2d626c75652e737667","type":null,"metadata":"{\"star\":\"0\",\"license\":\"\",\"language\":\"Python\",\"is_official\":false,\"latest_commit_time\":\"2025-03-13 16:34:40\"}","user_uuid":null,"tools":null,"sse_url":null,"sse_provider":null,"sse_params":null,"is_official":false,"server_command":null,"server_params":null,"server_config":null,"allow_call":false,"is_innovation":false,"is_dxt":false,"dxt_manifest":null,"dxt_file_url":null,"is_audit":false},"randomProjects":[],"currentServerKey":"$undefined"}]]}]

Research and data