RELATED 01
State Memory MCP
Local SQLite graph of tasks, decisions and blockers that coding assistants keep across sessions
41GitHub stars
176/wknpm downloads
38popularity score
vision-memory-mcp
Local cache of UI screenshots so a coding agent can recognise screens it has already seen
By PuterVision TypeScript Local · stdio Knowledge and memoryTestingDeveloper tools
37 popularity score ★ 40 ↓ 147/wk
Connect your client
Local (npm)
Add in one click
Opens the app, which asks before it adds the server.
Or add it by command or config
FirstTerminal
npm install -g @putervision/vision-memory-mcp
vision-memory-mcp init --yesClaude CodeTerminal
claude mcp add vision-memory -- vision-memory-mcp runClaude Desktopclaude_desktop_config.json
{
"mcpServers": {
"vision-memory": {
"command": "vision-memory-mcp",
"args": [
"run"
]
}
}
}Cursor~/.cursor/mcp.json
{
"mcpServers": {
"vision-memory": {
"command": "vision-memory-mcp",
"args": [
"run"
]
}
}
}VS Code.vscode/mcp.json
{
"servers": {
"vision-memory": {
"type": "stdio",
"command": "vision-memory-mcp",
"args": [
"run"
]
}
}
}Codex~/.codex/config.toml
[mcp_servers.vision-memory]
command = "vision-memory-mcp"
args = ["run"]Gemini CLITerminal
gemini mcp add --scope user vision-memory vision-memory-mcp -- runSettings 11
LANCEDB_PATH |
Optional: Where the store is kept; default .vision-memory-mcp relative to the working directory. |
|---|---|
VISION_MODEL_ENABLED |
Optional: true to call a vision model on cache misses; default false. |
VISION_MODEL_ENDPOINT |
Optional: OpenAI-compatible base URL for the fallback; default http://localhost:1234/v1. |
VISION_MODEL_NAME |
Optional: Model used for the fallback; default gpt-4o. |
OPENAI_API_KEY |
Optional: Key for the fallback when it points at OpenAI. |
STRICT_MODE |
Optional: true limits file paths to the project root and disables the fallback. |
OFFLINE_MODE |
Optional: true loads CLIP from local files only. |
CLIP_MODEL_PATH |
Optional: Path to a pre-downloaded CLIP model. |
MAX_LANCEDB_SIZE_MB |
Optional: Store size before old states are evicted; 500 MB in code. |
MAX_IMAGE_SIZE_MB |
Optional: Largest accepted image; default 10. |
TTL_DEFAULT_MS |
Optional: How long states are kept; default 7 days. |
What it does
vision-memory-mcp keeps a memory of the screens a coding agent has looked at. When the agent hands it a screenshot (as base64 or a local file path, optionally with an accessibility tree), it first checks an in-memory cache, then compares perceptual hashes (dHash) against stored states, then runs a CLIP embedding search. On a hit it returns the stored description and element handles, so the agent need not send the image to a vision model again. On a miss it can call a vision model you configure, or simply store the new state with the description you supply.
Around that cache it records transitions between screens as a graph, which it uses to suggest the next action (CSS selector and coordinates) and to plan shortest paths between screens. It can also hold design mockups as baselines for visual regression checks, take named snapshots of the store, ingest WebM or MP4 test recordings as keyframes, build hashed evidence packs, and export trajectories as JSON or as LLaVA / Qwen2-VL fine-tuning data. Several tools return hints meant for its companion server, state-memory-mcp, from the same maker.
Everything is kept in a LanceDB store in .vision-memory-mcp/ under the working directory by default (LANCEDB_PATH). The package is also a CLI: init scaffolds the store, .env and IDE rule files, and other commands inspect, snapshot, diagnose and open a graph viewer. Version 1.4.2 registers 15 tools in src/tools/handlers.ts, matching the README; older tool names from before 1.0 are still accepted through a compatibility shim but are not listed.
What it needs
| Runtime | Node.js 18+ |
|---|
Node.js 18.18.0 or newer. sharp and @lancedb/lancedb are optional dependencies with native builds; the doctor command checks them. The CLIP model (Xenova/clip-vit-base-patch32 by default) is downloaded on first use and needs about 300 MB of RAM; start with run --skip-model-load to use hashes only. Video ingest calls ffmpeg and ffprobe from PATH. The optional vision-model fallback needs an OpenAI-compatible endpoint (and OPENAI_API_KEY for OpenAI).
Limits
Without the vision-model fallback (off by default) it does not describe new screens itself; it stores what the agent passes in. Images over 10 MB are refused (MAX_IMAGE_SIZE_MB, not in the configuration guide), and file_path refuses paths under /etc, /proc, /sys, /dev, .ssh, .aws and .env (src/tools/handlers.ts). If ffmpeg is missing, video ingest falls back to synthetic frames rather than failing (src/core/video-pipeline.ts). The store default is 500 MB in src/config.ts, though the configuration guide shows 1000. Its makers say it does not suit backend-only work, live 60 fps video, or machines under 512 MB RAM.
Who it suits
Suits developers whose agent takes repeated screenshots of a web or app UI, for example during Playwright runs or frontend work, and who want to reuse earlier readings locally instead of paying for a vision call each time. It runs entirely on your machine and writes only to its own store. For work with no UI, its makers point to state-memory-mcp on its own, which we also list; that server tracks tasks as a graph and holds no images. Pick something else if you need the server to drive a browser itself: this one only remembers what it is shown.
Tools 15
analyze_screenshot- Check one screenshot, or a batch of up to 20, against the cache and store it on a miss.
recall_memory- Search stored screens by text or by an image, read-only.
get_session_context- Summary of recent states, cache hit rates, storage use and version.
predict_next_action- Suggest the next UI action and target from past transition success.
record_outcome- Log an action between two screens, or a blocker for state-memory-mcp.
get_navigation_paths- Find the shortest known path between two screens.
wait_for_visual_state- Poll until a given state appears in the store or time runs out.
compare_states- Diff two stored screens, or two ingested videos.
manage_video- Ingest WebM/MP4 recordings as keyframes and search their timelines.
create_evidence_pack- Bundle keyframes and fingerprints into a hashed evidence record.
export_trajectories- Export screen sequences as JSON or LLaVA / Qwen2-VL training data.
manage_visual_spec- Register design baselines and check screens against them.
manage_snapshot- Save, diff, export or restore checkpoints of the store.
undo_visual_mutation- Revert the latest stored state or transition.
forget_state- Delete a stored state with its hashes and embeddings.