# Agent memory · VideoDB

> Give your agent a memory of everything it sees.

A live feed is gone as soon as it plays. VideoDB reads it as it arrives, acts on what matters now, and keeps all of it as memory: an index of what was seen, and the frames behind any moment in about half a second.

Canonical: https://videodb.io/agent-memory · Add to Claude: https://videodb.io/claude · Add to ChatGPT: https://videodb.io/chatgpt · API key: https://console.videodb.io/auth · Talk to us: https://videodb.io/contact.md

Four steps take a live feed to memory:

1. **See**: cameras, drones and screens, as they arrive.
2. **Decide**: your events, checked on every window.
3. **Remember**: long-term memory, an index of what was seen.
4. **Recall**: short-term memory, the frames a task needs.

## Every window of the feed gets a decision

The feed is read a few seconds at a time. Each window is described, checked against the events you wrote, and written to the index. The ones that match also go out as alerts. A camera like this one, live: https://videodb.io/live-video.md

The page shows an example, invented: four minutes of a front door camera at night, read in 48 windows of five seconds, with one event to watch for, "a person is checking the lock". Forty-seven windows are described and indexed without an alert. One matches the event, and an alert goes out with its clip.

| Time | What the window wrote | Decision |
| --- | --- | --- |
| 02:13:55 | A car passes on the street | indexed |
| 02:14:00 | A person walks up the path to the door | indexed |
| 02:14:05 | A person stops at the front door and tries the handle | alert sent |
| 02:14:15 | The person walks back to the street | indexed |

## Long-term memory finds the moment, short-term memory shows it

In the morning someone asks what happened at the door. The index points to 02:14, and the frames from those seconds come back for a model to read.

- **Long-term**: the index of what was seen and decided, kept for months and written as the feed arrives. In the example it holds a courier leaving a parcel by the door on Monday at 14:02, a person trying the door handle on Tuesday at 02:14, and a car leaving the driveway on Tuesday at 07:28.
- **Short-term**: the frames a task needs, pulled from the index. Asked what happened at the door last night, the index points to 02:14:05 to 02:14:10, and those frames come back as a stream in 441 ms, the measured median for a five-second window.

How the frame times were measured: https://videodb.io/blog/frame-access-that-keeps-research-moving.md

## Memory that builds on itself

The frames never change. Everything above them is an index, and every task can add one.

- **Ask three ways**: by meaning, by a filter such as camera or time, and by count.
- **Write back**: what a task works out is indexed under its own name, so the same question is one lookup next time.
- **New indexes**: run a new prompt over frames you already have. Nothing is recorded again.
- **Combine**: one query can span several indexes, and one index name can cover every camera in a building.

## Why the memory sits outside the model

Four ways to give an agent a memory of video, and what each one leaves out.

| Approach | What it keeps | What it leaves out |
| --- | --- | --- |
| A longer context window | The last minute or so of video, in full detail | Anything older. Nothing can be searched, and the cost grows with every frame kept |
| Notes in a vector database | Searchable text over a long time, cheap to store | The picture. A note is written before the question is known, and there is no second look at the moment behind it |
| Video files and a clip pipeline | Every frame, in principle | An index, and time. Reaching one second means finding the file, decoding it and cutting a clip |
| VideoDB | The index and the frames in one system, live feeds included. Indexes that stack and combine | The model, which you bring |

On a 100-hour benchmark for visual moment retrieval, VideoDB finds a matching clip for more queries than any other pipeline tested. The benchmark: https://videodb.io/blog/a-faster-path-to-the-next-policy.md

## Recall in code, or in a sentence

The same memory answers the SDK, any MCP client, and a plain question in Claude or ChatGPT.

```python
import videodb

coll = videodb.connect().get_collection()
cam = coll.get_rtstream("rts-...")   # the front door camera
index = cam.get_index("...")         # its long-term memory

# Ask: the index turns "last night" into a moment
found = cam.search(query="a person at the front door at night",
                   index_id=index.id)
shot = max(found.get_shots(), key=lambda s: s.start)  # the newest one

# Recall: the frames around that moment, as a stream a model reads
frames_url = shot.generate_stream()

# Remember: index what the task worked out,
# so the same question is one lookup next time
```

What comes back: `text`, what was seen, as it was written; `start` and `end`, the seconds it covers; `stream_url`, those frames as a stream.

Asked of an agent in Claude or ChatGPT: "What happened at the front door last night?", "How many deliveries came this week?", "Show me the last time the gate was left open."

## Questions about agent memory

- **What is long-term memory for an agent?** A timestamped index of what its cameras and screens showed: short written records, each pointing back to the frames it came from. It is kept for days, weeks and months, and it stays small, because it holds text and pointers while the frames stay in storage.
- **What is short-term memory?** The frames the agent pulls back for the task in front of it: a few seconds around the moments its index points to, read now, for this question. Long-term memory says where to look. Short-term memory shows what was there.
- **What is real-time memory?** What is happening right now: the live feed, written into the index as it arrives, a few seconds behind the present. It keeps the long-term memory current, and it supplies the last few seconds when a task is about now.
- **Why keep the frames when the agent already has notes?** A note is written before the question is known. It holds what seemed important at the time, so it can miss the detail a later task needs, such as which hand was on the door or what the label said. The index gets the agent to the right second. The frames show what was in it.
- **Why not give the model a longer context window?** A day of video from one camera is 86,400 frames, even at one frame a second. Keeping hours of raw frames in a model's context is slow and costly, and most of them have nothing to do with the task. A memory outside the model keeps the whole record and hands the model only the seconds it needs.
- **Can I build this with a vector database and files in a bucket?** You can keep notes in a vector database and recordings in a bucket. What is missing is the path from a note back to its frames while the task is still running, and an index over the recordings themselves. VideoDB keeps the index and the frames in one system, and new indexes can be built over the same frames at any time.
- **Does it work for screens as well as cameras?** Yes. A screen is a live feed like any other. An agent that watched a desktop for a week needs the same two things: an index of what it saw, and the exact frames back when a task refers to them.
- **Where do the recordings stay?** In your own storage. VideoDB runs on top of the files, in our cloud paired with your bucket or entirely inside your account.

## Run in our cloud or yours

VideoDB is video infrastructure for agents and devices that see. 1,000+ developers and agents use it every week, and it has processed 300+ TB of video. Runs on AWS, Google Cloud and Azure, or your own cloud. SOC 2 Type II, GDPR and ISO 27001 compliant. Security: https://videodb.io/legal/security.md

## Start with one camera or one screen

Free to start, with $20 of credits and no credit card. For a fleet of feeds, tell us about them and we will set up a pilot with you. [Talk to us](https://videodb.io/contact.md) · [Add to Claude](https://videodb.io/claude) · [Add to ChatGPT](https://videodb.io/chatgpt)

## Use VideoDB in your agent

VideoDB lets an AI agent watch, search and clip video.

- **Claude**: [Add VideoDB from the plugin directory](https://videodb.io/claude)
- **ChatGPT**: [Add VideoDB from the plugin directory](https://chatgpt.com/plugins/videodb)
- **Claude Code**: `claude mcp add --transport http videodb https://mcp.videodb.io/mcp`
- **Cursor or any MCP client**: Add the remote MCP server and log in once: `https://mcp.videodb.io/mcp`

Free to start: $20 of credits, no credit card. [Steps for every client](https://docs.videodb.io/pages/mcp/connect).
