I’m building a personal assistant agent that needs to remember user preferences over weeks of interaction. Standard buffer memory gets too expensive and loses focus. Should I be using a Vector Database for 'Long-Term Memory' or is there a better way to summarize past interactions without losing key details?
3 answers
For long-running sessions, a hybrid memory approach is the gold standard. You should use a "Conversation Summary Buffer" for immediate context (the last 5-10 exchanges) and a Vector Database (like Pinecone or Weaviate) for long-term storage. Whenever the agent finishes a session, have a separate "Summarizer" agent extract "Key Entities" and "User Preferences" into a structured JSON format. Then, at the start of a new session, do a semantic search on the Vector DB to retrieve only the relevant memories. This prevents the "distraction" caused by loading thousands of irrelevant tokens from past chats that have nothing to do with the current request.
Does the Vector DB approach actually preserve the "tone" of the user, or do you find the agent becomes too robotic when it's just reading summarized facts from a database?
Look into "Entity Memory." Instead of storing the whole chat, you just store a knowledge graph of the user's world. It's much more efficient than a Vector DB for pure facts.
I agree with David. Knowledge graphs are significantly better for maintaining relationships between different pieces of information over long periods without the noise of vector similarity.
Charles, that's a subtle but important point. To answer you, we actually store "Raw Snippets" alongside the summaries in the Vector DB. When the agent retrieves a memory, it gets the fact (e.g., "User likes dark mode") plus a raw quote of how the user said it. This allows the agent to maintain a consistent persona. It’s like having a "Briefing Note" that includes both the data and the context. This dual-layer approach has kept our user retention rates high because the agent feels like it actually "knows" the person.