Initial commit

PROJECT.md

@@ -0,0 +1,32 @@
+# Project: mem0-auto-capture hook
+
+## Aim
+Provide a standalone hook that keeps OpenClaw agents in sync with Mem0 by capturing meaningful turns, recalling personal and knowledge-base memories, and supporting audio transcripts.
+
+## Scope
+- Auto-capture incoming messages when triggers fire.
+- Auto-recall both personal memories and a dedicated knowledge-user scope before each response.
+- Work with local STT for audio, deduplicate rapid captures, and gracefully degrade when Mem0 is unreachable.
+- Ship as a repository-enable hook that anyone can install in their OpenClaw gateway.
+
+## Code layout
+- `hook/handler.ts` — business logic. Reads config from `MEM0_*` env vars or `~/.openclaw/mem0.json`. Handles received/preprocessed/transcribed events and rewrites `context.bodyForAgent` with injected memory blocks when appropriate.
+- `package.json` & `tsconfig.json` — minimal metadata for building and for the OpenClaw hook loader.
+
+## Configuration
+Defaults are documented in `README.md`. Keep a sample `~/.openclaw/mem0.json` near the gateway so other developers can copy it.
+
+## Deployment
+1. `npm install` (growth step, optional). 2. `openclaw hooks install --link /path/to/hook`. 3. `openclaw gateway restart`. 4. `openclaw hooks list` → hook should be `ready`.
+
+## Testing
+- Use `openclaw hooks simulate message` or create fixtures and hit `handler`. Ensure STT and Mem0 endpoints are mocked.
+- Run while connected to the actual Mem0 server at `http://192.168.0.200:8420` to validate capture/recall flows.
+
+## Release checklist
+- Update `package.json` version.
+- Document new env var defaults in README.
+- Tag a Git release (e.g., `v1.0.0`).
+- If releasing publicly, include a license (MIT/Apache/…?).
+
+Maintainer: Lukas Goldschmidt / GOLEM assistant (builds, tests, deploys).

README.md

@@ -0,0 +1,84 @@
+# mem0-auto-capture hook
+
+A self-contained OpenClaw hook that wires incoming messages to your Mem0 instance for auto-capture and auto-recall. It runs as a message hook inside the OpenClaw gateway, transcribes audio, decides what to store, queries memories/knowledge before the agent responds, and injects the results back into the prompt.
+
+## Key behaviors
+
+1. **Auto-Capture** — whenever a received message matches the configured trigger (default: every message or the phrase “please remember”), the hook gathers the most recent lines from that session (controlled by `recentKeep`) and POSTs them to `<baseUrl>/memories` with the resolved user id. It deduplicates captures by hashing the text and short-circuiting identical payloads within one minute.
+
+2. **Auto-Recall** — before the agent answers (on the `preprocessed` or `transcribed` event) it runs parallel searches against `<baseUrl>/memories/search` and `/knowledge/search`, filters them by rerank score, and appends the results as `[MEMORY - Personal]` and `[MEMORY - Knowledge Base]` sections inside `bodyForAgent` so the LLM has both long-term and knowledge-base context.
+
+3. **Audio support** — if the event includes media, the hook posts the file to the local STT service at `http://192.168.0.200:5005/transcribe`, caches transcripts, and uses them for both capture and recall. It also rewrites `bodyForAgent` so that audio messages appear as text when the agent eats them.
+
+## Configuration
+
+All options are read from environment variables (prefixed `MEM0_`) or a JSON file at `~/.openclaw/mem0.json`. Missing values fall back to sensible defaults.
+
+| Option | Source | Default | Notes |
+|---|---|---|---|
+| `baseUrl` | `MEM0_BASE_URL` or config file (`baseUrl`) | `http://192.168.0.200:8420` | The Mem0 server root. |
+| `userId` | `MEM0_USER_ID` or config file | derived from `sessionKey` → agent ID, then fallback to `userId` or `default` | Used for personal memories. |
+| `recallLimit` | `MEM0_RECALL_LIMIT` | `5` | How many personal memories to admit. |
+| `knowledgeLimit` | `MEM0_KNOWLEDGE_LIMIT` | `5` | How many knowledge-base hits to inject. |
+| `knowledgeUserId` | `MEM0_KNOWLEDGE_USER_ID` | `knowledge_base` | The user id that holds knowledge-base memories. |
+| `rerankThreshold` | `MEM0_RERANK_THRESHOLD` | `0.002` | Filters injected memories with low rerank scores. |
+| `captureTrigger` | `MEM0_CAPTURE_TRIGGER` | `always` | `always` / `phrase` / `explicit`. |
+| `triggerPhrase` | `MEM0_TRIGGER_PHRASE` | `please remember` | Used when `captureTrigger` is `phrase`. |
+| `autoCapture` | `MEM0_AUTO_CAPTURE` | `true` | Enable/disable captures. |
+| `autoRecall` | `MEM0_AUTO_RECALL` | `true` | Enable/disable recalls. |
+| `recentKeep` | `MEM0_RECENT_KEEP` | `5` | How many lines of recent context to bundle for a capture. |
+
+> Example `~/.openclaw/mem0.json`:
+>
+> ```json
+> {
+>   "baseUrl": "http://192.168.0.200:8420",
+>   "userId": "lukas",
+>   "captureTrigger": "phrase",
+>   "triggerPhrase": "please remember",
+>   "autoCapture": true,
+>   "autoRecall": true
+> }
+> ```
+
+## Deployment
+
+1. Build + install dependencies (TypeScript only needs dev-time tooling; the runtime runs the `.ts` file directly):
+
+   ```bash
+   cd mem0-auto-capture-hook
+   npm install
+   ```
+
+2. Link the hook into OpenClaw (this installs the compiled `hook/handler.ts` directory):
+
+   ```bash
+   openclaw hooks install --link $(pwd)/hook
+   ```
+
+3. Restart the gateway so it reloads hooks:
+
+   ```bash
+   openclaw gateway restart
+   ```
+
+4. Confirm via `openclaw hooks list` that the `mem0-auto-capture` hook is ready.
+
+## Testing & troubleshooting
+
+- Logs are prefixed with `[mem0-...]` (e.g., `[mem0-auto-capture]`, `[mem0-auto-recall]`, `[mem0-stt]`). If something silently fails, check `/tmp/openclaw/...` logs for those tags.
+- Use `openclaw hooks simulate message` (or your own integration tests) to feed the hook a fake event and watch the HTTP calls.
+- If the hook can’t reach Mem0, the recall/capture branches simply log errors and continue; the agent still runs but without those enhancements.
+
+## What’s in the repo
+
+- `hook/handler.ts` — the TypeScript entry point that implements capture, recall, STT, and caching.
+- `package.json` + `tsconfig.json` — minimal metadata so other developers can install dependencies and build if needed.
+- `README.md` & `PROJECT.md` — explain intent, configuration, and deployment.
+
+## Next steps
+- Tag releases (e.g., `v1.0.0`) and push to Git so you can `openclaw hooks install` directly from the repo.
+- Automate builds/tests if you plan to extend the hook (e.g., add unit tests around the Mem0 request helpers).
+- If you want the hook to share code with other Mem0 extensions, consider turning it into a proper plugin that registers both the hook and helper tools.
+
+Happy to help set up the Git repo or draft a release template—shall I do that next?

hook/handler.ts

@@ -0,0 +1,495 @@
+import fs from "fs";
+import os from "os";
+import path from "path";
+
+// ---------------------------------------------------------------------------
+// Config — openclaw does NOT inject cfg into hook events.
+// Read from env vars; fall back to ~/.openclaw/mem0.json for convenience.
+// ---------------------------------------------------------------------------
+function loadPluginCfg() {
+  const cfgPath = path.join(os.homedir(), ".openclaw", "mem0.json");
+  let fileCfg: Record<string, unknown> = {};
+  try {
+    if (fs.existsSync(cfgPath)) {
+      fileCfg = JSON.parse(fs.readFileSync(cfgPath, "utf8"));
+    }
+  } catch {
+    // ignore malformed file
+  }
+  return {
+    baseUrl:
+      process.env.MEM0_BASE_URL ||
+      (fileCfg.baseUrl as string) ||
+      "http://192.168.0.200:8420",
+    userId:
+      process.env.MEM0_USER_ID || (fileCfg.userId as string) || undefined,
+    recallLimit: Number(
+      process.env.MEM0_RECALL_LIMIT || fileCfg.recallLimit || 5
+    ),
+    captureTrigger: (
+      process.env.MEM0_CAPTURE_TRIGGER ||
+      (fileCfg.captureTrigger as string) ||
+      "always"
+    ) as "always" | "phrase" | "explicit",
+    triggerPhrase:
+      process.env.MEM0_TRIGGER_PHRASE ||
+      (fileCfg.triggerPhrase as string) ||
+      "please remember",
+    autoCapture:
+      (process.env.MEM0_AUTO_CAPTURE ||
+        String(fileCfg.autoCapture ?? "true")) !== "false",
+    autoRecall:
+      (process.env.MEM0_AUTO_RECALL ||
+        String(fileCfg.autoRecall ?? "true")) !== "false",
+    recentKeep: Number(
+      process.env.MEM0_RECENT_KEEP || fileCfg.recentKeep || 5
+    ),
+    // Knowledge-base settings
+    knowledgeUserId:
+      process.env.MEM0_KNOWLEDGE_USER_ID ||
+      (fileCfg.knowledgeUserId as string) ||
+      "knowledge_base",
+    rerankThreshold: Number(
+      process.env.MEM0_RERANK_THRESHOLD || fileCfg.rerankThreshold || 0.002
+    ),
+    knowledgeLimit: Number(
+      process.env.MEM0_KNOWLEDGE_LIMIT || fileCfg.knowledgeLimit || 5
+    ),
+  };
+}
+
+// ---------------------------------------------------------------------------
+// Types
+// ---------------------------------------------------------------------------
+type HookEvent = {
+  type: string;
+  action: string;
+  sessionKey: string;
+  timestamp: Date;
+  messages: string[] | any[];
+  context: any;
+};
+
+interface MemoryResult {
+  id: string;
+  memory: string;
+  score: number;
+  rerank_score: number;
+  metadata?: Record<string, any>;
+}
+
+interface SearchResponse {
+  results: MemoryResult[];
+}
+
+// ---------------------------------------------------------------------------
+// In-memory state — capped to avoid unbounded growth
+// ---------------------------------------------------------------------------
+const MAX_SESSIONS = 500;
+const MAX_TRANSCRIPTS = 1000;
+
+class LRUMap<K, V> extends Map<K, V> {
+  private readonly maxSize: number;
+  constructor(maxSize: number) {
+    super();
+    this.maxSize = maxSize;
+  }
+  set(key: K, value: V): this {
+    if (this.size >= this.maxSize && !this.has(key)) {
+      this.delete(this.keys().next().value!);
+    }
+    return super.set(key, value);
+  }
+}
+
+const recentBySession = new LRUMap<string, string[]>(MAX_SESSIONS);
+const transcriptByMessageId = new LRUMap<string, string>(MAX_TRANSCRIPTS);
+
+// ---------------------------------------------------------------------------
+// Helpers
+// ---------------------------------------------------------------------------
+const LOCAL_STT_URL = "http://192.168.0.200:5005/transcribe";
+
+function getAgentIdFromSessionKey(sessionKey?: string): string | undefined {
+  if (!sessionKey) return undefined;
+  const parts = sessionKey.split(":");
+  if (parts.length >= 2 && parts[0] === "agent") return parts[1];
+  console.warn("[mem0] unexpected sessionKey format:", sessionKey);
+  return undefined;
+}
+
+function pushRecent(
+  sessionKey: string,
+  text: string,
+  keep: number
+): string[] {
+  const list = recentBySession.get(sessionKey) || [];
+  list.push(text);
+  while (list.length > keep) list.shift();
+  recentBySession.set(sessionKey, list);
+  return list;
+}
+
+function isMediaPlaceholder(text: string): boolean {
+  return /<media:[^>]+>/i.test(text);
+}
+
+function extractMessageText(context: any): string {
+  const transcript =
+    typeof context?.transcript === "string" ? context.transcript.trim() : "";
+  if (transcript) return transcript;
+  const candidates = [context?.bodyForAgent, context?.content, context?.body];
+  for (const candidate of candidates) {
+    if (typeof candidate === "string") {
+      const trimmed = candidate.trim();
+      if (trimmed && !isMediaPlaceholder(trimmed)) return trimmed;
+    }
+  }
+  return "";
+}
+
+function getAudioPath(context: any): string | undefined {
+  if (typeof context?.mediaPath === "string") return context.mediaPath;
+  if (
+    Array.isArray(context?.mediaPaths) &&
+    typeof context.mediaPaths[0] === "string"
+  ) {
+    return context.mediaPaths[0];
+  }
+  if (typeof context?.media?.path === "string") return context.media.path;
+  return undefined;
+}
+
+async function transcribeAudio(localPath: string): Promise<string> {
+  const buffer = fs.readFileSync(localPath);
+  const blob = new Blob([buffer]);
+  const form = new FormData();
+  form.append("file", blob, "audio.ogg");
+  const res = await fetch(LOCAL_STT_URL, { method: "POST", body: form });
+  if (!res.ok) throw new Error(`STT failed: ${res.status}`);
+  const data = await res.json();
+  const text =
+    typeof data?.text === "string" ? data.text.trim() : "";
+  if (!text) throw new Error("STT returned empty transcript");
+  return text;
+}
+
+// ---------------------------------------------------------------------------
+// mem0 search helpers — each returns null on failure so callers can degrade
+// gracefully when one endpoint is down.
+// ---------------------------------------------------------------------------
+async function mem0SearchMemories(
+  baseUrl: string,
+  userId: string,
+  query: string,
+  limit: number
+): Promise<MemoryResult[] | null> {
+  try {
+    const res = await fetch(`${baseUrl}/memories/search`, {
+      method: "POST",
+      headers: { "Content-Type": "application/json" },
+      body: JSON.stringify({ query, userId }),
+    });
+    if (!res.ok) {
+      console.error(`[mem0-recall] /memories/search returned ${res.status}`);
+      return null;
+    }
+    const data: SearchResponse = await res.json();
+    return Array.isArray(data?.results) ? data.results.slice(0, limit) : [];
+  } catch (err) {
+    console.error("[mem0-recall] /memories/search failed:", err);
+    return null;
+  }
+}
+
+async function mem0SearchKnowledge(
+  baseUrl: string,
+  knowledgeUserId: string,
+  query: string,
+  limit: number
+): Promise<MemoryResult[] | null> {
+  try {
+    const res = await fetch(`${baseUrl}/knowledge/search`, {
+      method: "POST",
+      headers: { "Content-Type": "application/json" },
+      body: JSON.stringify({ query, userId: knowledgeUserId }),
+    });
+    if (!res.ok) {
+      console.error(`[mem0-recall] /knowledge/search returned ${res.status}`);
+      return null;
+    }
+    const data: SearchResponse = await res.json();
+    return Array.isArray(data?.results) ? data.results.slice(0, limit) : [];
+  } catch (err) {
+    console.error("[mem0-recall] /knowledge/search failed:", err);
+    return null;
+  }
+}
+
+// ---------------------------------------------------------------------------
+// Result filtering & formatting
+// ---------------------------------------------------------------------------
+function filterByRerank(
+  results: MemoryResult[],
+  threshold: number
+): MemoryResult[] {
+  return results.filter(
+    (r) => typeof r.rerank_score === "number" && r.rerank_score >= threshold
+  );
+}
+
+function formatKnowledgeCitation(meta: Record<string, any> = {}): string {
+  const parts: string[] = [];
+  if (meta.source_file) parts.push(meta.source_file);
+  if (meta.chapter != null) parts.push(`ch.${meta.chapter}`);
+  if (meta.page_start != null && meta.page_end != null) {
+    parts.push(`pp.${meta.page_start}-${meta.page_end}`);
+  } else if (meta.page_start != null) {
+    parts.push(`p.${meta.page_start}`);
+  }
+  return parts.length > 0 ? `(from: ${parts.join(", ")})` : "";
+}
+
+function buildInjectionBlock(
+  personalResults: MemoryResult[] | null,
+  knowledgeResults: MemoryResult[] | null,
+  threshold: number
+): string {
+  const sections: string[] = [];
+
+  // Personal memories
+  const personal = personalResults
+    ? filterByRerank(personalResults, threshold)
+    : null;
+
+  if (personal === null) {
+    // endpoint was down — omit the section entirely, already logged
+  } else if (personal.length > 0) {
+    const lines = personal
+      .map((r) => `- ${r.memory}`)
+      .join("\n");
+    sections.push(`[MEMORY - Personal]\n${lines}`);
+  }
+
+  // Knowledge base — sort by rerank_score descending
+  const knowledge = knowledgeResults
+    ? filterByRerank(knowledgeResults, threshold).sort(
+        (a, b) => b.rerank_score - a.rerank_score
+      )
+    : null;
+
+  if (knowledge === null) {
+    // endpoint was down — omit the section entirely, already logged
+  } else if (knowledge.length > 0) {
+    const lines = knowledge
+      .map((r) => {
+        const citation = formatKnowledgeCitation(r.metadata || {});
+        return citation ? `- ${citation} ${r.memory}` : `- ${r.memory}`;
+      })
+      .join("\n");
+    sections.push(`[MEMORY - Knowledge Base]\n${lines}`);
+  }
+
+  return sections.join("\n\n");
+}
+
+// Build a quick hash to deduplicate captures
+function simpleHash(text: string): string {
+  let h = 0;
+  for (let i = 0; i < text.length; i++) {
+    h = (Math.imul(31, h) + text.charCodeAt(i)) | 0;
+  }
+  return h.toString(36);
+}
+
+const recentlyCaptured = new LRUMap<string, number>(200);
+const CAPTURE_DEDUP_MS = 60_000;
+
+// ---------------------------------------------------------------------------
+// Main handler
+// ---------------------------------------------------------------------------
+export default async function handler(event: HookEvent) {
+  console.log(
+    "[mem0-FIRE]",
+    JSON.stringify(
+      {
+        type: event.type,
+        action: event.action,
+        sessionKey: event.sessionKey,
+        contextKeys: Object.keys(event.context || {}),
+        content: event.context?.content?.slice(0, 80),
+        messagesLen: event.messages?.length,
+      },
+      null,
+      2
+    )
+  );
+
+  if (event.type !== "message") {
+    console.log("[mem0-FIRE] bailed: type is not message, got:", event.type);
+    return;
+  }
+
+  const pluginCfg = loadPluginCfg();
+  const userId =
+    getAgentIdFromSessionKey(event.sessionKey) ||
+    pluginCfg.userId ||
+    "default";
+
+  // ── Audio transcription (runs once per messageId) ────────────────────────
+  const messageId =
+    event.context?.messageId || event.context?.metadata?.messageId;
+  const audioPath = getAudioPath(event.context);
+  const hasTranscript =
+    typeof event.context?.transcript === "string" &&
+    event.context.transcript.trim().length > 0;
+
+  if (!hasTranscript && audioPath && fs.existsSync(audioPath)) {
+    if (messageId && transcriptByMessageId.has(messageId)) {
+      event.context.transcript = transcriptByMessageId.get(messageId);
+    } else {
+      try {
+        const text = await transcribeAudio(audioPath);
+        event.context.transcript = text;
+        if (messageId) transcriptByMessageId.set(messageId, text);
+      } catch (err) {
+        console.error("[mem0-stt] failed:", err);
+        if (
+          isMediaPlaceholder(
+            event.context?.bodyForAgent ||
+              event.context?.content ||
+              ""
+          )
+        ) {
+          return;
+        }
+      }
+    }
+  }
+
+  // Patch bodyForAgent with transcript for audio messages
+  const transcriptNow =
+    typeof event.context?.transcript === "string"
+      ? event.context.transcript.trim()
+      : "";
+  if (transcriptNow && isMediaPlaceholder(event.context?.bodyForAgent || "")) {
+    event.context.bodyForAgent = transcriptNow;
+  }
+
+  // ── Extract text (shared by both branches) ───────────────────────────────
+  function resolveText(): string {
+    let text = extractMessageText(event.context);
+    if (!text && Array.isArray(event.messages) && event.messages.length > 0) {
+      const m = event.messages[0];
+      if (typeof m === "string") text = m.trim();
+      else if (typeof m?.text === "string") text = m.text.trim();
+      else if (typeof m?.body === "string") text = m.body.trim();
+    }
+    return text;
+  }
+
+  // ── Auto-recall: queries both endpoints in parallel ───────────────────────
+  const runAutoRecall = async (text: string) => {
+    if (!pluginCfg.autoRecall) return;
+
+    const {
+      baseUrl,
+      knowledgeUserId,
+      recallLimit,
+      knowledgeLimit,
+      rerankThreshold,
+    } = pluginCfg;
+
+    console.log("[mem0-auto-recall] query:", text.slice(0, 120));
+
+    // Fire both searches in parallel — neither blocks the other
+    const [personalResults, knowledgeResults] = await Promise.all([
+      mem0SearchMemories(baseUrl, userId, text, recallLimit),
+      mem0SearchKnowledge(baseUrl, knowledgeUserId, text, knowledgeLimit),
+    ]);
+
+    console.log("[mem0-auto-recall]", {
+      userId,
+      personalCount: personalResults?.length ?? "error",
+      knowledgeCount: knowledgeResults?.length ?? "error",
+      threshold: rerankThreshold,
+    });
+
+    const injectionBlock = buildInjectionBlock(
+      personalResults,
+      knowledgeResults,
+      rerankThreshold
+    );
+
+    if (!injectionBlock) return; // nothing passed the threshold from either endpoint
+
+    event.context.bodyForAgent = `${text}\n\n${injectionBlock}`;
+    console.log("[mem0-injected-prompt]\n" + event.context.bodyForAgent);
+  };
+
+  // ── received: capture ────────────────────────────────────────────────────
+  if (event.action === "received") {
+    if (!pluginCfg.autoCapture) return;
+    const text = resolveText();
+    if (!text) return;
+    if (isMediaPlaceholder(text) && !transcriptNow) return;
+
+    const recent = pushRecent(
+      event.sessionKey || "global",
+      text,
+      pluginCfg.recentKeep
+    );
+
+    const { captureTrigger, triggerPhrase } = pluginCfg;
+    let shouldCapture = false;
+    if (captureTrigger === "always") {
+      shouldCapture = true;
+    } else if (captureTrigger === "phrase") {
+      shouldCapture = new RegExp(triggerPhrase, "i").test(text);
+    } else {
+      shouldCapture = /please\s+remember/i.test(text);
+    }
+    if (!shouldCapture) return;
+
+    const captureText = recent.join("\n");
+    const hash = simpleHash(captureText);
+    const lastCapture = recentlyCaptured.get(hash);
+    if (lastCapture && Date.now() - lastCapture < CAPTURE_DEDUP_MS) {
+      console.log("[mem0-auto-capture] skipped duplicate");
+      return;
+    }
+    recentlyCaptured.set(hash, Date.now());
+
+    try {
+      console.log("[mem0-auto-capture]", {
+        userId,
+        captureTrigger,
+        text: captureText.slice(0, 160),
+      });
+      await fetch(`${pluginCfg.baseUrl}/memories`, {
+        method: "POST",
+        headers: { "Content-Type": "application/json" },
+        body: JSON.stringify({ text: captureText, userId }),
+      });
+    } catch (err) {
+      console.error("[mem0-auto-capture] write failed:", err);
+    }
+    return;
+  }
+
+  // ── preprocessed: recall ─────────────────────────────────────────────────
+  if (event.action === "preprocessed") {
+    const text = resolveText();
+    if (!text) return;
+    if (!transcriptNow && isMediaPlaceholder(text)) return;
+    await runAutoRecall(text);
+    return;
+  }
+
+  // ── transcribed: recall ──────────────────────────────────────────────────
+  if (event.action === "transcribed") {
+    const text = extractMessageText(event.context);
+    if (!text) return;
+    await runAutoRecall(text);
+  }
+}

package.json

@@ -0,0 +1,12 @@
+{
+  "name": "mem0-auto-capture",
+  "version": "1.0.0",
+  "type": "module",
+  "devDependencies": {
+    "@types/node": "^25.3.5",
+    "typescript": "^5.9.3"
+  },
+  "openclaw": {
+    "hooks": ["./hook"]
+  }
+}

tsconfig.json

@@ -0,0 +1,14 @@
+{
+  "compilerOptions": {
+    "target": "ES2020",
+    "module": "NodeNext",
+    "moduleResolution": "NodeNext",
+    "outDir": ".",
+    "rootDir": ".",
+    "strict": true,
+    "esModuleInterop": true,
+    "skipLibCheck": true
+  },
+  "include": ["handler.ts"],
+  "exclude": ["node_modules", "dist"]
+}