exec-mcp/MEMORY_FIXES_PLAN.md

Memory Leak Remediation Plan — Remaining Fixes

Applied fixes (this commit):

• Fix 1: SQLite connection leak — get_connection() now a contextmanager that closes on exit (storage.py)
• Fix 2: WS handler no longer blocks on balance snapshot — deferred to single-worker ThreadPoolExecutor (services_orders.py)

---

Fix 3 — WS reconnect backoff (bitstamp_private_ws.py)

Problem: private_ws_main retries with a fixed 5-second delay after any exception. If Bitstamp drops the connection repeatedly (rate limit, network blip), this creates a tight reconnect loop. Each reconnect calls list_accounts (opens a DB connection) and _get_token (HTTP call to Bitstamp), amplifying load.

Fix: Replace the fixed WS_RECONNECT_SECONDS sleep with exponential backoff:

• Start at 5s, double on each consecutive failure, cap at 60s.
• Reset to 5s after a successful connection that lasts > 30 seconds.
• Log the backoff delay so tight loops are visible in logs.

Location: private_ws_main and _run_once in bitstamp_private_ws.py.

---

Fix 4 — _CACHE_LOCKS unbounded growth (services_bitstamp.py)

Problem: _CACHE_LOCKS dict grows every time a new cache_key is used (per account, per market symbol, etc.). Locks are never removed, even when the account is deleted or the cache entry expires.

Fix options (pick one):

• A: Use weakref.WeakValueDictionary — locks get GC'd when no caller holds a reference. Simplest, but the with _cache_lock(cache_key): pattern keeps a strong reference during use, so the lock survives the call and is collected after.
• B: Add a _remove_cache_lock(cache_key) call in invalidate_account_cache and account deletion paths. More explicit but requires wiring up cleanup at every deletion site.

Location: _CACHE_LOCKS, _cache_lock in services_bitstamp.py:24-46.

---

Fix 5 — api_cache table eviction (repo.py / server.py)

Problem: cache_put does an upsert, so existing keys don't duplicate. But expired rows are never deleted — cache_get skips them, but they stay in the SQLite file. Over weeks, the table grows and the file bloats. Same applies to order_records and account_balance_snapshots (no retention policy).

Fix: Add a _evict_expired_cache() function in repo.py:

def evict_expired_cache() -> int:
    now = utc_now_iso()
    with get_connection() as conn:
        count = conn.execute("DELETE FROM api_cache WHERE expires_at <= ?", (now,)).rowcount
        conn.commit()
    return count

Call it from the existing metadata refresh loop (_metadata_refresh_loop in server.py) every cycle — it already runs every 24 hours, which is sufficient for cache eviction.

For order_records and account_balance_snapshots, consider a configurable retention (e.g., keep last 90 days, or last 10,000 rows per account). This is a bigger change and should be a separate discussion.

Location: repo.py, _metadata_refresh_loop in server.py.

---

Note — Price lookups must remain uncached

Price lookups via crypto_client.py must always return fresh data. Caching is handled upstream. Do not add a local TTL cache here.

---

Deployment & Monitoring

1. After deploying Fixes 1+2, monitor container RSS — should stabilize at 50-150 MB instead of climbing to GBs.
2. Check logs for "private websocket supervisor failed" frequency — if reconnects are frequent, Fix 3 is needed urgently.
3. Check SQLite file size (data/exec_mcp.sqlite3) — if growing, Fix 5 is needed.
4. Check docker stats for the container after 24-48 hours of running to confirm memory is flat.