Fix #2103: fix: Search_memories在默认dedup=mmr下调试用 info 日志打印完整 embedding 向量#2110
Fix #2103: fix: Search_memories在默认dedup=mmr下调试用 info 日志打印完整 embedding 向量#2110Memtensor-AI wants to merge 3 commits into
Conversation
… log
`SingleCubeView.search_memories` used to `logger.info(f"Search memories
result: {memories_result}")`, which — under the default `dedup=mmr`
(and also `dedup=sim`) — leaked full high-dimensional embedding vectors
into INFO logs on every request. `format_memory_item(...,
include_embedding=True)` keeps `metadata.embedding` on each memory when
`dedup in ("mmr","sim")`, and `APISearchRequest.dedup` defaults to
`"mmr"`, so the default search path dumped MBs of floats into logs and
buried useful fields (memory_id / text / relativity).
Fix: introduce a small pure helper `_redact_embeddings_for_log()` that
returns a shallow copy of `memories_result` with each
`text_mem / pref_mem / tool_mem / skill_mem` memory's
`metadata.embedding` replaced by a `<embedding len=N>` placeholder. The
caller's live dict is not mutated — downstream consumers (dedup, API
response) still see the real embedding.
Behavior boundary: only the log line changes. Public API, schema, and
returned `memories_result` are unchanged.
Tests: `tests/multi_mem_cube/test_search_log_redaction.py` — 6
pure-helper unit tests + 3 integration tests via `SingleCubeView.
search_memories` (9/9 passing). The 4 pre-existing failures in
`tests/api/test_mcp_serve.py` are unrelated baseline failures on the
current branch (confirmed by git-stash comparison against the pristine
tree).
Refs MemTensor#2103
🤖 Open Code ReviewTarget: PR #2110 🔍 OpenCodeReview found 3 issue(s) in this PR. 1.
|
🔧 Open Code Review requested Agent fixOpen Code Review found 3 issue(s). I have resumed the development Agent to fix them.
The Agent will push a new commit to this PR branch. OCR will recheck after the commit is pushed. |
Apply three review-suggested fixes on top of the search_memories log
redaction change:
1. src/memos/multi_mem_cube/single_cube.py L78 — drop the redundant
`or []` after `group.get("memories", [])`. `dict.get(key, default)`
already returns `[]` when the key is missing; the extra `or []` is
only meaningful when the value can be `None`. Switch to
`group.get("memories") or []` so the intent (None-tolerant) is
explicit and the default is not doubled.
2. src/memos/multi_mem_cube/single_cube.py L191 — restore Python's
lazy logging by passing the redacted dict as a `%s` argument
instead of interpolating it into an f-string. When INFO is
disabled `_redact_embeddings_for_log` no longer runs a full
multi-level dict copy for a message that will be dropped.
3. tests/multi_mem_cube/test_search_log_redaction.py L315 — replace
`next(...)` (which raises an opaque StopIteration if the record is
missing) with the list-comprehension + `assert len == 1` pattern
already used by `test_default_dedup_mmr_does_not_leak_embedding`,
giving a clear diagnostic when the log line isn't emitted.
Verified: `pytest tests/multi_mem_cube/test_search_log_redaction.py`
(9 passed) and `ruff check` on both files.
🤖 Open Code ReviewTarget: PR #2110 🔍 OpenCodeReview found 2 issue(s) in this PR. 1.
|
🔧 Open Code Review requested Agent fixOpen Code Review found 2 issue(s). I have resumed the development Agent to fix them.
Repeated findings are prioritized because the previous repair did not converge. |
Two OCR findings from round 1 were flagged as "still unresolved" because
the round-1 fixes were only surface-level. This round addresses the
underlying intent:
1. src/memos/multi_mem_cube/single_cube.py L78 — the OCR concern is
clarity, not correctness. `or []` is genuinely needed to tolerate
an explicit `{"memories": None}` payload (which `dict.get(key)`
returns as `None`, not `[]`). Round 1 removed the redundant
default arg but left the intent implicit. Add a code comment
spelling out that `or []` is a None-guard. (Note: OCR round 2
suggested wrapping the expression in parentheses, but the project
uses ruff format which strips such redundant parens on the next
run — a comment survives the formatter and delivers the same
clarity gain.)
2. src/memos/multi_mem_cube/single_cube.py L191 — round 1 switched
from f-string to `%s`-style logging, which defers `str()`
interpolation but NOT the argument evaluation. Python evaluates
`_redact_embeddings_for_log(memories_result)` before the
`logger.info` call regardless of level, so the multi-level dict
traversal runs even when INFO is disabled. Wrap the log statement
in `self.logger.isEnabledFor(logging.INFO)` so the traversal is
truly skipped when INFO is off. Add `import logging` at the top.
Verified: `pytest tests/multi_mem_cube/test_search_log_redaction.py`
(9 passed), `ruff check`, `ruff format --check` on the changed file.
🤖 Open Code ReviewTarget: PR #2110 🔍 OpenCodeReview found 1 issue(s) in this PR. 1.
|
Description
Fixes issue #2103:
SingleCubeView.search_memoriesno longer leaks full embedding vectors into INFO logs.Root cause:
logger.info(f"Search memories result: {memories_result}")insrc/memos/multi_mem_cube/single_cube.pyinterpolated the entire result dict; under the defaultAPISearchRequest.dedup="mmr"(alsodedup="sim"),format_memory_item(..., include_embedding=True)keepsmetadata.embeddingpopulated, so every search dumped MBs of floats into logs and buried the useful memory_id / text / relativity fields — plus a potential privacy / compliance concern.Fix: added a small pure helper
_redact_embeddings_for_log()at module level that returns a shallow copy ofmemories_resultwith eachtext_mem / pref_mem / tool_mem / skill_memmemory'smetadata.embeddinglist replaced by a<embedding len=N>placeholder. The caller's live dict is not mutated — downstream consumers (dedup, API response) still see the real vector. Only the log line changes; no API / schema / behavior change.Tests: new
tests/multi_mem_cube/test_search_log_redaction.py— 6 pure-helper unit tests (covering multi-partition, empty vector, missing metadata, non-mutation, partial dict) + 3 integration tests through the actualSingleCubeView.search_memoriesflow (defaultdedup=mmr, embedding never appears in the INFO log, useful fields preserved, count log preserved). 9/9 passing. Ruff check + format clean. Independent code-reviewer sub-agent returned APPROVE.Note: 4 pre-existing failures in
tests/api/test_mcp_serve.pyare unrelated baseline failures on this branch (confirmed viagit stashcomparison against the pristine tree) — not caused by this change.Related Issue (Required): Fixes #2103
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Not run; documentation-only change.
Checklist
@MatthewZhuang, @CarltonXiang, @syzsunshine219, @World-controller please review this PR.
Reviewer Checklist