Skip to content

fix(chat): surface unavailable memory at startup - #3153

Merged
itomek merged 2 commits into
amd:mainfrom
mikemikimike:pr-2831
Sep 2, 2026
Merged

fix(chat): surface unavailable memory at startup#3153
itomek merged 2 commits into
amd:mainfrom
mikemikimike:pr-2831

Conversation

@mikemikimike

@mikemikimike mikemikimike commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

ChatAgent currently logs embedding initialization failures but does not show them in the chat UI, so every MemoryMixin-based chat session can silently lose memory. This change reports model-not-pulled and service-unreachable states through the existing console warning channel after the console is initialized, while preserving the explicit GAIA_MEMORY_DISABLED opt-out. Added a deterministic unit test for the warning boundary.

Tested: PYTHONPATH=src;hub/agents/chat/python python -m pytest hub/agents/chat/python/tests/test_chat_agent.py -q (8 passed); python -m black --check on changed files; git diff --check.

Refs #2831

@github-actions

Copy link
Copy Markdown
Contributor

Verdict: Request changes

This makes a chat session say out loud when memory failed to come up instead of only whispering it to a log file — the right fix for a real p1. But as written the message never reaches the two places the linked issue actually complains about.

The warning is printed before anyone is listening. In the Agent UI and in the flagship's TUI sidecar, the agent is built in "quiet" mode and only later handed the channel that carries text to the user. The new warning fires during construction, into the quiet channel, and is discarded. Only the interactive terminal gaia chat session will ever show it — and issue #2831 specifically calls out "no indication in the TUI". Remember the message at startup and emit it the first time the agent actually answers a turn (by which point the real output channel is attached), rather than at construction.

No evidence that a user sees the warning. The PR shows a passing unit test on the helper function, which proves the helper calls the console — not that a person with a missing embedding model gets told. A short paste of a real gaia chat startup with the embedding model absent would settle it, and the same run would have exposed the problem above.

Real-world evidence

N/A — no evidence bundle was produced for this run, and the PR description lists only unit tests plus formatting checks. The verdict rests on static review of the startup ordering; nothing here demonstrates the warning reaching a user on any surface.

🔍 Technical details

🔴 The warning is swallowed on every non-terminal surface (hub/agents/chat/python/gaia_agent_chat/agent.py:453)

_report_memory_unavailable(self) runs at the end of ChatAgent.__init__, so it uses whatever self.console is at construction time. Every UI/sidecar caller builds the agent with silent_mode=True and no output_handler, which makes self.console a SilentConsole (src/gaia/agents/base/agent.py:792_create_console, console.py:2525); SilentConsole.print_warning is a no-op. The real handler is attached afterwards:

  • src/gaia/ui/agent_loop.py:416 and src/gaia/ui/_chat_helpers.py:1852,1935,2081agent.console = sse_handler after ChatAgent(config)
  • hub/agents/gaia/python/gaia_agent/stdio.py:947agent.console = handler per query; the agent itself is built with silent_mode=True at stdio.py:1144
  • hub/agents/gaia/python/gaia_agent/{server.py:334,session_registry.py:47} — same silent_mode=True

Net effect: only gaia chat interactive (src/gaia/cli.py:693) and gaia_agent_chat/app.py:1008 with silent mode off actually print it. #2831's reported surface (the TUI) still shows nothing.

Suggested shape: record the pending message in __init__, flush it once from the query entry point after the console swap, e.g.

self._pending_memory_warning = (
    self.memory_unavailable_message()
    if getattr(self, "_memory_unavailable_reason", None) in (...)
    else None
)

then in process_query/_run_turn: if self._pending_memory_warning: self.console.print_warning(self._pending_memory_warning); self._pending_memory_warning = None. That works for both the construction-time console and the later-injected SSE/wire handler.

🟡 No real-surface evidence for a user-visible change

Per the rubric, a change to what a chat session prints wants the real gaia <subcommand> output — here, gaia chat started against a Lemonade with the embedding model unpulled, showing the warning banner. The unit test asserts the helper's plumbing only.

🟢 Duplicates the email agent's block instead of sharing it (agent.py:71)

EmailTriageAgent already carries the identical reason-filter + print_warning pair (hub/agents/email/python/gaia_agent_email/agent.py:986-990). Two copies of the same policy will drift the next time a reason constant is added. Better home is MemoryMixin next to memory_unavailable_message() (src/gaia/agents/base/memory.py:787) — e.g. report_memory_unavailable(self.console) — with both agents calling it.

🟢 Test covers only the reporting branch (hub/agents/chat/python/tests/test_chat_agent.py:62)

The skip path is the part with the actual policy in it (disabled_by_env and "no reason" must stay quiet), and it's untested. Also, console is a class attribute on the fake, so the assertion reads off the class rather than the instance — fine here, but it hides an instance/class mix-up if the helper ever changes.

def test_memory_failure_is_reported_to_chat_console():
    from gaia_agent_chat.agent import _report_memory_unavailable
    from gaia.agents.base.memory import (
        MEMORY_UNAVAILABLE_DISABLED_BY_ENV,
        MEMORY_UNAVAILABLE_MODEL_NOT_PULLED,
    )

    class Console:
        def __init__(self):
            self.messages = []

        def print_warning(self, message):
            self.messages.append(message)

    class FakeAgent:
        def __init__(self, reason):
            self._memory_unavailable_reason = reason
            self.console = Console()

        def memory_unavailable_message(self):
            return "Embedding model is unavailable; pull it and restart."

    reported = FakeAgent(MEMORY_UNAVAILABLE_MODEL_NOT_PULLED)
    _report_memory_unavailable(reported)
    assert reported.console.messages == [
        "Embedding model is unavailable; pull it and restart."
    ]

    # The explicit opt-out and a healthy store must stay quiet.
    for reason in (MEMORY_UNAVAILABLE_DISABLED_BY_ENV, None):
        quiet = FakeAgent(reason)
        _report_memory_unavailable(quiet)
        assert quiet.console.messages == []

Strengths

  • Correctly excludes MEMORY_UNAVAILABLE_DISABLED_BY_ENV, so the deliberate GAIA_MEMORY_DISABLED=1 opt-out (used by CI) doesn't nag — that distinction is easy to miss.
  • Reuses memory_unavailable_message() rather than re-deriving remedy text, so the model-not-pulled vs. service-down remedies stay distinct.
  • Small, well-scoped diff with a comment that explains the ordering constraint (memory init before Agent.__init__) rather than restating the code.

@mikemikimike

Copy link
Copy Markdown
Contributor Author

Addressed the review feedback in 56edcd0. MemoryMixin now owns one-shot reporting, ChatAgent defers the warning until the first user-visible turn (after TUI/SSE console injection), and EmailAgent reuses the shared path. Tests now cover one-shot behavior plus explicit GAIA_MEMORY_DISABLED/healthy quiet paths.

@github-actions github-actions Bot added agents agent::email Email agent changes labels Aug 30, 2026
@itomek itomek self-assigned this Aug 31, 2026

@itomek itomek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved this earlier, then went back and traced the turn hook — the email agent, which is the agent #2831 was actually filed against, still never shows this warning to anyone.

Moving the message off construction and onto the first real turn is the right call, and it works for chat and the flagship. But the email agent's own process_query hands off to a base implementation that stops there, so the new turn-start report is unreachable code for it. Its only remaining report is the one at construction time — the exact path the earlier review asked to move away from. On the email surface the behaviour is unchanged from before this PR: warning in the log, nothing on screen.

Second one: the report marks itself "already shown" even when it printed into a console that discards output. A session whose first turn goes through the non-streaming path spends the single shot on nothing, and every later turn in that session stays quiet.

Both are small — roughly fifteen lines. The test matters as much as the fix: the current one exercises the helper against a hand-written stub, so it passes unchanged with both of these present. There are already 28 email tests that build a real agent, so driving one through a turn and asserting the warning fires is cheap, and it's what would have caught this.

Last thing — I'd drop Fixes #2831 to Refs #2831 for now. As it stands a merge auto-closes a p1 whose "covers every MemoryMixin consumer" criterion this doesn't yet meet, plus two it doesn't touch (whether running without memory is permitted at all, and whether to accept an available embedding model rather than failing on an exact-id miss). Happy for those to be follow-ups, just not silently closed.

I'll get you a real run against a Lemonade with the embedding model unpulled — that evidence is easier to produce on my side, and it's the check that would have surfaced this.

🔍 Technical details

Email agent: the deferred report is unreachable

process_query defined in, in MRO order: ['EmailTriageAgent', 'Agent', 'MemoryMixin']
EmailTriageAgent -> calls super().process_query: True
Agent            -> calls super().process_query: False   <- chain stops here
MemoryMixin      -> calls super().process_query: True    <- never reached

EmailTriageAgent(Agent, MemoryMixin, ...) puts Agent ahead of MemoryMixin, and Agent.process_query (src/gaia/agents/base/agent.py:4426) calls _process_query_impl without delegating further. Chat and the flagship both resolve process_query to MemoryMixin.process_query, so they're fine.

No need to reorder the bases — just call it directly in hub/agents/email/python/gaia_agent_email/agent.py:1221:

    def process_query(self, user_input: str, *args, **kwargs):
        # UI/sidecar callers swap in the real console after __init__ —
        # report through whatever handler is attached now.
        self.report_memory_unavailable()
        self._reset_organize_counter()

One-shot flag spent on a console that renders nothing

  • SilentConsole.print_warning is a no-op (src/gaia/agents/base/console.py:2701)
  • agents are built with silent_mode = not streaming (src/gaia/ui/_chat_helpers.py:494)
  • the non-streaming process_query at :1607 has no agent.console = assignment before it (injections are at :1852, :1935, :2081)
  • both paths share one _agent_cache (lookups at :1427 non-streaming, :1847 streaming)

So: first turn non-streaming → flag set against a no-op console → every later streaming turn on that cached agent is silent. In report_memory_unavailable:

        target = console if console is not None else getattr(self, "console", None)
        if target is None or isinstance(target, SilentConsole):
            return False

console.py doesn't import memory, so that import is cycle-free.

Minorreport_memory_unavailable(console=None): no caller passes it. Drop the parameter or use it from the email agent.

Follow-up, not a blocker — in the Agent UI the warning arrives as a status event, and ChatView.tsx:995 consolidates consecutive status steps by overwriting the previous one's label and detail. So it flickers in the steps panel rather than persisting as the degraded-state indicator #2831 asks for. Read from the code, not observed live.

Worth keeping

  • Chat and flagship path is correct and verified — SSEOutputHandler.print_warning emits a real status/warning event (src/gaia/ui/sse_handler.py:502), so the injected handler does carry it.
  • Excluding GAIA_MEMORY_DISABLED from the warning is right, and easy to get wrong.
  • Consolidating the policy into MemoryMixin instead of a second copy in the email agent is the correct home.

…eal console

EmailTriageAgent's MRO puts Agent ahead of MemoryMixin, so Agent.process_query
never delegates into MemoryMixin.process_query the way ChatAgent does — the
turn-start notice added for the degraded-memory warning was unreachable code
for the email agent. It now reports directly from
EmailTriageAgent.process_query, so a UI/sidecar caller that swaps in the real
console after construction (agent.console = ...) still gets the notice on the
next turn.

Separately, report_memory_unavailable() was spending its one-shot flag on a
SilentConsole, whose print_warning is a no-op — the UI's non-streaming path
builds agents silent, and the same cached agent later serves streaming turns
with a real console that would never see the notice. It now skips (and does
not consume) the one-shot report when the current console is silent. Also
dropped the unused console= parameter, which no caller ever passed.
@itomek

itomek commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Pushed the two fixes from my last review straight to your branch rather than bounce it back — you'd already turned this around once, and it seemed silly to hold it another cycle over fifteen lines. Nothing needed from you. Shout if you'd rather have done it yourself and I'll know for next time.

The email agent now reports the degraded-memory notice from its own turn entry point instead of relying on the shared one it never actually reaches, and reporting into a console that renders nothing no longer burns the one-shot flag. I also added two tests that build a real email agent and drive it through a turn — both fail on your previous head and pass now, which is the coverage the earlier stub-based test couldn't give.

Also dropped Fixes #2831 to Refs #2831, so merging this doesn't auto-close a p1 whose remaining criteria it doesn't cover yet. Those are fine as follow-ups, I just didn't want them closed silently.

Still on me: the real run against a Lemonade with the embedding model unpulled. Once I post that, this is good to go.

🔍 Technical details

Pushed as 015d3313 on top of your 56edcd09 — fast-forward, no rebase or amend, your history is untouched.

1. Email agent — the deferred report was unreachable

EmailTriageAgent.__mro__ is [EmailTriageAgent, Agent, ABC, MemoryMixin, ...], and Agent.process_query calls _process_query_impl without delegating, so MemoryMixin.process_query never runs for this agent. Fixed by calling self.report_memory_unavailable() at the top of EmailTriageAgent.process_query rather than reordering the bases.

2. One-shot flag spent on a no-op console

SilentConsole.print_warning is a no-op, and _chat_helpers.py builds agents with silent_mode=True on the non-streaming path (:1472) and False on the streaming path (:1918), both sharing _agent_cache. report_memory_unavailable now returns early — without setting the flag — when the current console is a SilentConsole, so a later streaming turn on the same cached agent still gets the notice.

3. Dropped the unused console= parameter.

Verification

  • hub/agents/email/python/tests/test_email_memory.py — 33 passed (2 new)
  • tests/unit/test_memory_{mixin,store,router}.py — 694 passed, 2 skipped
  • New tests applied onto 56edcd09 unfixed: both fail. On 015d3313: both pass.
  • git merge-tree against current main: 0 conflicts; 289 tests pass on the merged tree.

I deliberately left test_email_memory.py unformatted by my local black (26.5.1) — it wants to reflow pre-existing lines that main's own copy also fails, so that's version drift here, not something to churn in your diff.

Still open, not blocking this PR — in the Agent UI the warning arrives as a status event and ChatView.tsx consolidates consecutive status steps, so it flickers rather than persisting as the degraded-state indicator #2831 asks for. Read from the code, not observed live. Worth a follow-up.

@itomek
itomek added this pull request to the merge queue Sep 2, 2026
Merged via the queue into amd:main with commit 0525d5b Sep 2, 2026
40 of 43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent::email Email agent changes agents

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants