Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 43 additions & 44 deletions guides/ai-agents/deep-research.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,20 @@ description: "Run a durable, multi-step AI investigation and get an evidence-bac

Deep research is a long-running mode for Lightdash AI agents. It is designed for questions that need several queries, competing explanations, and a reusable report rather than one immediate answer.

The agent investigates in the background using its Lightdash context and the sources you approve. It returns a structured Markdown report with confidence levels, supporting charts, citations for external sources, and explicit caveats.
The agent investigates in the background using its Lightdash context and configured sources. It returns a structured Markdown report with confidence levels, supporting charts, citations for external sources, and explicit caveats.

<Frame>
<img
src="/images/guides/ai-agents/deep-research/deep-research-settings.png"
alt="Deep Research prompt interface showing the question, research depth options, and selected MCP sources"
alt="Ask AI composer with Deep research mode enabled and highlighted by an arrow"
/>
</Frame>

## When to use deep research

<CardGroup cols={2}>
<Card title="Use deep research" icon="telescope">
Choose this mode for multi-step investigations that need cross-checking, several data cuts, or a report you can share and revisit.
Choose this mode for multi-step investigations that need cross-checking, several data cuts, or a report you can save and revisit.
</Card>
<Card title="Use Ask mode" icon="comment">
Stay in the default mode for a quick lookup, one chart, or an interactive conversation where you want to steer each follow-up.
Expand All @@ -40,32 +40,27 @@ Questions that work well include:

```mermaid
flowchart LR
A["Question, depth, and sources"] --> B["Durable background run"]
B --> C["Discover relevant context"]
C --> D["Query data and test explanations"]
D --> C
D --> E["Validate findings"]
A["Question and agent configuration"] --> B["Plan competing hypotheses"]
B --> C["Investigate hypotheses in parallel"]
C --> D["Compare supporting and falsifying evidence"]
D --> E["Independent judge synthesizes the report"]
E --> F["Report and chart snapshots"]
```

Deep research uses the selected AI agent's configuration, including its instructions, semantic-layer access, knowledge documents, project and repository context, and enabled tools. It records progress as it works and saves the final report in the originating thread.
Deep research uses the selected AI agent's configuration, including its instructions, semantic-layer access, knowledge documents, project and repository context, and enabled tools. It automatically inherits the organization's research limits and every MCP server attached to the agent; there is no per-run depth or source selection.

For each run, a planner creates distinct, falsifiable hypotheses. Separate investigators test them in parallel, looking for both supporting and contradicting evidence. An independent judge then compares their findings and produces the final report. This structure helps the agent distinguish correlation from causation and makes inconclusive results explicit instead of forcing a single explanation.

Because the run executes on the server, you can close the tab or leave the thread. Reopening the thread restores the run card and its latest state.

## Start a run

<Steps>
<Step title="Open an AI agent thread">
Start a new thread or open an existing thread that you own. Deep research is unavailable in read-only threads, such as another user's thread or a thread started in Slack.
</Step>
<Step title="Select Deep research">
Click **Deep research (Beta)** in the composer. The research settings appear above the message input.
<Step title="Open Ask AI">
Use the Ask AI composer on the homepage, start a new agent thread, or open an existing thread that you own. Deep research is unavailable in read-only threads, such as another user's thread or a thread started in Slack.
</Step>
<Step title="Choose a research depth">
Select **Low**, **Medium**, **High**, or **Extra High**. Medium is the default.
</Step>
<Step title="Review MCP sources">
Choose which MCP servers attached to the agent can participate in this run. All available attached servers are selected by default.
<Step title="Enable Deep research">
Select **Deep research** in a new conversation, or select the telescope icon in an existing conversation. The control changes color when the mode is active.
</Step>
<Step title="Describe the outcome you need">
Include the decision or question, relevant time period, important segments, and any definitions or constraints the agent should preserve.
Expand All @@ -75,26 +70,18 @@ Because the run executes on the server, you can close the tab or leave the threa
</Step>
</Steps>

## Research depth
## Organization-wide limits

Depth controls the run's resource budget, including model tokens, tool calls, warehouse queries, and result rows. It is not a fixed duration: warehouse latency does not use up the reasoning budget.
Organization admins set the safety limits inherited by every deep research run. Go to **Organization settings** → **Ask AI** → **Deep research** to configure:

<CardGroup cols={2}>
<Card title="Low" icon="gauge-low">
Up to **10 warehouse queries**. Best for a focused check of the strongest available evidence.
</Card>
<Card title="Medium" icon="gauge">
Up to **25 warehouse queries**. Best for a balanced investigation with validation and alternatives.
</Card>
<Card title="High" icon="gauge-high">
Up to **50 warehouse queries**. Best for a broad investigation with more competing explanations.
</Card>
<Card title="Extra High" icon="chart-simple">
Up to **100 warehouse queries**. Best for the widest evidence review on high-stakes questions.
</Card>
</CardGroup>
- **Maximum tokens** — total model tokens across planning, parallel investigations, and judging
- **Maximum tool calls** — total tool calls across the run
- **Maximum warehouse queries** — total semantic-layer and SQL queries across the run
- **Maximum hypotheses** — number of competing explanations the planner creates and investigators test

Each limit must be a positive whole number. When a run reaches a limit after producing usable findings, Lightdash preserves the best available report and marks the run as partially completed.

Start with the smallest depth likely to answer your question. A larger depth gives the agent more opportunities to explore and validate, but it does not guarantee a better answer when the semantic layer or source data is incomplete.
By default, a run can use up to 10 million model tokens, 1,000 tool calls, 100 warehouse queries, and 5 hypotheses. Organization admins can change these values to match their governance and cost requirements.

## Sources and permissions

Expand All @@ -103,24 +90,24 @@ Deep research can use:
- **Agent context and project data** — the semantic layer, saved Lightdash content, knowledge documents, and other context configured on the selected agent, subject to its [data access settings](/guides/ai-agents/data-access).
- **Warehouse queries** — semantic queries and, when the agent and user are allowed to use it, SQL.
- **Repository context** — project context and source-code tools configured on the agent.
- **MCP servers** — only the [agent-attached servers](/guides/ai-agents/mcp-servers) selected in the research settings.
- **MCP servers** — every [server attached to the agent](/guides/ai-agents/mcp-servers) and its enabled tools.

<Warning>
Deep research runs without pausing for approval on each warehouse query or MCP tool call. An enabled MCP server can expose write actions, and those actions can run unattended. Select only servers whose enabled tools you want the agent to use for this investigation.
Deep research runs without pausing for approval on each warehouse query or MCP tool call. An attached MCP server can expose write actions, and its enabled actions can run unattended. Review the agent's attached servers and enabled tools before starting a run.
</Warning>

Deep research does not grant new permissions. The run starts with the creator's access and the selected agent's configuration, and Lightdash revalidates that access while the investigation runs. Revoking access can stop an active run.
Deep research does not grant new permissions. The run starts with the creator's access and the selected agent's configuration, and Lightdash revalidates that access while the investigation runs. Revoking access or disconnecting a source can stop an active investigation or leave that source unavailable.

## Follow progress

The run card stays in the thread and shows the latest phase, elapsed time, warehouse-query count, finding count, and recent activity.
The run card stays next to the question that started it and shows the latest phase, elapsed time, warehouse-query count, finding count, and recent activity.

<AccordionGroup>
<Accordion title="Queued">
Lightdash accepted the run and is waiting for a background worker to start it.
</Accordion>
<Accordion title="Running">
The agent is gathering context, testing explanations, validating findings, or writing the report. Select **View activity** to inspect recent progress.
The agent is planning hypotheses, investigating them in parallel, or synthesizing the report. Select **View activity** to inspect recent progress.
</Accordion>
<Accordion title="Completed">
The full report is ready and saved in the thread.
Expand All @@ -138,6 +125,8 @@ The run card stays in the thread and shows the latest phase, elapsed time, wareh

Select **Stop research** while a run is queued or running to request cancellation. Cancellation is asynchronous, so a running tool call may reach its next safe checkpoint before the status changes.

Only one deep research run can be active in a thread at a time. While it is active, the deep research control is disabled, but you can continue sending regular chat messages in the same thread. The control becomes available again when the run completes, partially completes, fails, or is cancelled.

## Read the report

Select **Open full report** from a completed or partially completed run card. A report contains:
Expand Down Expand Up @@ -165,14 +154,20 @@ Warehouse-backed charts open on a snapshot of the data the agent used when it wr

Select **Live data** on a warehouse-backed chart to rerun its stored query and compare the latest result with the snapshot. Charts computed by the agent from derived or external data have no single warehouse query and cannot be refreshed.

### Report retention

Deep research report content and chart snapshots expire **30 days after the run completes**. The run's question, status, and completion date remain in the thread. After expiry, select **Run again** to start a new investigation from the original question using the agent's current configuration and the organization's current limits.

The regular chat agent can use the status and report from deep research runs in the same conversation when answering follow-up questions. Ask it to clarify a finding, compare evidence, or explain a limitation without pasting the report back into the chat.

## Get better reports

- State the decision you are trying to make, not only the metric you want to inspect.
- Define ambiguous terms such as *active customer*, *conversion*, or *retention*.
- Include the time range and comparison period.
- Name segments the agent must test, such as channel, region, plan, or product category.
- Ask it to test alternatives or contradictions instead of assuming one cause.
- Treat a partially completed report as a starting point and rerun at a larger depth only when the missing evidence matters.
- Treat a partially completed report as a starting point. Resolve unavailable sources or tighten the question before running it again; ask an organization admin to review the limits if runs repeatedly exhaust them.

## Access requirements

Expand All @@ -190,14 +185,18 @@ No. The run continues on the server and its state is saved in the thread.

Yes. You can start a run from an existing thread you own as long as the thread is not read-only and the agent's model is still available.

**Can I start another run while research is still active?**

Not in the same thread. You can keep chatting normally, or start deep research in another eligible thread. The control returns when the current run reaches a terminal state.

**Why did my run partially complete?**

The agent reached a resource budget or encountered a recoverable failure after producing a valid draft. Lightdash preserves the supported findings rather than discarding the report.

**Does deep research only read data?**

No. It uses the selected agent's configured tools. Warehouse queries run without individual approval, and selected MCP servers may include write actions. Review the source selection before starting.
No. It uses the selected agent's configured tools. Warehouse queries run without individual approval, and attached MCP servers may include write actions. Review the agent's configuration before starting.

**Can I continue chatting after the report is ready?**

Yes. The report remains attached to the thread, and you can continue the conversation with the same agent.
Yes. The report remains available in the thread for 30 days, and the same agent can use it as context for follow-up questions.
Binary file modified images/guides/ai-agents/deep-research/deep-research-settings.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading