From bea18fb8ec61efc422b80647352958909b806662 Mon Sep 17 00:00:00 2001 From: Max Phillips Date: Tue, 14 Jul 2026 13:12:55 -0500 Subject: [PATCH 1/7] [DOCS-15097] Document subqueries & Reference Table joins in log monitors - Cross-link Log Monitor and Advanced Search pages - Add advanced search filtering section to the log monitor page - Correct subquery match limit: default 1000, max 1,000,000 --- content/en/logs/explorer/advanced_search.md | 8 +++++++- content/en/monitors/types/log.md | 12 ++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/content/en/logs/explorer/advanced_search.md b/content/en/logs/explorer/advanced_search.md index 4c4698e484f..20876ec6386 100644 --- a/content/en/logs/explorer/advanced_search.md +++ b/content/en/logs/explorer/advanced_search.md @@ -14,12 +14,17 @@ further_reading: - link: "/logs/explorer/saved_views/" tag: "Documentation" text: "Learn about Saved Views" +- link: "/monitors/types/log/" + tag: "Documentation" + text: "Create Log Monitors" --- ## Overview When you need to further refine the results of a log search, use [subqueries](#filter-logs-with-subqueries) to compare your results against a secondary group of logs, or filter your logs using data from your [Reference Tables](#filter-logs-based-on-reference-tables). +You can use these advanced search options in the Log Explorer and in [log monitors][3], where they let you scope alerts to curated datasets such as curated entity lists, customer tiers, or correlated events. + ## Filter logs with subqueries Use subqueries when you want to filter the results of a query based on the results of a secondary query. See [subquery examples](#subquery-examples) for two example scenarios. @@ -41,7 +46,7 @@ This introduces new elements to the query editor: 1. Select whether you want to use the {{< ui >}}IN{{< /ui >}} or {{< ui >}}NOT IN{{< /ui >}} operator: - The results from the {{< ui >}}IN{{< /ui >}} operator only include logs where the attribute value is also found in the subquery results. For example, if you only want to see logs generated by `service:a` and is associated to a user who is also one of the top users of `service:b`. - The results from the {{< ui >}}NOT IN{{< /ui >}} operator exclude logs where the attribute value is found in the subquery results. For example, if you only want to see `status:error` logs, but also want to filter out `status:error` logs where users associated with those logs eventually end up with a `status:success` log. See [Filter outdated or superseded logs](#filter-outdated-or-superseded-logs) for a detailed example. -1. Optionally, reduce the number of subquery attribute values to match against. The default value and maximum value is `1000`. Choose between {{< ui >}}top{{< /ui >}} (highest frequency values) or {{< ui >}}bottom{{< /ui >}} (lowest frequency values). +1. Optionally, reduce the number of subquery attribute values to match against. The default value is `1000` and the maximum value is `1,000,000`. Choose between {{< ui >}}top{{< /ui >}} (highest frequency values) or {{< ui >}}bottom{{< /ui >}} (lowest frequency values). ### Subquery examples @@ -84,3 +89,4 @@ In this example, use `service:network_directory` as your main query, and then de [1]: https://app.datadoghq.com/logs [2]: /integrations/guide/reference-tables/?tab=manualupload +[3]: /monitors/types/log/ diff --git a/content/en/monitors/types/log.md b/content/en/monitors/types/log.md index cb439f8aeb4..f53f05c05af 100644 --- a/content/en/monitors/types/log.md +++ b/content/en/monitors/types/log.md @@ -71,6 +71,17 @@ As you define the search query, the graph above the search fields updates. * {{< ui >}}Simple-Alert{{< /ui >}}: Simple alerts aggregate over all reporting sources. You receive one alert when the aggregated value meets the set conditions. This works best to monitor a metric from a single host or the sum of a metric across many hosts. This strategy may be selected to reduce notification noise. * {{< ui >}}Multi Alert{{< /ui >}}: Multi alerts apply the alert to each source according to your group parameters. An alerting event is generated for each group that meets the set conditions. For example, you could group `system.disk.in_use` by `device` to receive a separate alert for each device that is running out of space. +### Filter with subqueries and Reference Tables + +Log monitors support [advanced log search][11], so you can scope alerts to curated datasets: + +- Use **subqueries** to filter monitor results against the results of a secondary query, for example to correlate across log sources or suppress noise from related events. +- Use **Reference Table** joins to filter or enrich monitor results with external metadata, such as customer tier, region, or a curated list of IP addresses or accounts. + +To add a subquery or Reference Table join, use the {{< ui >}}Σ Modify Query{{< /ui >}} menu in the monitor configuration. You can also create a monitor directly from an advanced search in the [Log Explorer][4]. + +For details on building these queries and their limits, see [Advanced Search][11]. + ### Set alert conditions Trigger when the query meets one of the following conditions compared to a threshold value: @@ -143,3 +154,4 @@ Include a sample of 10 logs in the alert notification: [8]: /monitors/notify/ [9]: /monitors/notify/variables/?tab=is_alert#matching-attributetag-variables [10]: /logs/log_configuration/flex_logs/ +[11]: /logs/explorer/advanced_search/ From a4e717b64d492773a19da32a7ed9739bf251455f Mon Sep 17 00:00:00 2001 From: Max Phillips Date: Tue, 14 Jul 2026 13:34:02 -0500 Subject: [PATCH 2/7] Apply editorial review: standardize terminology and tighten wording --- content/en/logs/explorer/advanced_search.md | 2 +- content/en/monitors/types/log.md | 14 ++++++-------- 2 files changed, 7 insertions(+), 9 deletions(-) diff --git a/content/en/logs/explorer/advanced_search.md b/content/en/logs/explorer/advanced_search.md index 20876ec6386..ae695eea0a7 100644 --- a/content/en/logs/explorer/advanced_search.md +++ b/content/en/logs/explorer/advanced_search.md @@ -23,7 +23,7 @@ further_reading: When you need to further refine the results of a log search, use [subqueries](#filter-logs-with-subqueries) to compare your results against a secondary group of logs, or filter your logs using data from your [Reference Tables](#filter-logs-based-on-reference-tables). -You can use these advanced search options in the Log Explorer and in [log monitors][3], where they let you scope alerts to curated datasets such as curated entity lists, customer tiers, or correlated events. +These advanced search options are available in the Log Explorer and in [log monitors][3], where they scope alerts to curated datasets such as entity lists, customer tiers, or correlated events. ## Filter logs with subqueries diff --git a/content/en/monitors/types/log.md b/content/en/monitors/types/log.md index f53f05c05af..60e076b04d5 100644 --- a/content/en/monitors/types/log.md +++ b/content/en/monitors/types/log.md @@ -30,7 +30,7 @@ further_reading: Logs are essential for security investigations, aiding in threat detection, compliance tracking, and security monitoring. Log Management systems correlate logs with observability data for rapid root cause detection. Log management also enables efficient troubleshooting, issue resolution, and security audits. -Once [log management is enabled][1] for your organization, you can create a logs monitor to alert you when a specified log type exceeds a user-defined threshold over a given period of time. The logs monitor only evaluates [indexed logs][2]. +Once [log management is enabled][1] for your organization, you can create a logs monitor to alert you when a specified log type exceeds a user-defined threshold over a given period of time. The logs monitor only evaluates [indexed logs][2]. **Note**: Log monitors have a maximum rolling time window of `2 days`. @@ -73,15 +73,13 @@ As you define the search query, the graph above the search fields updates. ### Filter with subqueries and Reference Tables -Log monitors support [advanced log search][11], so you can scope alerts to curated datasets: +Log monitors support [advanced search][11], so you can scope alerts to curated datasets: -- Use **subqueries** to filter monitor results against the results of a secondary query, for example to correlate across log sources or suppress noise from related events. -- Use **Reference Table** joins to filter or enrich monitor results with external metadata, such as customer tier, region, or a curated list of IP addresses or accounts. +- Use **subqueries** to filter monitor results against the results of a secondary query. For example, you can correlate across log sources or suppress noise from related events. +- Use **Reference Table joins** to filter or enrich monitor results with external metadata. For example, you can use customer tier, region, or a curated list of IP addresses or accounts. To add a subquery or Reference Table join, use the {{< ui >}}Σ Modify Query{{< /ui >}} menu in the monitor configuration. You can also create a monitor directly from an advanced search in the [Log Explorer][4]. -For details on building these queries and their limits, see [Advanced Search][11]. - ### Set alert conditions Trigger when the query meets one of the following conditions compared to a threshold value: @@ -94,9 +92,9 @@ Trigger when the query meets one of the following conditions compared to a thres #### No data and below alerts -`NO DATA` is a state given when no logs match the monitor query during the timeframe. +`NO DATA` is a state given when no logs match the monitor query during the time frame. -To receive a notification when all groups matching a specific query have stopped sending logs, set the condition to `below 1`. This notifies when no logs match the monitor query in a given timeframe across all aggregate groups. +To receive a notification when all groups matching a specific query have stopped sending logs, set the condition to `below 1`. This notifies when no logs match the monitor query in a given time frame across all aggregate groups. When splitting the monitor by any dimension (tag or facet) and using a `below` condition, the alert is triggered **if and only if** there are logs for a given group, and the count is below the threshold—or if there are no logs for **all** of the groups. From 638d9d569285b32ffaff5c252d9ba6fd15626601 Mon Sep 17 00:00:00 2001 From: Max Phillips Date: Tue, 14 Jul 2026 14:52:57 -0500 Subject: [PATCH 3/7] make sentence structure parallel --- content/en/monitors/types/log.md | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/content/en/monitors/types/log.md b/content/en/monitors/types/log.md index 60e076b04d5..b6b8bddaf3d 100644 --- a/content/en/monitors/types/log.md +++ b/content/en/monitors/types/log.md @@ -67,18 +67,17 @@ As you define the search query, the graph above the search fields updates. * **Third facet**: 5 top values * **Fourth facet**: 2 top values -4. Configure the alerting grouping strategy (optional): - * {{< ui >}}Simple-Alert{{< /ui >}}: Simple alerts aggregate over all reporting sources. You receive one alert when the aggregated value meets the set conditions. This works best to monitor a metric from a single host or the sum of a metric across many hosts. This strategy may be selected to reduce notification noise. - * {{< ui >}}Multi Alert{{< /ui >}}: Multi alerts apply the alert to each source according to your group parameters. An alerting event is generated for each group that meets the set conditions. For example, you could group `system.disk.in_use` by `device` to receive a separate alert for each device that is running out of space. - -### Filter with subqueries and Reference Tables +4. Modify the query with additional options (optional): -Log monitors support [advanced search][11], so you can scope alerts to curated datasets: + Use the {{< ui >}}Σ Modify Query{{< /ui >}} menu to refine the query with: + * **Functions and formulas**: Transform and combine your query results. See [functions][11] and [formulas][12]. + * **Advanced search**: Scope alerts to curated datasets, such as entity lists, customer tiers, or correlated events. See [subqueries and Reference Table joins][13]. -- Use **subqueries** to filter monitor results against the results of a secondary query. For example, you can correlate across log sources or suppress noise from related events. -- Use **Reference Table joins** to filter or enrich monitor results with external metadata. For example, you can use customer tier, region, or a curated list of IP addresses or accounts. + You can also create a monitor directly from an advanced search in the [Log Explorer][4]. -To add a subquery or Reference Table join, use the {{< ui >}}Σ Modify Query{{< /ui >}} menu in the monitor configuration. You can also create a monitor directly from an advanced search in the [Log Explorer][4]. +5. Configure the alerting grouping strategy (optional): + * {{< ui >}}Simple-Alert{{< /ui >}}: Simple alerts aggregate over all reporting sources. You receive one alert when the aggregated value meets the set conditions. This works best to monitor a metric from a single host or the sum of a metric across many hosts. This strategy may be selected to reduce notification noise. + * {{< ui >}}Multi Alert{{< /ui >}}: Multi alerts apply the alert to each source according to your group parameters. An alerting event is generated for each group that meets the set conditions. For example, you could group `system.disk.in_use` by `device` to receive a separate alert for each device that is running out of space. ### Set alert conditions @@ -152,4 +151,6 @@ Include a sample of 10 logs in the alert notification: [8]: /monitors/notify/ [9]: /monitors/notify/variables/?tab=is_alert#matching-attributetag-variables [10]: /logs/log_configuration/flex_logs/ -[11]: /logs/explorer/advanced_search/ +[11]: /logs/explorer/analytics/#functions +[12]: /logs/explorer/analytics/#formulas +[13]: /logs/explorer/advanced_search/ From ee504c358a9b3323998df2c6e972ffe043e92d57 Mon Sep 17 00:00:00 2001 From: Max Phillips Date: Tue, 14 Jul 2026 14:54:19 -0500 Subject: [PATCH 4/7] adjust commas --- content/en/logs/explorer/advanced_search.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/logs/explorer/advanced_search.md b/content/en/logs/explorer/advanced_search.md index ae695eea0a7..9dccb7088d5 100644 --- a/content/en/logs/explorer/advanced_search.md +++ b/content/en/logs/explorer/advanced_search.md @@ -23,7 +23,7 @@ further_reading: When you need to further refine the results of a log search, use [subqueries](#filter-logs-with-subqueries) to compare your results against a secondary group of logs, or filter your logs using data from your [Reference Tables](#filter-logs-based-on-reference-tables). -These advanced search options are available in the Log Explorer and in [log monitors][3], where they scope alerts to curated datasets such as entity lists, customer tiers, or correlated events. +These advanced search options are available in the Log Explorer and [log monitors][3], where they scope alerts to curated datasets, such as entity lists, customer tiers, or correlated events. ## Filter logs with subqueries From d9feca23dae135b4c545a34a4388ca5a17755bb2 Mon Sep 17 00:00:00 2001 From: Max Phillips Date: Tue, 14 Jul 2026 15:05:07 -0500 Subject: [PATCH 5/7] disambiguate advanced search options scope --- content/en/logs/explorer/advanced_search.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/logs/explorer/advanced_search.md b/content/en/logs/explorer/advanced_search.md index 9dccb7088d5..69266f6aa4f 100644 --- a/content/en/logs/explorer/advanced_search.md +++ b/content/en/logs/explorer/advanced_search.md @@ -23,7 +23,7 @@ further_reading: When you need to further refine the results of a log search, use [subqueries](#filter-logs-with-subqueries) to compare your results against a secondary group of logs, or filter your logs using data from your [Reference Tables](#filter-logs-based-on-reference-tables). -These advanced search options are available in the Log Explorer and [log monitors][3], where they scope alerts to curated datasets, such as entity lists, customer tiers, or correlated events. +Advanced search options are available in the Log Explorer and [log monitors][3]. In log monitors, they scope alerts to curated datasets, such as entity lists, customer tiers, or correlated events. ## Filter logs with subqueries From 253d92736a04de1fbcb271c62b16a04c4043ae04 Mon Sep 17 00:00:00 2001 From: Max Phillips Date: Tue, 14 Jul 2026 16:26:09 -0500 Subject: [PATCH 6/7] refine intro sentence --- content/en/logs/explorer/advanced_search.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/en/logs/explorer/advanced_search.md b/content/en/logs/explorer/advanced_search.md index 69266f6aa4f..0a965729aeb 100644 --- a/content/en/logs/explorer/advanced_search.md +++ b/content/en/logs/explorer/advanced_search.md @@ -21,9 +21,9 @@ further_reading: ## Overview -When you need to further refine the results of a log search, use [subqueries](#filter-logs-with-subqueries) to compare your results against a secondary group of logs, or filter your logs using data from your [Reference Tables](#filter-logs-based-on-reference-tables). +When you need to further refine log search results or build curated datasets for analysis and monitoring, use [subqueries](#filter-logs-with-subqueries) to compare your results against a secondary group of logs, or filter your logs using data from your [Reference Tables](#filter-logs-based-on-reference-tables). -Advanced search options are available in the Log Explorer and [log monitors][3]. In log monitors, they scope alerts to curated datasets, such as entity lists, customer tiers, or correlated events. +Advanced search options are available in the Log Explorer and [log monitors][3]. ## Filter logs with subqueries From a44128cf36feac46053f5d0e9db0aac1071fdd20 Mon Sep 17 00:00:00 2001 From: Max Phillips Date: Wed, 15 Jul 2026 10:14:04 -0500 Subject: [PATCH 7/7] Apply suggestions from code review Co-authored-by: Bryce Eadie --- content/en/monitors/types/log.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/content/en/monitors/types/log.md b/content/en/monitors/types/log.md index b6b8bddaf3d..b26d0f84ee8 100644 --- a/content/en/monitors/types/log.md +++ b/content/en/monitors/types/log.md @@ -67,11 +67,9 @@ As you define the search query, the graph above the search fields updates. * **Third facet**: 5 top values * **Fourth facet**: 2 top values -4. Modify the query with additional options (optional): - - Use the {{< ui >}}Σ Modify Query{{< /ui >}} menu to refine the query with: +4. Optionally, use the {{< ui >}}Σ Modify Query{{< /ui >}} menu to refine the query with: * **Functions and formulas**: Transform and combine your query results. See [functions][11] and [formulas][12]. - * **Advanced search**: Scope alerts to curated datasets, such as entity lists, customer tiers, or correlated events. See [subqueries and Reference Table joins][13]. + * **Advanced search**: Scope alerts to curated datasets, such as entity lists, customer tiers, or correlated events. See [subqueries][13] and [Reference Tables][14]. You can also create a monitor directly from an advanced search in the [Log Explorer][4]. @@ -153,4 +151,5 @@ Include a sample of 10 logs in the alert notification: [10]: /logs/log_configuration/flex_logs/ [11]: /logs/explorer/analytics/#functions [12]: /logs/explorer/analytics/#formulas -[13]: /logs/explorer/advanced_search/ +[13]: /logs/explorer/advanced_search/#filter-logs-with-subqueries +[14]: /logs/explorer/advanced_search/#filter-logs-based-on-reference-tables