Recorded Future - HTTP Data Collector Deprecation

[Erik Mangsten (ErikMangstenRecFut)]

Required items, please complete

Change(s):

• Replaced HTTP Data collector with DCR/DCE

Reason for Change(s):

• HTTP Data Collector is deprecation

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR migrates the Recorded Future solution’s ingestion path away from the deprecated HTTP Data Collector connector to the Azure Monitor Logs Ingestion API (DCE/DCR), introducing new _V2_CL tables and updating workbook/analytic rule queries accordingly.

Changes:

• Added shared “Data Connectors infrastructure” ARM template to deploy the DCE, DCRs, tables, and connector tile definition.
• Updated playbook ARM templates to send data via MSI-authenticated HTTP to the Logs Ingestion API, with optional automatic role assignment.
• Updated workbooks, analytic rules, and documentation to use the new _V2_CL tables/columns.

Reviewed changes

Copilot reviewed 28 out of 30 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
Solutions/Recorded Future/Workbooks/RecordedFutureThreatActorHunting.json	Updates Threat Actor hunting workbook queries to use RecordedFutureThreatMap_V2_CL and dynamic data format.
Solutions/Recorded Future/Workbooks/RecordedFuturePlaybookAlertOverview.json	Updates workbook table selection and queries to RecordedFuturePlaybookAlerts_V2_CL and new column names.
Solutions/Recorded Future/Workbooks/RecordedFutureMalwareThreatHunting.json	Updates Malware hunting workbook queries to use RecordedFutureThreatMapMalware_V2_CL and dynamic data format.
Solutions/Recorded Future/Workbooks/RecordedFutureAlertOverview.json	Updates Alerts workbook to RecordedFutureClassicAlerts_V2_CL and new column names/structures.
Solutions/Recorded Future/Playbooks/readme.md	Adds migration guidance, table rename mapping, and new deployment prerequisites for DCE/DCR infrastructure.
Solutions/Recorded Future/Playbooks/ThreatHunting/readme.md	Removes references to the deprecated Data Collector connector for threat hunting playbooks.
Solutions/Recorded Future/Playbooks/ThreatHunting/RecordedFuture-ThreatMapMalware-Importer/azuredeploy.json	Migrates Malware Threat Map importer to Logs Ingestion API + optional RBAC assignment.
Solutions/Recorded Future/Playbooks/ThreatHunting/RecordedFuture-ThreatMap-Importer/azuredeploy.json	Migrates Threat Map importer to Logs Ingestion API + optional RBAC assignment.
Solutions/Recorded Future/Playbooks/Sandboxing/readme.md	Updates sandbox docs for _V2_CL table names and removes deprecated connector mention.
Solutions/Recorded Future/Playbooks/Sandboxing/RecordedFuture-Sandbox_StorageAccount/azuredeploy.json	Migrates storage sandbox playbook to Logs Ingestion API + optional RBAC assignment.
Solutions/Recorded Future/Playbooks/Sandboxing/RecordedFuture-Sandbox_Outlook_Attachment/azuredeploy.json	Migrates Outlook sandbox playbook to Logs Ingestion API + optional RBAC assignment.
Solutions/Recorded Future/Playbooks/Alerts/readme.md	Updates alert playbook documentation for new _V2_CL tables and connector changes.
Solutions/Recorded Future/Playbooks/Alerts/RecordedFuture-Playbook-Alert-Importer/azuredeploy.json	Migrates playbook-alert ingestion to Logs Ingestion API + optional RBAC assignment.
Solutions/Recorded Future/Playbooks/Alerts/RecordedFuture-Alert-Importer/azuredeploy.json	Migrates classic alert ingestion to Logs Ingestion API, adds RBAC assignments for DCR + workspace read.
Solutions/Recorded Future/Package/createUiDefinition.json	Updates solution install UI to include “Data Connectors” step and updated text.
Solutions/Recorded Future/Data/Solution_RecordedFuture.json	Adds Data Connector definition file to solution packaging list.
Solutions/Recorded Future/Data Connectors/azuredeploy.json	New: deploys shared DCE/DCR/table infrastructure and connector tile.
Solutions/Recorded Future/Data Connectors/RecordedFuture_ConnectorDefinition.json	New: packaged connector definition tile for Content Hub discovery.
Solutions/Recorded Future/Data Connectors/README.md	New: documents connector tile duplication and infrastructure template usage.
Solutions/Recorded Future/Analytic Rules/IncidentCreation/RecordedFutureSandboxStorageAccount.yaml	Updates analytic rule to query RecordedFutureSandboxResults_V2_CL and new fields; adds required connector metadata.
Solutions/Recorded Future/Analytic Rules/IncidentCreation/RecordedFutureSandboxEmailAttachment.yaml	Updates analytic rule to query RecordedFutureSandboxResults_V2_CL and new fields; adds required connector metadata.
Solutions/Recorded Future/Analytic Rules/IncidentCreation/RecordedFuturePlaybookAlerts.yaml	Updates analytic rule to query RecordedFuturePlaybookAlerts_V2_CL and new fields; adds required connector metadata.
Solutions/Recorded Future/Analytic Rules/IncidentCreation/RecordedFutureAlerts.yaml	Updates analytic rule to query RecordedFutureClassicAlerts_V2_CL and new fields; adds required connector metadata.
.script/tests/KqlvalidationsTests/CustomTables/RecordedFutureThreatMap_V2_CL.json	New: adds custom table schema fixture for KQL validation.
.script/tests/KqlvalidationsTests/CustomTables/RecordedFutureThreatMapMalware_V2_CL.json	New: adds custom table schema fixture for KQL validation.
.script/tests/KqlvalidationsTests/CustomTables/RecordedFutureSandboxResults_V2_CL.json	New: adds custom table schema fixture for KQL validation.
.script/tests/KqlvalidationsTests/CustomTables/RecordedFuturePlaybookAlerts_V2_CL.json	New: adds custom table schema fixture for KQL validation.
.script/tests/KqlvalidationsTests/CustomTables/RecordedFutureClassicAlerts_V2_CL.json	New: adds custom table schema fixture for KQL validation.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Erik Mangsten (ErikMangstenRecFut)]

v-shukore Seems like someone has made changes to master, that conflicts the "mainTemplate.json" that is generated by the provided createSolutionV3.ps1.