Skip to content

VaronisSaaS 3.2.0: CCF connector ingests into dedicated VaronisAlerts… - #14947

Open
vdobrotskyi-varonis wants to merge 1 commit into
Azure:masterfrom
vdobrotskyi-varonis:users/VitaliiD/ccf-dedicated-table
Open

VaronisSaaS 3.2.0: CCF connector ingests into dedicated VaronisAlerts…#14947
vdobrotskyi-varonis wants to merge 1 commit into
Azure:masterfrom
vdobrotskyi-varonis:users/VitaliiD/ccf-dedicated-table

Conversation

@vdobrotskyi-varonis

Copy link
Copy Markdown
Contributor

Root cause:

  • The Azure Function connector ingests via the legacy HTTP Data Collector API, which creates VaronisAlerts_CL implicitly as a Classic custom-log table. Classic custom-log tables permanently carry additional standard columns (Computer, RawData, MG, ManagementGroupName).
  • The CCF connector uses a DCR, which requires an explicitly declared output table schema.
  • Azure table schema updates are additive-only — existing columns can never be removed or retyped, and those extra columns are standard columns, so they cannot be declared in the table resource either.

The result is an unresolvable conflict: the DCR output schema can never match the pre-existing table. This was confirmed in a lab that reproduced the customer workspace (identical 41-column schema); migrating the table from Classic to DataCollectionRuleBased did not resolve it, and neither did declaring or omitting the extra columns.

The alert fields themselves are unchanged — this is purely a table-provenance issue that only affects workspaces upgrading from the Azure Function connector. Fresh workspaces were never affected.

Changes

  • VaronisSaaS_Table.json — table renamed to VaronisAlertsV2_CL (schema/columns unchanged, 33 columns).
  • VaronisSaaS_DCR.jsonoutputStream changed to Custom-VaronisAlertsV2_CL.
  • VaronisSaaS_ConnectorDefinition.json — graph queries, sample queries, dataTypes, lastDataReceivedQuery and connectivityCriteria updated to VaronisAlertsV2_CL.
  • Analytic Rules (4) — now query VaronisAlertsV2_CL.
  • Workbook — all 9 queries changed to union isfuzzy=true VaronisAlerts_CL, VaronisAlertsV2_CL so existing customers keep seeing their historical alerts alongside new data.
  • .script/tests/KqlvalidationsTests/CustomTables/VaronisAlertsV2_CL.json — new KQL validation fixture.
  • ReleaseNotes.md — 3.2.0 entry; solution repackaged at 3.2.0.

The deprecated Azure Function connector is unchanged and continues to write to VaronisAlerts_CL.

Validation

Ran .script/local-validation/build-and-validate.ps1 -SolutionName VaronisSaaS:

Validator Result
KQL ✅ 8 passed / 13 files
ARM-TTK (mainTemplate + createUiDefinition) ✅ 48 passed, 0 failed
Field Types ✅ passed
Classic App Insights ✅ passed
Hyperlink Validation ✅ passed

Live deployment verified: deployed Package/mainTemplate.json into a lab workspace that had a pre-existing Classic VaronisAlerts_CL table (created by the legacy Azure Function connector) — the CCF connector, DCR, and DCE now deploy successfully.

@vdobrotskyi-varonis
vdobrotskyi-varonis requested review from a team as code owners August 19, 2026 15:34
@v-atulyadav
v-atulyadav requested a lite review from Copilot August 20, 2026 03:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Updates the VaronisSaaS solution to avoid DCR schema conflicts in upgraded workspaces by ingesting CCF connector data into a new dedicated custom table (VaronisAlertsV2_CL) while keeping legacy historical visibility.

Changes:

  • Renames the CCF output table to VaronisAlertsV2_CL and updates the DCR output stream accordingly.
  • Updates connector definition, analytic rules, and solution versioning to reference VaronisAlertsV2_CL.
  • Updates workbook queries to union legacy (VaronisAlerts_CL) and new (VaronisAlertsV2_CL) tables for continuity.

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
Solutions/VaronisSaaS/Workbooks/VaronisSaaS.json Workbook queries updated to union old/new alert tables.
Solutions/VaronisSaaS/ReleaseNotes.md Adds 3.2.0 entry describing the new ingestion table.
Solutions/VaronisSaaS/Package/mainTemplate.json Bumps solution/assets version to 3.2.0 and updates embedded content to use VaronisAlertsV2_CL.
Solutions/VaronisSaaS/Data/Solution_VaronisSaaS.json Updates solution version metadata to 3.2.0.
Solutions/VaronisSaaS/Data Connectors/VaronisSaaS_CCF/VaronisSaaS_Table.json Renames table resource/schema name to VaronisAlertsV2_CL.
Solutions/VaronisSaaS/Data Connectors/VaronisSaaS_CCF/VaronisSaaS_DCR.json Updates DCR output stream to Custom-VaronisAlertsV2_CL.
Solutions/VaronisSaaS/Data Connectors/VaronisSaaS_CCF/VaronisSaaS_ConnectorDefinition.json Updates connector queries/criteria to VaronisAlertsV2_CL.
Solutions/VaronisSaaS/Analytic Rules/VaronisMediumSeverityAlertsDetected.yaml Points rule query + required dataTypes to VaronisAlertsV2_CL.
Solutions/VaronisSaaS/Analytic Rules/VaronisLowSeverityAlertsDetected.yaml Points rule query + required dataTypes to VaronisAlertsV2_CL.
Solutions/VaronisSaaS/Analytic Rules/VaronisInformationalAlertsDetected.yaml Points rule query + required dataTypes to VaronisAlertsV2_CL.
Solutions/VaronisSaaS/Analytic Rules/VaronisHighSeverityAlertsDetected.yaml Points rule query + required dataTypes to VaronisAlertsV2_CL.
.script/tests/KqlvalidationsTests/CustomTables/VaronisAlertsV2_CL.json Adds KQL validation fixture for the new table schema.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread Solutions/VaronisSaaS/Workbooks/VaronisSaaS.json Outdated
Comment thread Solutions/VaronisSaaS/Workbooks/VaronisSaaS.json
…V2_CL table

The CCF Push data connector previously shared the legacy VaronisAlerts_CL Classic custom-log table with the deprecated Azure Function connector. Classic tables carry immutable standard columns that conflict with the DCR-based schema the CCF connector needs, causing deployment failures on upgrade. This change gives the CCF connector its own dedicated table, VaronisAlertsV2_CL:

- New Data Connectors/VaronisSaaS_CCF/VaronisSaaS_Table.json defines VaronisAlertsV2_CL.
- DCR outputStream, ConnectorDefinition queries/dataTypes, and all 4 Analytic Rules updated to reference VaronisAlertsV2_CL (rule versions bumped 1.0.0 -> 1.0.1).
- Workbook queries updated to union VaronisAlerts_CL and VaronisAlertsV2_CL so historical and new data both show up, with a month-index off-by-one bug (months[getmonth(x)] vs months[getmonth(x)-1]) fixed across all time-series tiles.
- Added KQL validation fixture for the new table.

The legacy Azure Function connector is unaffected and continues writing to VaronisAlerts_CL.
@vdobrotskyi-varonis
vdobrotskyi-varonis force-pushed the users/VitaliiD/ccf-dedicated-table branch from b5a02d2 to 53cdb48 Compare August 20, 2026 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants