Skip to content

Enable EVP flagevaluation system tests for .NET#7186

Draft
leoromanovsky wants to merge 3 commits into
mainfrom
leo.romanovsky/ffe-evp-flagevaluation-enable-dotnet-clean
Draft

Enable EVP flagevaluation system tests for .NET#7186
leoromanovsky wants to merge 3 commits into
mainfrom
leo.romanovsky/ffe-evp-flagevaluation-enable-dotnet-clean

Conversation

@leoromanovsky

@leoromanovsky leoromanovsky commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Motivation

Enable the merged server-side EVP flagevaluation system tests for .NET after the .NET SDK PR validated the shared contract and local system-test path.

Changes

  • Changes tests/ffe/test_flag_eval_evp.py in manifests/dotnet.yml from missing_feature (FFL-2446) to v3.47.0.

Decisions

Validation Evidence

  • git diff --check - PASS.
  • System-test image package check: /app/app.deps.json contains Datadog.Trace/3.47.0 and Datadog.FeatureFlags.OpenFeature/2.3.0; /opt/datadog/Datadog.Trace.ClrProfiler.Native.so is present.
  • SYSTEM_TEST_BUILD_TIMEOUT=1800 ./build.sh --library dotnet --weblog-variant poc - PASS on 2026-06-20.
  • TEST_LIBRARY=dotnet WEBLOG_VARIANT=poc ./run.sh +l dotnet FEATURE_FLAGGING_AND_EXPERIMENTATION -k "test_flag_eval_evp" - PASS on 2026-06-20, 8 passed, 2627 deselected in 85.95s with Library: dotnet@3.47.0.
  • Degradation payload evidence from the passing run: one flagevaluation payload emitted 10001 rows with total evaluation_count=12000, including a capped aggregate row with evaluation_count=2000 and omitted targeting_key/context.
  • Real-backend dogfooding prefix is recorded in the .NET SDK PR: server-evp-e2e-dotnet-1781931066-user-.

@github-actions

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

manifests/dotnet.yml                                                    @DataDog/apm-dotnet @DataDog/asm-dotnet

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Jun 20, 2026

Copy link
Copy Markdown

Pipelines  Tests

Fix all issues with BitsAI

⚠️ Warnings

🚦 3 Pipeline jobs failed

Testing the test | System Tests (dotnet, dev) / End-to-end #1 / poc 1   View in Datadog   GitHub Actions

🧪 8 Tests failed

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic.test_ffe_evp_flagevaluation_basic[poc] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-basic-flag
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_evp_flagevaluation_event.<locals>.<lambda> at 0x7f9e7a3dc0e0>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic object at 0x7f9eaadd3560>

    def test_ffe_evp_flagevaluation_basic(self) -> None:
        assert self.r.status_code == 200, f"Flag evaluation failed: {self.r.text}"
...
tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Burst_Aggregation.test_ffe_evp_flagevaluation_burst_aggregation[poc] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-burst-aggregation-flag
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_evp_flagevaluation_event.<locals>.<lambda> at 0x7f9e7a3dfec0>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Burst_Aggregation object at 0x7f9eaadd29c0>

    def test_ffe_evp_flagevaluation_burst_aggregation(self) -> None:
        for index, response in enumerate(self.responses):
...
View all 8 test failures

Testing the test | System Tests (dotnet, dev) / End-to-end #1 / uds 1   View in Datadog   GitHub Actions

🧪 8 Tests failed

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic.test_ffe_evp_flagevaluation_basic[uds] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-basic-flag
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_evp_flagevaluation_event.<locals>.<lambda> at 0x7fa2dee751c0>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic object at 0x7fa31040fbf0>

    def test_ffe_evp_flagevaluation_basic(self) -> None:
        assert self.r.status_code == 200, f"Flag evaluation failed: {self.r.text}"
...
tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Burst_Aggregation.test_ffe_evp_flagevaluation_burst_aggregation[uds] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-burst-aggregation-flag
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_evp_flagevaluation_event.<locals>.<lambda> at 0x7fa2dee5c2c0>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Burst_Aggregation object at 0x7fa3108cf410>

    def test_ffe_evp_flagevaluation_burst_aggregation(self) -> None:
        for index, response in enumerate(self.responses):
...
View all 8 test failures

Testing the test | all-jobs-are-green   View in Datadog   GitHub Actions

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 43c3969 | Docs | Datadog PR Page | Give us feedback!

@datadog-system-tests-org

datadog-system-tests-org Bot commented Jun 20, 2026

Copy link
Copy Markdown

Pipelines

⚠️ Warnings

🚦 3 Pipeline jobs failed

Testing the test | System Tests (dotnet, dev) / End-to-end #1 / poc 1   View in Datadog   GitHub Actions

Testing the test | System Tests (dotnet, dev) / End-to-end #1 / uds 1   View in Datadog   GitHub Actions

Testing the test | all-jobs-are-green   View in Datadog   GitHub Actions

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 43c3969 | Docs | Give us feedback!

Base automatically changed from leo.romanovsky/ffe-evp-flagevaluation-contract-fix to main June 22, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant