Skip to content

csharp-code-coverage.md

Latest commit

 

History

History
90 lines (61 loc) · 4.61 KB

csharp-code-coverage.md

File metadata and controls

90 lines (61 loc) · 4.61 KB
Raw

C# Code Coverage with Coverlet

This document outlines the integration of Coverlet, a cross-platform code coverage framework for .NET, into the C# test execution process for the AgentSimMiddleware project.

Overview

Code coverage helps assess the quality of tests by indicating the percentage of code exercised by those tests. By integrating Coverlet with dotnet test, we can automatically generate coverage reports in a standard format (like Cobertura XML) as part of our continuous integration pipeline.

Integration Steps

  1. Add Coverlet.Collector NuGet Package: The Coverlet.Collector NuGet package has been added to the tests/csharp/AgentSimMiddleware.Tests/AgentSimMiddleware.Tests.csproj project. This package enables Coverlet to collect code coverage data during test execution.

    <ItemGroup>
  <PackageReference Include="MSTest" Version="4.0.1" />
  <PackageReference Include="Coverlet.Collector" Version="6.0.0" />
</ItemGroup>

  2. C# Project Reference in Test Project: To ensure the test project can correctly build and test the C# components of AgentSimMiddleware, the TestNetStandardLibrary.csproj has been updated to include the C# source files from src/csharp. This allows AgentSimMiddleware.Tests.csproj to reference TestNetStandardLibrary, which now encompasses all relevant C# logic including the UnityAdapter.

    The TestNetStandardLibrary.csproj now includes:

    <ItemGroup>
  <Compile Include="..\src\csharp\**\*.cs" />
</ItemGroup>

    And the AgentSimMiddleware.Tests.csproj references it:

    <ItemGroup>
  <ProjectReference Include="..\..\..\TestNetStandardLibrary\TestNetStandardLibrary.csproj" />
</ItemGroup>

  3. Correct TargetFramework: The TargetFramework for AgentSimMiddleware.Tests.csproj has been set to net8.0 for compatibility with the current .NET SDK and its dependencies.

XML Documentation Generation

To enable C# documentation coverage analysis, all C# projects are configured to generate XML documentation files (.xml) during their build process. This is achieved by setting the <GenerateDocumentationFile>true</GenerateDocumentationFile> property in the <PropertyGroup> section of each .csproj file.

The following projects now generate XML documentation:

  • benchmarker/benchmarker.csproj
  • TestNetStandardLibrary/TestNetStandardLibrary.csproj
  • tests/csharp/AgentSimMiddleware.Tests/AgentSimMiddleware.Tests.csproj

Verification steps confirmed that .xml files are generated in the respective bin/Debug (or bin/Release) folders alongside the compiled assemblies. For example:

  • TestNetStandardLibrary/bin/Debug/netstandard2.1/TestNetStandardLibrary.xml
  • benchmarker/bin/Debug/net8.0/benchmarker.xml
  • tests/csharp/AgentSimMiddleware.Tests/bin/Debug/net8.0/AgentSimMiddleware.Tests.xml

run_csharp_tests.ps1 Script

A PowerShell script, tools/run_csharp_tests.ps1, has been created to automate the execution of C# tests with code coverage collection.

Script Parameters

  • -ProjectPath: (Optional) The relative path to the C# test project (AgentSimMiddleware.Tests.csproj). Defaults to tests/csharp/AgentSimMiddleware.Tests/AgentSimMiddleware.Tests.csproj.
  • -ResultsDirectory: (Optional) The relative path where test results and coverage reports should be stored. Defaults to coverage_report/csharp.

How it Works

The script performs the following actions:

  1. Cleans the specified test project (dotnet clean).
  2. Restores NuGet packages for the test project (dotnet restore).
  3. Executes dotnet test with the following key arguments:
    • --collect:"XPlat Code Coverage": This argument instructs Coverlet to collect code coverage data.
    • --results-directory "<path>": Specifies the output directory for test results and coverage reports.
    • --logger:"trx;LogFileName=test_results.trx": Generates a TRX format test results file.
    • --logger:"html;LogFileName=test_results.html": Generates an HTML test results file.

Usage

To run the C# tests and collect code coverage, execute the PowerShell script from the project root:

./tools/run_csharp_tests.ps1

Coverage reports (e.g., coverage.cobertura.xml) will be generated in a subdirectory within coverage_report/csharp. The exact path will include a GUID (e.g., coverage_report/csharp/<GUID>/coverage.cobertura.xml).

Verification

After running the script, verify the presence of a coverage.cobertura.xml file in the specified results directory to confirm successful coverage data generation.