[DRAFT] Add SDK v2 telemetry and hardening by bmehta001 · Pull Request #878 · microsoft/Foundry-Local

bmehta001 · 2026-07-13T17:34:35Z

Summary

Adds SDK v2 telemetry plumbing and 1DS event emission across manager, catalog, downloads, EP registration, model/session usage, and web-service routes.
Hardens related packaging/runtime paths discovered during review: WinML/DirectML staging, ORT/GenAI runtime copies, download resume safety, streaming shutdown, and JS/Python native lifetime/loading.
Merged latest origin/main and preserved platform file I/O split with explicit close-error handling.

Validation

python sdk_v2/cpp/build.py --build --config RelWithDebInfo
Focused C++ tests: FileWriter, DownloadManager retry/legacy repair, AzureBlobDownloaderResume, SessionManager/WebService/SSE, EP/catalog focused tests.
npm run build in sdk_v2/js
JS manager/catalog disposal focused tests.

Notes

Draft because the branch is large; planning split follow-up PRs to separate telemetry from hardening/packaging fixes.

Port the C# Foundry Core telemetry event taxonomy to C++ in advance of the 1DS bridge. The interface refactor is the source of truth — all backend implementations and call sites are wired against it. ITelemetry additions: - 4 new `Action` values (EpDownloadAttempt, EpDownloadAndRegister, ModelFileDownload, ModelInference). - 4 new payload structs (`EpDownloadAttemptInfo`, `EpDownloadAndRegisterInfo`, `ModelUsageInfo`, `DownloadInfo`). - 3 new virtual methods (`RecordEpDownloadAttempt`, `RecordEpDownloadAndRegister`, `RecordDownload`). - Replaced `RecordModelUsage(model_id, prompt_tokens, completion_tokens, duration_ms)` with `RecordModelUsage(const ModelUsageInfo&)` so callers can populate richer fields (TimeToFirstToken, EP, memory). - Extended `RecordModelId(action, model_id)` to take `status` and `user_agent`; added `RecordException(action, ex, user_agent)` overload. New supporting code (all in `sdk_v2/cpp/src/telemetry/`): - `telemetry_environment.{h,cc}` — `IsCiEnvironment` / `IsTestingMode` / `IsTruthyValue` / `GetEnv`. 13 CI env-var names ported verbatim from neutron-server's `TelemetryEnvironment.cs`. - `telemetry_metadata.{h,cc}` — captures per-process metadata (`app_session_guid`, version, os_name / os_version / cpu_arch, test_mode flag) for stamping on every event. - `ep_download_tracker.{h,cc}` — RAII tracker that emits one `EPDownloadAndRegister` event per bootstrapper, recording stage transitions (initial -> download -> register). Default failure semantics: dtor records remaining stages as `kFailure`; `Done()` records them as `kSkipped` for happy-path early exit. - `download_tracker.{h,cc}` — RAII tracker that emits one `Download` event per `DownloadManager::DownloadModel` call. `TelemetryLogger` (the fallback / mirror) implements every new method, formatting events as `[Telemetry] EventName Field=value ...` at Debug. Test stubs (`NullTelemetry` and the `RecordingTelemetry` used by `telemetry_test.cc`) were updated for the new interface. Build verified RelWithDebInfo --no_telemetry. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

OneDsTelemetry is the production ITelemetry implementation. It embeds a TelemetryLogger mirror so every event is also logged locally to ILogger for diagnostics, then conditionally uploads to 1DS through the Microsoft cpp-client-telemetry SDK. Suppression model (three-state): - CI environment detected -> skip `LogManager::Initialize`; local logging still happens, but no upload occurs. - Tenant token empty (build did not pass `-DFOUNDRY_LOCAL_TELEMETRY_TOKEN`) -> same as CI: no upload, only local logging. - Otherwise -> upload. Every event is stamped with `test=true` when `FOUNDRY_TESTING_MODE` is truthy, `test=false` otherwise. Common context, propagated to every event via `ILogger::SetContext`: - `app_name` (from `Configuration::app_name`). - `app_session_guid` (random v4 UUID; on Windows the special `UTCReplace_AppSessionGuid` field also gets the OS app session GUID). - `version` (from the generated `version.h`). - `os_name` / `os_version` / `cpu_arch` (Win: `GetVersionExA` + `GetNativeSystemInfo`; POSIX: `uname`). The `MICROSOFT_KEYWORD_CRITICAL_DATA` (bit 47) policy flag is set on every event via `SetPolicyBitFlags` so the data passes 1DS classifier. Build / packaging: - `vcpkg.json`: `telemetry` feature pulls `cpp-client-telemetry`. The port is pending in microsoft/vcpkg#52316; until that merges, builds must pass `--no_telemetry`. - `CMakeLists.txt`: new `FOUNDRY_LOCAL_USE_TELEMETRY` option (default ON) and `FOUNDRY_LOCAL_TELEMETRY_TOKEN` advanced cache variable. Calls `find_package(MSTelemetry CONFIG REQUIRED)`, includes `one_ds_telemetry.cc` and `configure_file`-generates `one_ds_tenant_token.h` containing the build-time token, then links `MSTelemetry::mat` and defines `FOUNDRY_LOCAL_HAS_1DS=1`. - `build.py`: new `--no_telemetry` and `--telemetry_token` flags; forwards through to CMake / VCPKG_MANIFEST_FEATURES. Build verified RelWithDebInfo --no_telemetry; --use_telemetry will be validated once the vcpkg port lands. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Plumb the typed telemetry interface through the orchestration layer so that real workloads produce the new EPDownloadAttempt, EPDownloadAndRegister and Download events. Manager: - Construct `telemetry_` before `ep_detector_` so the detector can be handed a non-null sink at ctor time. When `FOUNDRY_LOCAL_HAS_1DS` is defined, the implementation is `OneDsTelemetry` and reads the build-time token from the generated `one_ds_tenant_token.h`; otherwise it falls back to `TelemetryLogger`. - Member-declaration order in `manager.h` reorganised so consumers (`ep_detector_`, `download_manager_`) appear after the providers (`telemetry_`). C++ destroys in reverse declaration order, so this keeps the raw `ITelemetry*` held by `EpDetector` and `DownloadManager` valid for their entire lifetime. - Explicit `~Manager()` Shutdown reset order updated to match: the telemetry sink is reset last (after ep_detector and download_manager). EpDetector: - New optional `ITelemetry* telemetry` ctor arg (default nullptr) so unit tests that instantiate EpDetector directly continue to compile. - `DownloadAndRegisterEps` emits one `EPDownloadAndRegister` event per bootstrapper via `EpDownloadTracker` and a single aggregate `EPDownloadAttempt` event at the end with attempt / success / fail counts and an overall status. - Exceptions thrown from a bootstrapper invoke `EpDownloadTracker::RecordException` before re-throwing so the failure is recorded regardless of how it propagates. DownloadManager: - New optional `ITelemetry* telemetry` ctor arg. When non-null, `DownloadModel` wraps the call with a `DownloadTracker` that captures lock-wait, enumeration and download timings, total bytes, file count, max concurrency and final status (Success / Skipped / Failure). `RecordException` is invoked on the exception path before re-throwing. - `DownloadModel` now takes an optional `user_agent` parameter so HTTP-driven downloads can attribute the event to the calling client. DownloadBlobsToDirectory: - New optional `BlobDownloadStats*` out parameter populated with `total_size_bytes`, `file_count`, `enumeration_ms`, `download_ms`. All existing 4-argument callers (including unit tests) keep working because the parameter defaults to nullptr. Verified RelWithDebInfo --no_telemetry: 756 tests pass; 54 skipped (environmental, missing local test data); 0 functional failures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Now that microsoft/vcpkg#52316 has merged, bump the manifest baseline from 256acc64 to 44819aa2 (current master) so the cpp-client-telemetry 3.10.161.1 port resolves. Also rename the using-declaration in one_ds_telemetry.cc's anonymous namespace from `using ::Microsoft::Applications::Events::ILogger;` to `using MatILogger = ::Microsoft::Applications::Events::ILogger;` so it no longer collides with the local `fl::ILogger` interface from src/logger.h. Three callsites (SetCommonContext, SafeLog, GetMatLogger) updated; the OneDsTelemetry ctor's `ILogger& logger` parameter correctly resolves to fl::ILogger after the rename. Verified with: python sdk_v2/cpp/build.py --use_telemetry --config RelWithDebInfo --skip_examples foundry_local.dll = 12.19 MB (telemetry on, empty token). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add /OPT:REF, /OPT:ICF, and /INCREMENTAL:NO to the foundry_local shared library target for Release and RelWithDebInfo configs. MSVC's /DEBUG flag (needed for PDB generation) silently disables these optimizations unless they are explicitly re-enabled, leaving all unreferenced symbols from statically-linked dependencies (notably cpp-client-telemetry/mat.lib) in the final binary. Also add -Wl,--gc-sections (Linux) and -Wl,-dead_strip (macOS) equivalents for non-Windows builds. Additionally, declare sqlite3 with default-features:false in vcpkg.json to drop the unused json1 extension from the SQLite transitive dependency. Measured impact (RelWithDebInfo, x64-windows, telemetry ON): foundry_local.dll: 12.19 MB -> 4.51 MB (-7.68 MB, -63%) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…t) foundation Reworks the telemetry core ahead of broadening route/inference coverage: - New InvocationContext {user_agent, correlation_id, indirect} threaded through the ITelemetry interface in place of the loose user_agent/indirect params. Direct() mints a fresh correlation id; AsIndirect() derives a caused-by child that reuses it. ActionTracker now carries the context and guarantees an id. - Every event (Action/Error/Model/ModelId/Download/EP*) now carries a CorrelationId so all events from one operation can be grouped. The Model event also gains Stream and Direct. - indirect now means "happened as a consequence of another action": the per-provider EPDownloadAndRegister is marked indirect and shares the overall EPDownloadAttempt's correlation id. - Add ActionStatus::kClientError to separate 4xx client rejects from 5xx/internal failures (wired into handlers in a follow-up). - Prune dead Action enum entries (kModelDownload/kModelDelete/kCoreAudioTranscribe) and add kServiceRequestUnmatched for the upcoming router catch-all. - Share the v4 UUID generator (MakeGuidV4Hex) between metadata and correlation. Builds clean (/W4 /WX); telemetry + ActionTracker unit tests pass.

…to-end - Session::ProcessRequest now emits the previously-dead Model (kModelInference) event once per inference with model id, tokens, duration, stream flag and the caller's correlation id / indirect flag. Adds Session::SetRequestContext so an HTTP route can stage an indirect child context (shared correlation id), and a virtual ExecutionProvider() hook for the EP field. - Chat completions handler: derive a Direct route context from the User-Agent header; map 4xx early-returns (empty body, invalid JSON, model not found/loaded) to kClientError instead of the default kFailure; emit kSessionCreate (indirect) on the HTTP path; stage the indirect session context. - Streaming fix: the route action is no longer recorded (as a premature success with ~0ms) when the handler returns. The route ActionTracker is moved into the streaming thread and records on completion with the real duration and terminal status, so mid-stream failures surface as failures instead of vanishing. Builds clean (/W4 /WX); telemetry/webservice/sse tests pass.

…, sampling) Extends the chat-route pattern to every handler: - Streaming completion fix also applied to the audio transcriptions route (its ActionTracker moved into the streaming thread). - Embeddings, audio, responses (create + get/list/delete/input_items), and the model load/unload/list/retrieve routes now derive a Direct context from the User-Agent header, map 4xx early-returns to kClientError, and (for the inference routes) emit kSessionCreate and stage an indirect session context. - Instrument GET /models/loaded (kModelList), previously untracked. - Sample GET /status: emit a kServiceStatus action at most once per hour per process so the orchestrator heartbeat doesn't dominate volume. Builds clean (/W4 /WX); telemetry/webservice/sse tests pass.

…ests - UnmatchedRouteInterceptor: a request interceptor that, before routing, checks the router for a matching route. When none matches (unknown path or wrong method) it records a kServiceRequestUnmatched action (kClientError) and replies 404, so requests that reach the service but no handler are no longer invisible. - Tests (WebServiceTelemetryTest, using a capturing ITelemetry + empty catalog, no real model required): * unmatched route records kServiceRequestUnmatched with the right status, user agent, Direct flag and a correlation id; * an empty chat-completions body records kClientError (not kFailure) and emits no Model event; * three rapid GET /status calls record kServiceStatus exactly once (hourly sampling). Builds clean (/W4 /WX); 72 telemetry/webservice/sse tests pass.

Every access to a model catalog source is now recorded, with success/failure: - New CatalogFetch typed event carrying operation ("FetchAll" or the cached-id "FetchByIds" lookup), endpoint/region/format parsed from the catalog URL, status, duration, model count, error message, and a correlation id shared across the accesses of one refresh. - FetchAllModelInfosWithCachedModels gains optional telemetry params (defaulted, so the snapshot tool and existing tests are unchanged) and emits an event for the primary fetch and, when it runs, the secondary cached-id lookup. - AzureModelCatalog parses the catalog URL into endpoint/region/format (https://ai.azure.com/api/eastus/ux/v1.0 -> {ai.azure.com, eastus, ux/v1.0}; "static" for the embedded snapshot) and threads an ITelemetry from Manager. - Tests assert the primary and secondary accesses are each tracked with the right operation, status, endpoint, correlation id and model count. Builds clean (/W4 /WX); catalog + telemetry + webservice tests pass.

…ions - Drop the UTCReplace_ magic from the per-process correlation GUID: it only fires on the Windows UTC transmission path (not our direct upload, and never off-Windows), so it was dead weight. The field is now plainly "AppSessionGuid" — a stable per-process correlation id on every platform. - Add ITelemetry::StartSession()/EndSession() (default no-op; OneDsTelemetry maps them to 1DS LogSession(Started/Ended), TelemetryLogger logs them). Manager opens a session when the web service starts and closes it on stop, so events carry the standard, cross-platform usage-session id (ext.app.sesId) and the backend gets session duration. This is additive — it complements the per-run AppSessionGuid and the per-operation CorrelationId. Builds clean (/W4 /WX); telemetry/webservice/catalog tests pass.