Add Foundry Local as a chat and embedding provider in the AI Chat Web template - #7586
Add Foundry Local as a chat and embedding provider in the AI Chat Web template#7586luisquintanilla wants to merge 6 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Adds Foundry Local as a selectable chat + embeddings provider for the AI Chat Web project template, including generated configuration, package references, and a new snapshot test/verified snapshot for the provider.
Changes:
- Adds
--provider foundrylocaltemplate option and wires Foundry Local startup + OpenAI-compatible clients in the generated app. - Updates template defaults/config for Foundry Local (models + endpoint) and adjusts embedding vector dimensions accordingly.
- Registers the new NuGet dependency and expands snapshot testing to cover the new provider.
Reviewed changes
Copilot reviewed 46 out of 46 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Services/SemanticSearch.cs | Verified snapshot output for Foundry Local variant (semantic search service). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Services/Ingestion/PdfPigReader.cs | Verified snapshot output for Foundry Local variant (PDF ingestion reader). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Services/Ingestion/DocumentReader.cs | Verified snapshot output for Foundry Local variant (document reader dispatch). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Services/Ingestion/DataIngestor.cs | Verified snapshot output for Foundry Local variant (ingestion pipeline wiring). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Services/IngestedChunk.cs | Verified snapshot output for Foundry Local variant (vector schema/dimensions). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/README.md | Verified snapshot output for Foundry Local setup instructions. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Properties/launchSettings.json | Verified snapshot output with fixed ports for deterministic testing. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Program.cs | Verified snapshot output for Foundry Local runtime wiring. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Routes.razor | Verified snapshot output (routing). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Error.razor | Verified snapshot output (error page). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/ChatSuggestions.razor.css | Verified snapshot output (chat UI styles). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/ChatSuggestions.razor | Verified snapshot output (chat suggestions component). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/ChatMessageList.razor.js | Verified snapshot output (chat message list JS). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/ChatMessageList.razor.css | Verified snapshot output (chat message list styles). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/ChatMessageList.razor | Verified snapshot output (chat message list component). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/ChatMessageItem.razor.css | Verified snapshot output (chat message item styles). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/ChatMessageItem.razor | Verified snapshot output (chat message item rendering/citations). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/ChatInput.razor.js | Verified snapshot output (chat input JS). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/ChatInput.razor.css | Verified snapshot output (chat input styles). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/ChatInput.razor | Verified snapshot output (chat input component). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/ChatHeader.razor.css | Verified snapshot output (chat header styles). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/ChatHeader.razor | Verified snapshot output (chat header component). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/ChatCitation.razor.css | Verified snapshot output (citation styles). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/ChatCitation.razor | Verified snapshot output (citation link construction). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/Chat.razor.css | Verified snapshot output (chat page styles). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Pages/Chat/Chat.razor | Verified snapshot output (chat page + tool usage). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Layout/SurveyPrompt.razor.css | Verified snapshot output (survey prompt styles). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Layout/SurveyPrompt.razor | Verified snapshot output (survey prompt component). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Layout/MainLayout.razor.css | Verified snapshot output (layout styles). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Layout/MainLayout.razor | Verified snapshot output (main layout). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Layout/LoadingSpinner.razor.css | Verified snapshot output (spinner styles). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/Layout/LoadingSpinner.razor | Verified snapshot output (spinner component). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/App.razor | Verified snapshot output (app host markup). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/Components/_Imports.razor | Verified snapshot output (Razor imports). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/appsettings.json | Verified snapshot output (Foundry Local configuration). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/appsettings.Development.json | Verified snapshot output (Foundry Local dev configuration). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.foundryl_l_webHttpPort_9996_webHttpsPort_9995.verified/aichatweb/aichatweb.csproj | Verified snapshot output (Foundry Local + AI.OpenAI package references). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/AIChatWebSnapshotTests.cs | Adds snapshot coverage for --provider foundrylocal. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.Web/Services/IngestedChunk.cs | Sets embedding vector dimensions for Foundry Local outputs. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.Web/README.md | Documents Foundry Local setup path in non-Aspire template output. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.Web/Program.cs | Adds Foundry Local startup + OpenAI-compatible chat/embedding wiring. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.Web/appsettings.json | Adds Foundry Local configuration block (conditional). |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.Web/appsettings.Development.json | Adds Foundry Local dev configuration block (conditional). |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.Web/AIChatWeb-CSharp.Web.csproj-in | Adds Foundry Local package references for non-Aspire outputs. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/.template.config/template.json | Adds the new provider choice + computed symbol + default model symbols. |
| eng/packages/ProjectTemplates.props | Registers Microsoft.AI.Foundry.Local package version for templates. |
| { | ||
| "choice": "foundrylocal", | ||
| "displayName": "Foundry Local (on-device)", | ||
| "description": "Uses Foundry Local with the qwen3-4b and qwen3-embedding-0.6b models" | ||
| } |
There was a problem hiding this comment.
Making the options mutually exclusive is good. We can follow a different approach though, similar to how we handle qdrant.
🎉 Good job! The coverage increased 🎉
Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1482171&view=codecoverage-tab |
jeffhandley
left a comment
There was a problem hiding this comment.
I've got a local session to incorporate this feedback and to make Foundry Local the default, replacing GitHub models. I will push to your branch.
| async Task<string> EnsureFoundryModelAsync(string alias) | ||
| { | ||
| var model = await foundryCatalog.GetModelAsync(alias) | ||
| ?? throw new InvalidOperationException( | ||
| $"Foundry Local model '{alias}' was not found in the catalog. Run 'foundry model list' to see available models."); | ||
| if (!await model.IsCachedAsync()) | ||
| { | ||
| Console.WriteLine($"Foundry Local: downloading model '{alias}' (first run only)..."); | ||
| await model.DownloadAsync(_ => { }); | ||
| } | ||
| if (!await model.IsLoadedAsync()) | ||
| { | ||
| await model.LoadAsync(); | ||
| } | ||
| return model.Id; | ||
| } |
There was a problem hiding this comment.
| async Task<string> EnsureFoundryModelAsync(string alias) | |
| { | |
| var model = await foundryCatalog.GetModelAsync(alias) | |
| ?? throw new InvalidOperationException( | |
| $"Foundry Local model '{alias}' was not found in the catalog. Run 'foundry model list' to see available models."); | |
| if (!await model.IsCachedAsync()) | |
| { | |
| Console.WriteLine($"Foundry Local: downloading model '{alias}' (first run only)..."); | |
| await model.DownloadAsync(_ => { }); | |
| } | |
| if (!await model.IsLoadedAsync()) | |
| { | |
| await model.LoadAsync(); | |
| } | |
| return model.Id; | |
| } | |
| async Task<string> EnsureFoundryModelAsync(string modelAlias) | |
| { | |
| var model = await foundryCatalog.GetModelAsync(modelAlias) | |
| ?? throw new InvalidOperationException( | |
| $"Foundry Local model '{modelAlias}' was not found in the catalog. Run 'foundry model list' to see available models."); | |
| if (!await model.IsCachedAsync()) | |
| { | |
| Console.WriteLine($"Foundry Local: downloading model '{modelAlias}' (first run only)..."); | |
| await model.DownloadAsync(_ => { }); | |
| } | |
| if (!await model.IsLoadedAsync()) | |
| { | |
| await model.LoadAsync(); | |
| } | |
| return model.Id; | |
| } |
| { | ||
| "choice": "foundrylocal", | ||
| "displayName": "Foundry Local (on-device)", | ||
| "description": "Uses Foundry Local with the qwen3-4b and qwen3-embedding-0.6b models" | ||
| } |
There was a problem hiding this comment.
Making the options mutually exclusive is good. We can follow a different approach though, similar to how we handle qdrant.
…plate - Emit a build-time #error from Program.Aspire.cs when IsFoundryLocal is set, covering the Qdrant vector store which forces Aspire. - Add foundrylocal to the execution test provider matrix and skip foundrylocal combined with Aspire or Qdrant. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ichatweb template Update the matching verified snapshot to keep it in sync. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… the aichatweb template GitHub Models is no longer available to new customers: https://github.blog/changelog/2026-06-16-github-models-is-no-longer-available-to-new-customers/ - Set the AiServiceProvider default to foundrylocal and remove the githubmodels choice and the IsGHModels symbol. - Remove GitHub Models wiring from Program.cs, Program.Aspire.cs, AppHost.cs, the project file, and both READMEs. - Remove githubmodels from the execution test matrix and switch the Aspire snapshot case to --provider=openai. - Regenerate the default snapshot for Foundry Local, add the openai+Aspire snapshot, and remove the orphaned GitHub Models Aspire snapshot. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Microsoft.AI.Foundry.Local.Core package injects a RuntimeIdentifier at build time, so restore must be RID-aware for the generated project to resolve its per-RID native assets. Declare RuntimeIdentifiers on the Foundry Local project and update the default and explicit Foundry Local snapshots. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Heads-up: reasoning models render an inline
|
🎉 Good job! The coverage increased 🎉
Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1500131&view=codecoverage-tab |
IngestedChunk.Key was always Guid, but only Qdrant uses a Guid key; the local (SQLite) and Azure AI Search stores use string keys (VectorStoreWriter<string>, the collection registrations, and SemanticSearch), so the generated app threw at runtime on those stores. Make Key conditional (Guid for Qdrant, string otherwise) and regenerate the affected snapshots. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| #if (IsQdrant) | ||
| [VectorStoreKey(StorageName = "key")] | ||
| [JsonPropertyName("key")] | ||
| public required Guid Key { get; set; } | ||
| #else | ||
| [VectorStoreKey(StorageName = "key")] | ||
| [JsonPropertyName("key")] | ||
| public required string Key { get; set; } | ||
| #endif |
There was a problem hiding this comment.
@adamsitnik Does this match your understanding that only Qdrant uses Guid keys?
There was a problem hiding this comment.
My understanding is that Azure Foundry local provides an embedding generator (hence to proper VectorDimensions change above) but I don't think it provides any vector store. So we should not change the Key here.
🎉 Good job! The coverage increased 🎉
Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1500536&view=codecoverage-tab |
adamsitnik
left a comment
There was a problem hiding this comment.
PTAL at my comments, I think that all we need to update for the ingestion is VectorDimensions count, as I expect that Azure Foundry Local does not come with a vector store.
| #if (IsQdrant) | ||
| [VectorStoreKey(StorageName = "key")] | ||
| [JsonPropertyName("key")] | ||
| public required Guid Key { get; set; } | ||
| #else | ||
| [VectorStoreKey(StorageName = "key")] | ||
| [JsonPropertyName("key")] | ||
| public required string Key { get; set; } | ||
| #endif |
There was a problem hiding this comment.
My understanding is that Azure Foundry local provides an embedding generator (hence to proper VectorDimensions change above) but I don't think it provides any vector store. So we should not change the Key here.
| This project is configured to use Foundry Local, which runs models on your workstation through a local OpenAI-compatible endpoint. It does not need an API key. | ||
|
|
||
| ### 1. Install Foundry Local | ||
| Install Foundry Local for your operating system by following the [Foundry Local documentation](https://learn.microsoft.com/azure/ai-foundry/foundry-local/). |
There was a problem hiding this comment.
I think this is not needed, is also mentioned in the SDK reference docs:
The SDK doesn't require the Foundry Local CLI to be installed
|
This submission has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 14 days. It will be closed if no further activity occurs within 7 days of this comment. |
Fixes #7585
What
Adds Foundry Local as a chat and embedding provider option in the AI Chat Web template
(
Microsoft.Extensions.AI.Templates,AIChatWeb-CSharp). Foundry Local is Microsoft'son-device inference runtime, so this gives the template a Microsoft-owned local option
alongside the existing third-party Ollama path. Both the chat
IChatClientand the embeddingIEmbeddingGeneratorare wired.Select it with
--provider foundrylocal(or the "Foundry Local (on-device)" choice in theinteractive flow).
How it works
Foundry Local is OpenAI-compatible, so it reuses the template's existing
OpenAIClient -> AsIChatClient() / AsIEmbeddingGenerator()shape. The generated app usesFoundryLocalManagerto start the local web service and load the models, then points thestandard official OpenAI client at the local endpoint (keyless on localhost).
Defaults:
qwen3-4bfor chat (Apache-2.0, tool-calling capable),qwen3-embedding-0.6bforembeddings (1024-dimension vectors). Both model ids are overridable via template parameters.
Changes
template.json: newfoundrylocalchoice,IsFoundryLocalsymbol, default modelparameters (
qwen3-4b/qwen3-embedding-0.6b).Program.cs:#elif (IsFoundryLocal)wiring block..csproj-in: addsMicrosoft.AI.Foundry.LocalandMicrosoft.Extensions.AI.OpenAI.eng/packages/ProjectTemplates.props: registersMicrosoft.AI.Foundry.Local1.2.3.Services/IngestedChunk.cs:VectorDimensionsbecomes per-provider (1024 for Foundry Local).README.md,appsettings*.json: Foundry Local notes.Testing
store) against Foundry Local.
References
AddFoundryLocalhosting integration):https://github.com/luisquintanilla/foundry-local-aichatweb/tree/aspire
Microsoft Reviewers: Open in CodeFlow