Remove GenAI packages while preserving TorchSharp support - #7738
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7738 +/- ##
==========================================
+ Coverage 69.89% 70.26% +0.36%
==========================================
Files 1487 1419 -68
Lines 276323 272569 -3754
Branches 28295 27938 -357
==========================================
- Hits 193142 191519 -1623
+ Misses 75694 73638 -2056
+ Partials 7487 7412 -75
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
One or more issues must be addressed before approval.
Review effort: Lite
Findings: None
What changed in this PR
Removes the GenAI packages and their exclusive support code while retaining TorchSharp, tokenizer packages, CUDA test configuration, and historical release notes.
Changes:
- Deletes GenAI source, tests, samples, configurations, approvals, and design documentation.
- Removes GenAI-only package/version entries and third-party notice.
- Updates documentation and Copilot repository guidance.
| File | Description |
|---|---|
| THIRD-PARTY-NOTICES.TXT | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Phi.Tests/SemanticKernelTests.cs | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Phi.Tests/Phi2Tests.cs | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Phi.Tests/Microsoft.ML.GenAI.Phi.Tests.csproj | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Phi.Tests/AutoGenTests.cs | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Phi.Tests/Approvals/Phi3Tests.TokenizerTest.approved.txt | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Phi.Tests/Approvals/Phi3Tests.Phi3Mini128KLayerSizeTest.approved.txt | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Phi.Tests/Approvals/Phi2Tests.TokenizerTest.approved.txt | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Phi.Tests/Approvals/Phi2Test.TokenizerTest.approved.txt | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Mistral.Tests/Microsoft.ML.GenAI.Mistral.Tests.csproj | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Mistral.Tests/Approvals/Mistral_V0_3Tests.TokenizerTest.approved.txt | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Mistral.Tests/Approvals/Mistral_7B_Instruct_V0_3Tests.TokenizerTest.approved.txt | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Mistral.Tests/Approvals/Mistral_7B_Instruct_V0_3Tests.ItBuildChatTemplateWithToolsFromAutoGenChatHistory.approved.txt | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Mistral.Tests/Approvals/Mistral_7B_Instruct_V0_3Tests.ItBuildChatTemplateFromAutoGenChatHistory.approved.txt | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.LLaMA.Tests/Microsoft.ML.GenAI.LLaMA.Tests.csproj | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.LLaMA.Tests/LLaMA3_2Tests.cs | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.LLaMA.Tests/LLaMA3_1Tests.TokenizerTest.received.txt | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.LLaMA.Tests/LLaMA3_1Tests.cs | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.LLaMA.Tests/Approvals/LLaMA3_1Tests.TokenizerTest.approved.txt | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.LLaMA.Tests/Approvals/LLaMA3_1Tests.ItBuildChatTemplateFromSemanticKernelChatHistory.approved.txt | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.LLaMA.Tests/Approvals/LLaMA3_1Tests.ItBuildChatTemplateFromMEAIChatHistory.approved.txt | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.LLaMA.Tests/Approvals/LLaMA3_1Tests.ItBuildChatTemplateFromAutoGenChatHistory.approved.txt | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Core.Tests/QuantizedLinearTests.cs | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Core.Tests/Microsoft.ML.GenAI.Core.Tests.csproj | Updated as part of this pull request. |
| test/Microsoft.ML.GenAI.Core.Tests/CausalLMDatasetTest.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Utils.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Resource/README.md | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Resource/Config/phi-3-mini-4k-instruct-config.json | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Resource/Config/phi-3-medium-4k-instruct-config.json | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Resource/Config/phi-2-config.json | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/README.md | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Phi3/Phi3TokenizerHelper.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Phi3/Phi3ForCausalLM.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Phi3/Phi3ChatTemplateBuilder.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Phi3/Phi3CausalLMTextGenerationService.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Phi3/Phi3CausalLMChatCompletionService.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Phi3/Phi3CausalLMChatClient.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Phi3/Phi3CausalLMAgent.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Phi2/Phi2TokenizerHelper.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Phi2/Phi2ForCausalLM.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Phi2/Config.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Module/Phi3SuScaledRotaryEmbedding.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Module/Phi3Model.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Module/Phi3MLP.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Module/Phi2RotaryEmbedding.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Module/Phi2MLP.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Module/Phi2DecoderLayer.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Microsoft.ML.GenAI.Phi.csproj | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Phi/Extension/SemanticKernelExtension.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Mistral/Resource/Config/mistral-7B-instruct-v0.3.json | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Mistral/MistralTokenizerHelper.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Mistral/MistralModel.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Mistral/MistralMLP.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Mistral/MistralForCausalLM.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Mistral/MistralDecoderLayer.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Mistral/MistralConfig.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Mistral/Microsoft.ML.GenAI.Mistral.csproj | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.LLaMA/Resource/Config/meta-llama-3.2-3B-Instruct.json | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.LLaMA/Resource/Config/meta-llama-3.2-1B-Instruct.json | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.LLaMA/Resource/Config/meta-llama-3.1-8B-Instruct.json | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.LLaMA/Resource/Config/meta-llama-3.1-70B-Instruct.json | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.LLaMA/Resource/Config/meta-llama-3.1-405B-Instruct.json | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.LLaMA/Module/LlamaModel.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.LLaMA/Module/LlamaMLP.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.LLaMA/Module/LlamaDecoderLayer.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.LLaMA/Microsoft.ML.GenAI.LLaMA.csproj | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.LLaMA/LlamaTokenizerHelper.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.LLaMA/LlamaTextCompletionService.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.LLaMA/LlamaChatCompletionService.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.LLaMA/LlamaCausalLMAgent.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.LLaMA/Llama3CausalLMChatClient.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.LLaMA/Llama3_1ChatTemplateBuilder.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/Utility/IChatTemplateBuilder.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/Utility/Cache.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/Trainer/CausalLMSupervisedFineTuningTrainer.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/Trainer/CausalLMDataset.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/Pipeline/CausalLMModelOutput.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/Pipeline/CausalLMModelInput.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/Module/RotaryEmbedding.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/Module/RMSNorm.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/Module/NewGELUActivation.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/Module/IQuantizeModule.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/Module/IDynamicLoadModule.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/Module/GenAILinear.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/Module/DynamicLoadingModule.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/Microsoft.ML.GenAI.Core.csproj | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/Extension/TensorExtension.cs | Updated as part of this pull request. |
| src/Microsoft.ML.GenAI.Core/CausalLMPipelineChatClient.cs | Updated as part of this pull request. |
| eng/Versions.props | Updated as part of this pull request. |
| docs/samples/Microsoft.ML.GenAI.Samples/Program.cs | Updated as part of this pull request. |
| docs/samples/Microsoft.ML.GenAI.Samples/Phi3Mini/SemanticKernelSample.cs | Updated as part of this pull request. |
| docs/samples/Microsoft.ML.GenAI.Samples/Phi3Mini/AutoGenSample.cs | Updated as part of this pull request. |
| docs/samples/Microsoft.ML.GenAI.Samples/Microsoft.ML.GenAI.Samples.csproj | Updated as part of this pull request. |
| docs/samples/Microsoft.ML.GenAI.Samples/MEAI/Phi3.cs | Updated as part of this pull request. |
| docs/samples/Microsoft.ML.GenAI.Samples/MEAI/Llama3_1.cs | Updated as part of this pull request. |
| docs/samples/Microsoft.ML.GenAI.Samples/Llama/SFT_Llama_3_2_1B.cs | Updated as part of this pull request. |
| docs/samples/Microsoft.ML.GenAI.Samples/Llama/LlamaSample.cs | Updated as part of this pull request. |
| docs/README.md | Updated as part of this pull request. |
| docs/project-docs/components-and-dependencies.md | Updated as part of this pull request. |
| docs/gen-ai/Usage.md | Updated as part of this pull request. |
| docs/gen-ai/Tokenizer.md | Updated as part of this pull request. |
| docs/gen-ai/README.md | Updated as part of this pull request. |
| docs/gen-ai/Package Structure.md | Updated as part of this pull request. |
| docs/gen-ai/DynamicLoading.md | Updated as part of this pull request. |
| docs/gen-ai/CausalLMPipeline.md | Updated as part of this pull request. |
| docs/gen-ai/CausalLanguageModel.md | Updated as part of this pull request. |
| docs/gen-ai/Benchmark && Evaluation.md | Updated as part of this pull request. |
| Directory.Packages.props | Updated as part of this pull request. |
| .github/copilot-instructions.md | Updated as part of this pull request. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
I see you have the CI errors solved, so we can eventually merge this PR and abandon #7719. |
Remove the GenAI libraries, tests, samples, model data, and exclusive dependencies from current main. Preserve the TorchSharp CUDA test setup and document the boundary between source removal and published package support. Related to #7720 and #7719. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove the package-removal section and its index link after the initial CI runs completed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
806f9ec to
64968f0
Compare
Summary
Fixes #7720.
Implement the removal proposed in #7719 on current
main. This is a separate PR; #7719 remains unchanged and open.Microsoft.ML.GenAI.*libraries, four test projects, sample project, model configurations, approval files, and design documents.Microsoft.ML.TorchSharp,Microsoft.ML.Tokenizers.*, shared dependency versions, and the existing CUDA test setup. In particular, retain both theTorchSharp-cuda-windowsversion and its commented test-project reference.