Publish generator dev versions during RegenPreview so emitter deps are restorable in CI#11280
Publish generator dev versions during RegenPreview so emitter deps are restorable in CI#11280jorgerangel-msft with Copilot wants to merge 15 commits into
Conversation
|
Azure Pipelines: Successfully started running 1 pipeline(s). 1 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
…e restorable in CI Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
commit: |
…ve opt-in parameter) Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
…ish to ADO feed only Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
…h.yml Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
…ps1 npm publish Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
…only scope Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
|
@copilot pull latest main and address the merge conflicts |
There was a problem hiding this comment.
Pull request overview
This PR restructures the http-client-csharp publish pipeline’s azure-sdk-for-net regeneration flow to support publishing generator npm packages to an ADO feed and pinning regenerated *-emitter-package.json artifacts to restorable versions (instead of host-only file: paths), so downstream azure-sdk-for-net CI can restore emitter dependencies.
Changes:
- Split
Submit-AzureSdkForNetPr.ps1into discretePrepare/BuildGenerators/Regeneratephases that share a single on-disk azure-sdk-for-net checkout and generator package folder. - Added
Get-NextGeneratorVersion+Update-EmitterPackageArtifacthelpers inRegenPreview.psm1to compute next feed versions and rewrite emitter-package artifacts for either localfile:or published-version mode. - Centralized npm publishing to the ADO feed via a reusable pipeline step template and updated emitter pipeline stages to use it.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/http-client-csharp/eng/scripts/Submit-AzureSdkForNetPr.ps1 | Adds phased execution model and wires generator build/version stamping + artifact rewriting into the PR creation flow. |
| packages/http-client-csharp/eng/scripts/RegenPreview.psm1 | Adds helper functions to compute next publish version and to rewrite emitter-package artifacts for file vs published mode. |
| packages/http-client-csharp/eng/pipeline/publish.yml | Updates the pipeline to run Prepare/BuildGenerators/Publish/Regenerate as separate steps with shared state directories. |
| eng/emitters/pipelines/templates/steps/publish-to-devops-feed.yml | Introduces a reusable step template to publish packed *.tgz packages to the DevOps npm feed. |
| eng/emitters/pipelines/templates/stages/emitter-stages.yml | Replaces inline publish script with the new reusable publish template. |
| $unbrandedTgz = Get-ChildItem -Path $BuildArtifactsPath -Filter "typespec-http-client-csharp-*.tgz" -Recurse | Select-Object -First 1 | ||
| if (-not $unbrandedTgz) { | ||
| throw "Could not find unbranded emitter .tgz in build artifacts at: $BuildArtifactsPath" | ||
| } | ||
| $unbrandedPackagePath = $unbrandedTgz.FullName | ||
| Write-Host "Using unbranded package from build artifacts: $unbrandedPackagePath" | ||
|
|
There was a problem hiding this comment.
Fixed in 5521a55. The BuildGenerators phase now copies the unbranded typespec-http-client-csharp-*.tgz into $DebugFolder and uses that staged path for the downstream Azure/mgmt generator steps.
| $azurePackageName = "azure-typespec-http-client-csharp-$azureVer.tgz" | ||
| $unbrandedPackageName = "typespec-http-client-csharp-$unbrandedVer.tgz" | ||
| $azurePackagePath = Join-Path $DebugFolder $azurePackageName | ||
| $unbrandedPackagePath = Join-Path $DebugFolder $unbrandedPackageName | ||
|
|
| $PublishRegistry = $null | ||
| $PublishNpmrcPath = $null | ||
| if ($RegenerateAzureLibraries -or $RegenerateMgmtLibraries) { | ||
| $PublishRegistry = "https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-js/npm/registry/" | ||
| $resolvedPublishNpmrc = Join-Path $PSScriptRoot "../../.npmrc" | ||
| if (Test-Path $resolvedPublishNpmrc) { | ||
| $PublishNpmrcPath = (Resolve-Path $resolvedPublishNpmrc).Path | ||
| Write-Host "Generator packages will be published to $PublishRegistry using .npmrc at $PublishNpmrcPath" | ||
| } else { | ||
| Write-Host "Generator packages will be published to $PublishRegistry using ambient npm configuration" | ||
| } |
There was a problem hiding this comment.
@copilot fix this. PublishGeneratorPackages should be on when running in CI mode only
There was a problem hiding this comment.
Fixed in 7a791a8. Submit-AzureSdkForNetPr.ps1 now has an explicit -PublishGeneratorPackages switch, and publish.yml only passes it from the CI pipeline when Azure or mgmt regeneration is requested, so local regen stays on file: references by default.
| .PARAMETER PublishRegistry | ||
| When provided, dependencies are pinned to their published versions (see PublishedDependencyVersions) | ||
| instead of a local "file:" path, and the packed .tgz is published to this registry so the resulting | ||
| emitter artifacts reference a version that CI can restore. |
…able Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
…ions Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
The regen-preview pipeline rewrites the Azure/mgmt
*-emitter-package.jsonartifacts to point at host-onlyfile:.tgzpaths, so CI on the resulting azure-sdk-for-net PR always fails to restore those dependencies (example). This adds an opt-in path to publish the locally built generator packages to the registry configured in.npmrcand reference the published version instead.Changes
RegenPreview.psm1Publish-GeneratorPackage: publishes a.tgzto a registry; treats an already-published version as success so re-runs are idempotent.Update-EmitterPackageArtifact: shared helper to rewrite an emitter-package.json + lock file in eitherfile:mode (default) or published-version mode.Update-GeneratorPackagegains-PublishRegistry/-NpmrcPath: pins deps to the published version (notfile:) and publishes the packed.tgz.Update-AzureGenerator/Update-MgmtGeneratorthread these through and use the shared helper.Submit-AzureSdkForNetPr.ps1: new-PublishGeneratorPackagesswitch that resolves the registry + authenticated.npmrcand drives publishing through the Azure/mgmt regeneration paths.publish.yml: new opt-inPublishGeneratorPackagespipeline parameter.With the switch off (default), behavior is unchanged — deps stay pinned via
file:. Publishing order (unbranded from the main stage → Azure → mgmt) ensures each package's dependencies are resolvable from the feed; the localRegenPreview.ps1loop is untouched since it cleans up and never commits.Result
The emitter artifact references a published version restorable by CI: