Repository navigation
Update to latest script version; verify dependency versions - #1944
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Eager NuGet version resolution during package definition can block unrelated targets such as Clean during a feed outage, even when tools are cached.
1 open issue
What changed in this PR
This PR advances #1943 by updating the Cake recipe and adding latest-version dependency checks for NUnit packages and bundled agents.
Changes:
- Upgrade NUnit.Cake.Recipe to
2.0.0-beta.5.2. - Check dependency versions and expand console package content validation.
- Make bundled-agent references readonly and display recipe version badges.
| File | Description |
|---|---|
| README.md | Add NuGet and MyGet version badges for NUnit.Cake.Recipe. |
| KnownExtensions.cake | Make agent fields and the bundled-agent array readonly; document version defaults. |
| build.cake | Upgrade the recipe and strengthen dependency and package-content checks. |
Review effort: Balanced
| HasDirectory("lib/net462").WithFile("nunit.common.dll"), | ||
| HasDirectory("lib/netstandard2.0").WithFile("nunit.common.dll"), | ||
| HasDependency("NUnit.Engine.Api") | ||
| HasDependency("NUnit.Engine.Api").LatestDevBuild |
There was a problem hiding this comment.
@copilot Show me a brief example of code to defer version resolution until DependencyCheck.ApplyTo runs
There was a problem hiding this comment.
For example, DependencyCheck can store a resolver instead of calling NuGet in the selector getter:
private Func<string> _versionResolver;
public DependencyCheck LatestDevBuild
{
get
{
_versionResolver = () => new PackageReference(_packageId).LatestDevBuild.Version;
return this;
}
}
public override bool ApplyTo(DirectoryPath testDirPath)
{
_packageVersion ??= _versionResolver?.Invoke();
// existing check logic...
}LatestStableRelease would set the same resolver using PackageReference(...).LatestStableRelease.Version. This keeps the existing call site, but performs feed access only when ApplyTo runs.
oznetmaster
left a comment
There was a problem hiding this comment.
@CharliePoole, I checked the published NUnit.Cake.Recipe 2.0.0-beta.5.3 payload, including the deferred DependencyCheck correction. That part now defers correctly, but I found three remaining points:
- The agent checks in build.cake still resolve during package definition. ExtensionSpecifier.LatestStableRelease calls PackageReference.LatestStableRelease immediately. Use the deferred DependencyCheck selector instead, for example:
HasDependency(KnownExtensions.Net462PluggableAgent.NuGetPackage).LatestStableReleaseThe same change applies to the other three agent checks. The current four expressions make four feed calls before any verification runs, so the earlier concern about unrelated targets depending on feed availability is only partly resolved.
-
In the recipe, PackageReference.LatestReleaseIncludingPreReleases uses DEVELOPMENT_SOURCE, making it equivalent to LatestDevBuild. Based on the policy described in #1943, it should query PRODUCTION_SOURCE with prereleases enabled. This selector is not used by the new console call sites, but it is incorrect in the recipe API supplied by this update.
-
If a configured latest-version lookup finds no matching package, its version stays null. DependencyCheck.ApplyTo then treats the check as intentionally unversioned and can accept a single older installed version. An explicitly requested latest policy should fail with a resolution diagnostic in this case; an ordinary HasDependency with no version should retain its existing existence-check behavior.
Validation: an offline harness compiling the published recipe classes against fake feed/filesystem boundaries reproduced all three issues. Three cases failed before the isolated corrections; all six cases passed afterward, including controls for deferred verification and intentionally unversioned dependencies. This was not a full Cake invocation or a real feed-outage test. No maintained source or published package was modified.
Prepared by Neil Colvin with assistance from OpenAI Codex.
|
@oznetmaster
This PR will be marked as a draft until the recipe work is complete. Delaying this PR shouldn't cause any serious merge problems, since the remaining changes will be limited to the I like the idea of a validation harness for recipe changes. That would keep us from having to update myget in order to actually test. Let's discuss making that permanent. Next step after I convert this to a draft: an issue or issues in |
|
@CharliePoole, I verified the updated agent call sites now use the deferred DependencyCheck selectors. That addresses the console-side portion of the first finding. Moving the remaining fixes into recipe PRs makes sense. For the permanent harness, I suggest compiling the maintained recipe sources directly, with controlled feed/filesystem boundaries, so regressions can be tested without first publishing to MyGet. We should retain a separate Cake smoke test rather than treating the isolated harness as complete integration coverage. Prepared by Neil Colvin with assistance from OpenAI Codex. |
|
@oznetmaster If I expected future conflicts, I'd merge this now, but otherwise I see no need. We'll wait for the fixes in the recipe and then apply them here as needed. |
|
@CharliePoole, the recipe follow-up is now validated against the published NUnit.Cake.Recipe 2.0.0-beta.5.8 package following merged recipe PRs #135 and #136. I checked the downloaded package against recipe main d546405: all 29 recipe source files match. The maintained harness, loading the public package instead of local recipe source, passes all 104 cases. In an isolated checkout of #1944 at 9f4d385, I changed only the #load version from beta.5.3 to beta.5.8. The real Package target builds all four packages, installs them and passes all six package/symbol verification stages, including the deferred API/extensibility and stable-agent dependency checks. The original three review findings are addressed by your updated call sites and the recipe corrections. For completeness, the broader execution results are qualified: all 52 Framework-console package scenarios pass. The modern console passes 37 of 52, with 15 failures. Repeating both console suites with the original beta.5.3 recipe and exactly the same built packages gives identical per-scenario outcomes. These failures therefore reproduce independently of adopting beta.5.8; I am not claiming a fully passing modern-console package suite. The failure evidence includes Framework-agent launch-path errors and agent compatibility rejection for the .NET 7 cases; those need separate investigation before assigning causes or proposing fixes. The minimal follow-up here is updating the recipe #load to 2.0.0-beta.5.8. No engine, launcher, package checks or test expectations were changed for this validation. Publication was disabled and the local feed was confined to the isolated validation directory. Native Linux/macOS validation is not claimed. Prepared by Neil Colvin with assistance from OpenAI Codex. |
|
@CharliePoole, the local-execution failures described in my earlier recipe validation now have separate, tested corrections: Framework-agent #15 fixes the device-prefixed executable paths, and console/engine #1947 makes launcher metadata selection use the resolved execution runtime. Both draft PRs have passing hosted CI and CLA checks. The combined isolated #1944 validation passes all 52 Framework-console scenarios. Its modern-console suite passes 49/52 with this workstation's normal runtime catalogue; the three remaining comparisons expect older TargetFrameworkName values. All 52 pass with the expected legacy catalogue entries supplied privately, while the actual workers for those legacy-target cases remain .NET 8. No package assertion was weakened or global runtime installation changed. The existing comparison reads TargetFrameworkName, so I am keeping selected-target reporting separate from actual worker-CLR coverage. This does not change the recipe conclusion: beta.5.8 passed the package and dependency checks, and those earlier execution failures reproduced with beta.5.3 as well. The recipe-pin follow-up can remain separate from the two execution corrections. Prepared by Neil Colvin with assistance from OpenAI Codex. |
|
@oznetmaster Are you still requesting changes in this PR as indicated by GitHub? I can override and merge it anyway if not. |
|
@CharliePoole, your call-site changes address the console-side finding. The remaining recipe corrections are published in NUnit.Cake.Recipe 2.0.0-beta.5.8, but this PR still loads beta.5.3. Please update that pin to beta.5.8. We have already validated that combination in an isolated copy, including package and dependency verification. Once the updated head passes CI, I can replace the changes-requested review with approval; there should be no need to override it. Prepared by Neil Colvin with assistance from OpenAI Codex. |
|
OK... the status just was not clear. |

Step 2 of issue #1943
Verification determines whether the correct expected version is installed. Consistency among various files is not checked. This will be done as a separate step.