[Jakub Jareš (nohwnd)]

Fixes #15043

Root Cause

VSTestCLIRunSettings was declared as string[] in ITestTask, VSTestTask, and VSTestTask2. When MSBuild processes a string[] task parameter, it wraps each element in an ITaskItem. On Unix, ITaskItem.ItemSpec normalizes path separators, silently converting \ to /.

This corrupted any run setting containing backslash characters — such as regex patterns passed via:

dotnet test -- NUnit.Where='namespace =~ /Abc\.Space1($|\.)/'

The \. sequences became //, causing the regex to fail.

Fix

Changed VSTestCLIRunSettings from string[] to string in ITestTask, VSTestTask, and VSTestTask2. This bypasses ITaskItem creation entirely, so no path normalization occurs. In TestTaskUtils.CreateCommandLineArguments, the string is now split manually by newlines and semicolons (maintaining backward compatibility with both separators).

Tests

• Updated existing CreateArgumentShouldAddOneEntryForCLIRunSettings and CreateArgumentShouldAddCLIRunSettingsArgAtEnd tests to use the new string type with newline separator.
• Added CreateArgumentShouldPreserveBackslashesInCLIRunSettings that verifies backslashes in CLI run settings are not converted to forward slashes.
• Added VSTestCLIRunSettingsMustBindAsStringToSurviveUnixPathNormalization, which asserts the parameter stays a string on both tasks. That is the regression worth guarding against and it runs on every platform.

Why CI was red

RunDotnetTestWithCLIRunSettingsContainingBackslashes failed on Linux and macOS with Namespace//.Class//b. That doubled slash is what ITaskItem.ItemSpec produces, so the inner dotnet test was still running the old string[] binding even though this branch declares the property as string. I checked the binding on Linux with a small MSBuild project that takes the same value as both a string and a string[] parameter:

Parameter type	Value the task received
string	TestRunParameters.Parameter(name=\"pattern\",;value=\"Namespace\\.Class\\b\")
string[]	TestRunParameters.Parameter(name=/"pattern/", value=/"Namespace//.Class//b/")

The string[] row is exactly what CI reported, so the fix itself is correct and the test was simply not loading the task we build. It now passes /p:VSTestTaskAssemblyFile to point the VSTest targets at the task from our own package instead of whatever Microsoft.TestPlatform.Build.dll the SDK ships, and it checks that file exists first so a wrong path fails with a clear message instead of quietly falling back.

🤖 Fix submitted by Issue Repro Triage & Auto-Fix 🔍

🔍 Triaged by Issue Repro Triage & Auto-Fix 🔍

🤖

[Copilot]

Pull request overview

This PR fixes Unix-specific corruption of backslashes in VSTestCLIRunSettings by avoiding MSBuild ITaskItem path normalization for that parameter, ensuring CLI run settings (e.g., regex patterns) are passed through unchanged.

Changes:

• Changed VSTestCLIRunSettings on VSTestTask/VSTestTask2 (and ITestTask) from string[] to string to avoid MSBuild item normalization.
• Updated TestTaskUtils.CreateCommandLineArguments to manually split the run settings string by newline/semicolon and append them after --.
• Updated/added unit tests and adjusted tracked public API entries.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
test/Microsoft.TestPlatform.Build.UnitTests/TestTaskUtilsTests.cs	Updates tests to use the new string runsettings format and adds a backslash-preservation test.
src/Microsoft.TestPlatform.Build/Tasks/VSTestTask2.cs	Changes VSTestCLIRunSettings type to string? on the ToolTask-based MSBuild task.
src/Microsoft.TestPlatform.Build/Tasks/VSTestTask.cs	Changes VSTestCLIRunSettings type to string? on the Task-based MSBuild task.
src/Microsoft.TestPlatform.Build/Tasks/TestTaskUtils.cs	Implements manual splitting/handling of CLI run settings while keeping -- as the final argument group.
src/Microsoft.TestPlatform.Build/Tasks/ITestTask.cs	Updates the internal task interface to the new string? type.
src/Microsoft.TestPlatform.Build/PublicAPI/PublicAPI.Unshipped.txt	Updates the tracked public API surface to reflect the property type change.

[Jakub Jareš (nohwnd)]

🧠 Expert Review — [fix] Fix backslash normalization in VSTestCLIRunSettings on Unix

Summary

The fix is correct and well-reasoned. The root cause analysis is accurate: MSBuild wraps each element of a string[] task parameter in an ITaskItem, and on Unix, ITaskItem.ItemSpec normalizes path separators, silently converting \\ to /. Changing the property to string bypasses ITaskItem creation entirely.

Dimensions Checked

Public API Surface Protection ✅
The string[]? → string? signature change for VSTestCLIRunSettings is correctly reflected in PublicAPI.Unshipped.txt for both VSTestTask and VSTestTask2. Since this property was never in PublicAPI.Shipped.txt, there is no binary-compat break with previously released assemblies.

Backward Compatibility & Rollback Safety ✅
The .targets file in this repo invokes the task with VSTestCLIRunSettings="$(VSTestCLIRunSettings)" (a scalar MSBuild property string), not @(ItemGroup) syntax. Changing the task parameter type from string[] to string does not break this usage. MSBuild will coerce @(ItemGroup) to a semicolon-separated string when assigned to a string parameter anyway, and the new splitting logic handles that.

Null Safety & Boundary Validation ✅

• Split(['\n', ';'], StringSplitOptions.RemoveEmptyEntries) eliminates empty slots from consecutive delimiters.
• arg.Trim() handles \r in Windows-style (\r\n) line endings.
• The extra IsNullOrEmpty(trimmed) guard is redundant after RemoveEmptyEntries but defensively catches whitespace-only entries after trimming. Harmless.

Acceptance Test Coverage Design ✅
A new unit test CreateArgumentShouldPreserveBackslashesInCLIRunSettings directly validates the fix. The test uses a verbatim string @"NUnit.Where=namespace =~ /Abc\\.Space1($|\\.)/"and asserts that the literal Abc\\.Space1 (two backslashes) survives round-tripping through CreateCommandLineArguments.

Minor Observation (Non-Blocking)

Semicolon in setting values: Splitting by ; could misfire if a setting value itself contains a semicolon (e.g., SomeSetting=A;B). This is pre-existing behavior — the prior string[] design had the exact same limitation, since MSBuild also splits by ; when building ITaskItem[] for array parameters. This PR does not regress the situation, and the comment in the code explains the delimiter contract clearly.

Description Alignment ✅

The PR description accurately reflects the root cause (ITaskItem path normalization on Unix), the fix (change type to string, split manually), and the tests added. No gaps between description and diff.

---

🧠 Reviewed by Expert Code Reviewer

🧠 Reviewed by Expert Code Reviewer 🧠

[Jakub Jareš (nohwnd)]

🧠 Expert Review — [fix] Fix backslash normalization in VSTestCLIRunSettings on Unix

Summary

The fix is correct and well-scoped. Changing VSTestCLIRunSettings from string[] to string correctly bypasses MSBuild's ITaskItem path normalization. The PublicAPI.Unshipped.txt updates, the split/trim logic, and the unit tests all look sound.

Dimensions Checked

Public API Surface Protection ✅
PublicAPI.Unshipped.txt is updated for both VSTestTask and VSTestTask2. The property was never in PublicAPI.Shipped.txt, so there is no binary-compat break with released assemblies.

Backward Compatibility & Rollback Safety ✅
The .targets file passes VSTestCLIRunSettings as a scalar MSBuild property string. MSBuild coerces @(ItemGroup) to a semicolon-separated string when assigned to a string parameter, and the new split logic handles ; explicitly — preserving backward compatibility.

Null Safety & Boundary Validation ✅
Split(['\\n', ';'], RemoveEmptyEntries) + Trim() handles \r\n line endings and consecutive delimiters. The extra IsNullOrEmpty guard after RemoveEmptyEntries is redundant but harmless.

Acceptance Test Coverage Design ⚠️ (see inline comment)
The integration test is gated to Windows, but the bug only manifests on Unix. The unit test is the primary regression guard here; the integration test provides no coverage for the actual failure scenario.

Description Alignment ✅

The PR description accurately reflects the root cause, fix, and tests added. No gaps.

---

🧠 Reviewed by Expert Code Reviewer 🧠

🧠 Reviewed by Expert Code Reviewer 🧠

[Jakub Jareš (nohwnd)]

Commit pushed: efd477a

🔧 Iterated by PR Iteration Agent 🔧

[Copilot]

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

[Jakub Jareš (nohwnd)]

needs latest main merged where we run the dotnet test tests on unix as well, and re-test, as pointed out by expert reviewer.

[Jakub Jareš (nohwnd)]

Commit pushed: 01c4982

🔧 Iterated by PR Iteration Agent 🔧

[Jakub Jareš (nohwnd)]

🧠 Expert Review — [fix] Fix backslash normalization in VSTestCLIRunSettings on Unix

Summary

This iteration addresses the outstanding finding from the prior review: the integration test RunDotnetTestWithCLIRunSettingsContainingBackslashes no longer carries [TestCategory("Windows-Review")] and will now run on Linux/macOS CI — directly exercising the failure scenario the fix addresses.

All other dimensions remain sound from prior analysis.

Dimensions Checked

Public API Surface Protection ✅
PublicAPI.Unshipped.txt correctly updated for both VSTestTask and VSTestTask2. Neither property was ever in PublicAPI.Shipped.txt, so there is no binary-compat break.

Backward Compatibility & Rollback Safety ✅
The .targets file passes VSTestCLIRunSettings as a scalar MSBuild property string. MSBuild coerces @(ItemGroup) to a semicolon-separated string when assigning to a string parameter, and the split-by-; logic handles that path explicitly.

Null Safety & Boundary Validation ✅
Split(['\n', ';'], RemoveEmptyEntries) + Trim() correctly handles \r\n Windows line endings and consecutive delimiters. The extra IsNullOrEmpty(trimmed) guard after RemoveEmptyEntries is redundant but harmless.

Acceptance Test Coverage Design ✅ (prior finding resolved)
The integration test now runs on all CI platforms. The BackslashTests.cs assertion logic is correct: input Namespace\.Class\b contains backslashes and no forward slashes; if ITaskItem normalization regresses, the value becomes Namespace/.Class/b (contains /, loses \), and both assertions catch it.

Source Build & Cross-Platform Compliance ✅
The fix is purely at the MSBuild task parameter type level — no platform-specific #if guards needed. The split logic is portable.

Description Alignment ✅

PR description accurately reflects root cause, fix, and test coverage. No gaps between description and diff.

---

🧠 Reviewed by Expert Code Reviewer 🧠

🧠 Reviewed by Expert Code Reviewer 🧠

[Jakub Jareš (nohwnd)]

Commit pushed: 4bbce47

🔧 Iterated by PR Iteration Agent 🔧

[Copilot]

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

[Jakub Jareš (nohwnd)]

🧠 Expert Review — [fix] Fix backslash normalization in VSTestCLIRunSettings on Unix

Summary

The fix is correct and the core logic remains sound (same conclusion as prior reviews). One finding has regressed in the current HEAD vs the previously reviewed commit.

Dimensions Checked (activated by routing: src/Microsoft.TestPlatform.Build/, test/...IntegrationTests/, test/TestAssets/)

Public API Surface Protection ✅
PublicAPI.Unshipped.txt correctly updated for both VSTestTask and VSTestTask2. Neither property was ever in PublicAPI.Shipped.txt — no binary-compat break.

Backward Compatibility & Rollback Safety ✅
MSBuild coerces @(ItemGroup) to a semicolon-separated string when assigning to a string parameter, and the split-by-; logic handles that path explicitly. Existing callers are unaffected.

Null Safety & Boundary Validation ✅
Split(['\\n', ';'], RemoveEmptyEntries) + Trim() handles \r\n Windows line endings. The extra IsNullOrEmpty guard after RemoveEmptyEntries is redundant but harmless.

Acceptance Test Coverage Design ⚠️ — Regressed
[TestCategory("Windows-Review")] is present again on RunDotnetTestWithCLIRunSettingsContainingBackslashes. The prior review (run 26647762464, commit 01c4982) confirmed this was resolved. The current HEAD (4bbce47) has re-introduced the gate, leaving the integration test with zero Unix coverage for a Unix-only bug. See inline comment.

Description Alignment ✅

PR description accurately reflects root cause, fix, and test strategy. No gaps.

---

🧠 Reviewed by Expert Code Reviewer 🧠

🧠 Reviewed by Expert Code Reviewer 🧠

🧠 Reviewed by Expert Code Reviewer 🧠