Skip to content

Migrate to xunit.v3 #10890

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

Migrate to xunit.v3 #10890

wants to merge 8 commits into from

Conversation

Youssef1313
Copy link
Member

@Youssef1313 Youssef1313 commented May 28, 2025 •
edited
Loading

TODO:

  • Add dotnet.config

@dotnet-policy-service dotnet-policy-service bot added PR metadata: Label to tag PRs, to facilitate with triage draft labels May 28, 2025
@Youssef1313
Copy link
Member Author

CI fails due to missing package in our feed. Started a mirror here

@Youssef1313
Copy link
Member Author

/azp run

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@Youssef1313
Copy link
Member Author

Wpf seems to depend on https://github.com/dotnet/winforms/blob/main/src/Common/tests/TestUtilities/System.Private.Windows.Core.TestUtilities.csproj. So probably worth moving winforms first

@Youssef1313
Copy link
Member Author

winforms moved now with dotnet/winforms#13540

@Youssef1313 Youssef1313 force-pushed the xunitv3 branch 2 times, most recently from 8a4ded7 to 1a898fc Compare June 16, 2025 13:22
@Youssef1313
Copy link
Member Author 

=== COMMAND LINE === 
"D:\a\_work\1\s\artifacts\bin\System.Xaml.Tests\x64\Debug\net10.0\System.Xaml.Tests.exe"    --hangdump --hangdump-timeout 10min --crashdump --results-directory "D:\a\_work\1\s\artifacts\TestResults\Debug" --report-xunit --report-xunit-filename "System.Xaml.Tests_net10.0_x64.xml" --report-xunit-html --report-xunit-html-filename "System.Xaml.Tests_net10.0_x64.html" --report-trx --report-trx-filename "System.Xaml.Tests_net10.0_x64.trx" --auto-reporters off 
[createdump] Writing full dump for process 3616 to file D:\a\_work\1\s\artifacts\TestResults\Debug\System.Xaml.Tests_3616_crash.dmp
[createdump] Dump successfully written in 2905ms
[+0/x0/?0] System.Xaml.Tests.dll (net10.0|x64)(3s)

Unhandled exception. System.ArgumentNullException: Value cannot be null. (Parameter 'typeName')
   at System.ArgumentNullException.Throw(String paramName)
   at System.ArgumentNullException.ThrowIfNull(Object argument, String paramName)
   at System.ComponentModel.TypeConverterAttribute..ctor(String typeName)
   at System.Reflection.CustomAttribute.CreateCustomAttributeInstance(RuntimeModule module, RuntimeType type, IRuntimeMethodInfo ctor, IntPtr& blob, IntPtr blobEnd, Int32& namedArgs)
   at System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder`1& attributes, RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1 derivedAttributes)
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeType type, RuntimeType caType, Boolean inherit)
   at System.Attribute.GetCustomAttributes(MemberInfo element, Boolean inherit)
   at Xunit.Sdk.ReflectionExtensions.GetMatchingCustomAttributes(Type type, Type attributeType) in /_/src/xunit.v3.common/Extensions/ReflectionExtensions.cs:line 234
   at Xunit.v3.TestCollectionFactoryBase.Get(Type testClass) in /_/src/xunit.v3.core/Framework/TestCollectionFactoryBase.cs:line 37
   at Xunit.v3.XunitTestFrameworkDiscoverer.CreateTestClass(Type class) in /_/src/xunit.v3.core/Framework/XunitTestFrameworkDiscoverer.cs:line 58
   at Xunit.v3.TestFrameworkDiscoverer`1.<>c__DisplayClass10_0.<<Find>b__0>d.MoveNext() in /_/src/xunit.v3.core/Framework/TestFrameworkDiscoverer.cs:line 76
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__124_1(Object state)
   at System.Threading.QueueUserWorkItemCallback.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback()
Test host process with PID '3616' crashed, a dump file was generated
Test application process didn't exit gracefully, exit code is '7'

cc @bradwilson

@bradwilson
Copy link

Hard to tell but this looks like a custom attribute might be throwing in a constructor somewhere...?

@bradwilson
Copy link

I'd be curious to know what happens if you try using 3.0.0-pre.25 because I just removed calls to GetCustomAttributes to help ignore attributes that are throwing in their constructors.

@Youssef1313
Copy link
Member Author

Youssef1313 commented Jun 18, 2025 •
edited
Loading

Updating helped. There are other failures that I'm investigating now.

@bradwilson I'm curious whether you're planning to ship xunit.v3 3.x stable going to be released some time soon?

@bradwilson
Copy link

Right now, our release plan is to allow third parties chances to find issues with breaking changes, we will give them a month to review the changes.

The most recent build shipped 3 days ago. Assuming there are no further builds that need to be shipped to account for new breaking changes (and thus reset the clock), then a final release should be in the middle of July.

@Youssef1313
Copy link
Member Author

@bradwilson Thanks for clarifying!

@Youssef1313
Copy link
Member Author

Youssef1313 commented Jun 18, 2025 •
edited
Loading

Note to self: This is currently blocked as current version of xunit.stafact isn't compatible with xunit.v3 3.x.

@Youssef1313 Youssef1313 mentioned this pull request Jun 19, 2025
Closed
@Youssef1313
Copy link
Member Author

Now only needs fixing coverage.

@Youssef1313 Youssef1313 marked this pull request as ready for review July 26, 2025 09:19
@Copilot Copilot AI review requested due to automatic review settings July 26, 2025 09:19
@Youssef1313 Youssef1313 requested a review from a team as a code owner July 26, 2025 09:19
@Youssef1313
Copy link
Member Author

@dipeshmsft Can I have a review here please? Thanks!

Copilot
Copilot AI reviewed Jul 26, 2025
View reviewed changes
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR migrates the WPF project from xUnit v2 to xUnit v3, updating the test framework across multiple test projects and build configuration files.

  • Updates all test project files to use xUnit v3 and configure them as executable projects
  • Replaces xUnit v2 package references with xUnit v3 equivalents and updates version numbers
  • Migrates test runner configuration from VSTest to Microsoft Testing Platform with new crash/hang dump extensions

Reviewed Changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
WindowsBase.Tests.csproj Adds OutputType property for executable test project
SplashScreenTests.cs Skips failing test and updates file encoding
ComponentDispatcherTests.cs Skips failing test
System.Xaml.Tests.csproj Adds OutputType property for executable test project
System.Printing.Tests.csproj Adds OutputType property for executable test project
PresentationFramework.Tests.csproj Adds OutputType property for executable test project
PresentationFramework.Fluent.Tests.csproj Adds OutputType property and test runner workaround
PresentationCore.Tests.csproj Adds OutputType property for executable test project
Directory.Build.props Removes coverlet collector package reference
MultiTargeting.props Replaces xUnit v2 packages with xUnit v3 package
Versions.props Updates package versions for xUnit v3 migration
Testing.targets Migrates from VSTest to Microsoft Testing Platform configuration
CodeCoverage.runsettings Removes VSTest-specific configuration file
CodeCoverage.proj Updates coverage report file patterns
CodeCoverage.config Adds new code coverage configuration for Microsoft Testing Platform
dotnet.config Adds configuration to specify Microsoft Testing Platform as test runner
Directory.Build.props Sets XUnitV3 as the test runner name
Comments suppressed due to low confidence (2)

eng/Versions.props:89

  • The XUnitStaFactPackageVersion 3.0.13 may not be a valid version. The package xunit.stafact typically follows semantic versioning, and version 3.0.13 appears to be significantly higher than expected release versions. Please verify this version exists in the package repository.
    <XUnitStaFactPackageVersion>3.0.13</XUnitStaFactPackageVersion>

eng/Versions.props:98

  • The VerifyXunitV3Version 30.1.0 appears to be an unusually high version number. Verify.Xunit packages typically use lower version numbers. Please confirm this version exists in the package repository.
    <VerifyXunitV3Version>30.1.0</VerifyXunitV3Version>

@Youssef1313 Youssef1313 removed the draft label Jul 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

Copilot code review Copilot Copilot left review comments

@dipeshmsft dipeshmsft Awaiting requested review from dipeshmsft

@singhashish-wpf singhashish-wpf Awaiting requested review from singhashish-wpf

@Kuldeep-MS Kuldeep-MS Awaiting requested review from Kuldeep-MS

At least 1 approving review is required to merge this pull request.

Assignees

@Youssef1313 Youssef1313

Labels
PR metadata: Label to tag PRs, to facilitate with triage
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Youssef1313 @bradwilson