Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RunTests target hides exception thrown during NUnit fixture setup #14812

Open
sbomer opened this issue Jun 5, 2024 · 0 comments
Open

RunTests target hides exception thrown during NUnit fixture setup #14812

sbomer opened this issue Jun 5, 2024 · 0 comments

Comments

@sbomer
Copy link
Member

sbomer commented Jun 5, 2024

I hit a scenario where running tests via arcade hides some important messages indicating that test setup failed and some tests were skipped. Running tests normally using the dotnet sdk surfaces this information as expected. Repro steps:

  • git clone https://github.com/dotnet/cecil
  • cd cecil
  • git checkout ff87dfd51654949f6379059c258b7f8c13e10d29
  • ./build.sh -test (succeeds without indicating there were any issues)
    Downloading 'https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh'
      Attempting to install 'sdk v9.0.100-preview.4.24267.66' from public_location.
    dotnet-install: Attempting to download using primary link https://dotnetcli.azureedge.net/dotnet/Sdk/9.0.100-preview.4.24267.66/dotnet-sdk-9.0.100-preview.4.24267.66-linux-x64.tar.gz
    dotnet-install: Remote file https://dotnetcli.azureedge.net/dotnet/Sdk/9.0.100-preview.4.24267.66/dotnet-sdk-9.0.100-preview.4.24267.66-linux-x64.tar.gz size is 240934655 bytes.
    dotnet-install: Extracting archive from https://dotnetcli.azureedge.net/dotnet/Sdk/9.0.100-preview.4.24267.66/dotnet-sdk-9.0.100-preview.4.24267.66-linux-x64.tar.gz
    dotnet-install: Downloaded file size is 240934655 bytes.
    dotnet-install: The remote and local file sizes are equal.
    dotnet-install: Installed version is 9.0.100-preview.4.24267.66
    dotnet-install: Adding to current process PATH: `/home/svbomer/tmp/cecil/.dotnet`. Note: This change will be visible only when sourcing script.
    dotnet-install: Note that the script does not resolve dependencies during installation.
    dotnet-install: To check the list of dependencies, go to https://learn.microsoft.com/dotnet/core/install, select your operating system and check the "Dependencies" section.
    dotnet-install: Installation finished successfully.
    
    Build succeeded in 1.5s
    Restore complete (1.1s)
      Mono.Cecil succeeded (2.9s) → artifacts/bin/Mono.Cecil/Debug/netstandard2.0/Mono.Cecil.dll
      Mono.Cecil.Rocks succeeded (0.8s) → artifacts/bin/Mono.Cecil.Rocks/Debug/netstandard2.0/Mono.Cecil.Rocks.dll
      Mono.Cecil.Mdb succeeded (0.8s) → artifacts/bin/Mono.Cecil.Mdb/Debug/netstandard2.0/Mono.Cecil.Mdb.dll
      Mono.Cecil.Pdb succeeded (1.0s) → artifacts/bin/Mono.Cecil.Pdb/Debug/netstandard2.0/Mono.Cecil.Pdb.dll
      Mono.Cecil.Tests succeeded (2.4s) → artifacts/bin/Mono.Cecil.Tests/Debug/net9.0/Mono.Cecil.Tests.dll
      Mono.Cecil.Mdb.Tests succeeded (0.6s) → artifacts/bin/Mono.Cecil.Mdb.Tests/Debug/net9.0/Mono.Cecil.Mdb.Tests.dll
      Mono.Cecil.Pdb.Tests succeeded (0.7s) → artifacts/bin/Mono.Cecil.Pdb.Tests/Debug/net9.0/Mono.Cecil.Pdb.Tests.dll
      Mono.Cecil.Rocks.Tests succeeded (0.8s) → artifacts/bin/Mono.Cecil.Rocks.Tests/Debug/net9.0/Mono.Cecil.Rocks.Tests.dll
    
    Build succeeded in 34.5
    
  • .dotnet/dotnet test Mono.Cecil.sln (logs a warning indicating some tests didn't run because setup failed):
    Restore complete (1.3s)
      Mono.Cecil succeeded (1.0s) → artifacts/bin/Mono.Cecil/Debug/netstandard2.0/Mono.Cecil.dll
      Mono.Cecil.Mdb succeeded (0.2s) → artifacts/bin/Mono.Cecil.Mdb/Debug/netstandard2.0/Mono.Cecil.Mdb.dll
      Mono.Cecil.Rocks succeeded (0.2s) → artifacts/bin/Mono.Cecil.Rocks/Debug/netstandard2.0/Mono.Cecil.Rocks.dll
      Mono.Cecil.Pdb succeeded (0.2s) → artifacts/bin/Mono.Cecil.Pdb/Debug/netstandard2.0/Mono.Cecil.Pdb.dll
      Mono.Cecil.Tests succeeded (0.7s) → artifacts/bin/Mono.Cecil.Tests/Debug/net9.0/Mono.Cecil.Tests.dll
      Mono.Cecil.Tests test succeeded with 2 warning(s) (23.9s)
        /home/svbomer/tmp/cecil/.dotnet/sdk/9.0.100-preview.4.24267.66/Microsoft.TestPlatform.targets(46,5): warning : SetUp failed for test fixture Mono.Cecil.Tests.ILProcessorTests
        /home/svbomer/tmp/cecil/.dotnet/sdk/9.0.100-preview.4.24267.66/Microsoft.TestPlatform.targets(46,5): warning : 
          An exception was thrown while loading the test.
          System.ArgumentException: Unknown framework version 9.5 (Parameter 'version')
             at NUnit.Framework.Internal.RuntimeFramework.ThrowInvalidFrameworkVersion(Version version)
             at NUnit.Framework.Internal.RuntimeFramework.InitFromFrameworkVersion(Version version)
             at NUnit.Framework.Internal.RuntimeFramework..ctor(RuntimeType runtime, Version version)
             at NUnit.Framework.Internal.RuntimeFramework.<>c.<.cctor>b__33_0()
             at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
             at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
             at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
             at System.Lazy`1.CreateValue()
             at NUnit.Framework.Internal.RuntimeFramework.get_CurrentFramework()
             at NUnit.Framework.Internal.PlatformHelper..ctor()
             at NUnit.Framework.TestCaseAttribute.BuildFrom(IMethodInfo method, Test suite)+MoveNext()
             at NUnit.Framework.Internal.Builders.DefaultTestCaseBuilder.BuildFrom(IMethodInfo method, Test parentSuite)
             at NUnit.Framework.Internal.Builders.NUnitTestFixtureBuilder.BuildTestCase(IMethodInfo method, TestSuite suite)
             at NUnit.Framework.Internal.Builders.NUnitTestFixtureBuilder.AddTestCasesToFixture(TestFixture fixture, IPreFilter filter)
             at NUnit.Framework.Internal.Builders.NUnitTestFixtureBuilder.BuildFrom(ITypeInfo typeInfo, IPreFilter filter, ITestFixtureData testFixtureData)
             at NUnit.Framework.TestFixtureAttribute.BuildFrom(ITypeInfo typeInfo, IPreFilter filter)+MoveNext()
             at NUnit.Framework.Internal.Builders.DefaultSuiteBuilder.BuildFrom(ITypeInfo typeInfo, IPreFilter filter)
      Mono.Cecil.Pdb.Tests succeeded (0.2s) → artifacts/bin/Mono.Cecil.Pdb.Tests/Debug/net9.0/Mono.Cecil.Pdb.Tests.dll
      Mono.Cecil.Rocks.Tests succeeded (0.3s) → artifacts/bin/Mono.Cecil.Rocks.Tests/Debug/net9.0/Mono.Cecil.Rocks.Tests.dll
      Mono.Cecil.Mdb.Tests succeeded (0.3s) → artifacts/bin/Mono.Cecil.Mdb.Tests/Debug/net9.0/Mono.Cecil.Mdb.Tests.dll
      Mono.Cecil.Pdb.Tests test succeeded (1.3s)
      Mono.Cecil.Rocks.Tests test succeeded with 2 warning(s) (1.7s)
        /home/svbomer/tmp/cecil/.dotnet/sdk/9.0.100-preview.4.24267.66/Microsoft.TestPlatform.targets(46,5): warning : SetUp failed for test fixture Mono.Cecil.Tests.DocCommentIdTests
        /home/svbomer/tmp/cecil/.dotnet/sdk/9.0.100-preview.4.24267.66/Microsoft.TestPlatform.targets(46,5): warning : 
          An exception was thrown while loading the test.
          System.ArgumentException: Unknown framework version 9.5 (Parameter 'version')
             at NUnit.Framework.Internal.RuntimeFramework.ThrowInvalidFrameworkVersion(Version version)
             at NUnit.Framework.Internal.RuntimeFramework.InitFromFrameworkVersion(Version version)
             at NUnit.Framework.Internal.RuntimeFramework..ctor(RuntimeType runtime, Version version)
             at NUnit.Framework.Internal.RuntimeFramework.<>c.<.cctor>b__33_0()
             at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
             at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
             at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
             at System.Lazy`1.CreateValue()
             at NUnit.Framework.Internal.RuntimeFramework.get_CurrentFramework()
             at NUnit.Framework.Internal.PlatformHelper..ctor()
             at NUnit.Framework.TestCaseAttribute.BuildFrom(IMethodInfo method, Test suite)+MoveNext()
             at NUnit.Framework.Internal.Builders.DefaultTestCaseBuilder.BuildFrom(IMethodInfo method, Test parentSuite)
             at NUnit.Framework.Internal.Builders.NUnitTestFixtureBuilder.BuildTestCase(IMethodInfo method, TestSuite suite)
             at NUnit.Framework.Internal.Builders.NUnitTestFixtureBuilder.AddTestCasesToFixture(TestFixture fixture, IPreFilter filter)
             at NUnit.Framework.Internal.Builders.NUnitTestFixtureBuilder.BuildFrom(ITypeInfo typeInfo, IPreFilter filter, ITestFixtureData testFixtureData)
             at NUnit.Framework.TestFixtureAttribute.BuildFrom(ITypeInfo typeInfo, IPreFilter filter)+MoveNext()
             at NUnit.Framework.Internal.Builders.DefaultSuiteBuilder.BuildFrom(ITypeInfo typeInfo, IPreFilter filter)
      Mono.Cecil.Mdb.Tests test succeeded (5.2s)
    
    Test summary: total: 342, failed: 0, succeeded: 335, skipped: 7, duration: 29.4s
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants