Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update sourcegen #3880

Draft
wants to merge 27 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
cc0d3be
Update source gen
nohwnd Sep 27, 2024
990d154
Get assembly
nohwnd Sep 30, 2024
0969391
Update source generation to latest
nohwnd Sep 30, 2024
45cb700
Merge branch 'main' into update-sourcegen
nohwnd Sep 30, 2024
acec139
down to just trim and aot warnings from libs
nohwnd Oct 1, 2024
b21bf50
revert proj
nohwnd Oct 1, 2024
2b6ee6c
fixes
nohwnd Oct 1, 2024
8294a2d
fixes
nohwnd Oct 1, 2024
b546d0c
Merge branch 'main' into update-sourcegen
Evangelink Oct 31, 2024
0f8c9cb
Merge branch 'main' into update-sourcegen
Evangelink Nov 1, 2024
c4a2bb8
Extend test with different init/cleanup attributes
Nov 5, 2024
2520e34
Fix cleanup methods in test
Nov 5, 2024
0b651fa
Revert VersionPrefix change and update source gen
Youssef1313 Nov 6, 2024
91414af
Merge branch 'main' into update-sourcegen
Youssef1313 Nov 6, 2024
e89c7d3
Update Version.Details.xml as well
Youssef1313 Nov 6, 2024
b0ead66
Revert launchSettings.json change
Youssef1313 Nov 6, 2024
e298e0a
Add more tests for native AOT
Youssef1313 Nov 6, 2024
def4e8b
Refactor tests, add more tests
Youssef1313 Nov 6, 2024
ad94d29
Fix typo
Youssef1313 Nov 6, 2024
88ff2bc
Address analyzer warning
Youssef1313 Nov 6, 2024
9477851
More tests
Youssef1313 Nov 6, 2024
873bd97
Formatting error
Youssef1313 Nov 8, 2024
5e41dba
Move to UnconditionalSuppressMessage attribute
Youssef1313 Nov 8, 2024
430bb08
Suppress
Youssef1313 Nov 12, 2024
31001ac
Avoid using #pragma warning disable ILxxxx
Youssef1313 Nov 19, 2024
ce4198b
Merge branch 'main' into update-sourcegen
Youssef1313 Nov 21, 2024
04f6470
Adjust
Youssef1313 Nov 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions eng/Version.Details.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@
<Uri>https://github.com/microsoft/testanywhere</Uri>
<Sha>d8fa8c790db1eb9f5eee1d4d8be1c8c21c3c3ef9</Sha>
</Dependency>
<Dependency Name="MSTest.Engine" Version="1.0.0-alpha.24473.2">
<Dependency Name="MSTest.SourceGeneration" Version="1.0.0-alpha.24555.1">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's sync versions:

Suggested change
<Dependency Name="MSTest.SourceGeneration" Version="1.0.0-alpha.24555.1">
<Dependency Name="MSTest.SourceGeneration" Version="1.0.0-alpha.24554.7">

<Uri>https://github.com/microsoft/testanywhere</Uri>
<Sha>aa2fcc8616d988b234bc1d218465b20c56d0b82f</Sha>
<Sha>3c99c34d0f99de8dacfe909073722bd8ba45f2d8</Sha>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<Sha>3c99c34d0f99de8dacfe909073722bd8ba45f2d8</Sha>
<Sha>4d80102c4530845265843db1454a4abda4128ba2</Sha>

</Dependency>
</ToolsetDependencies>
</Dependencies>
4 changes: 2 additions & 2 deletions eng/Versions.props
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@
<!-- MSTest version -->
<VersionPrefix>3.7.0</VersionPrefix>
Youssef1313 marked this conversation as resolved.
Show resolved Hide resolved
<!-- Testing Platform version -->
<TestingPlatformVersionPrefix>1.5.0</TestingPlatformVersionPrefix>
<TestingPlatformVersionPrefix>1.6.0</TestingPlatformVersionPrefix>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs to be reset

Suggested change
<TestingPlatformVersionPrefix>1.6.0</TestingPlatformVersionPrefix>
<TestingPlatformVersionPrefix>1.5.0</TestingPlatformVersionPrefix>

<PreReleaseVersionLabel>preview</PreReleaseVersionLabel>
</PropertyGroup>
<PropertyGroup Label="MSTest prod dependencies - darc updated">
<MicrosoftDotNetBuildTasksTemplatingPackageVersion>10.0.0-beta.24570.1</MicrosoftDotNetBuildTasksTemplatingPackageVersion>
<MicrosoftTestingExtensionsCodeCoverageVersion>17.13.0-preview.24562.1</MicrosoftTestingExtensionsCodeCoverageVersion>
<!-- comment to facilitate merge conflicts -->
<MicrosoftTestingInternalFrameworkVersion>1.5.0-preview.24569.1</MicrosoftTestingInternalFrameworkVersion>
<MSTestEngineVersion>1.0.0-alpha.24473.2</MSTestEngineVersion>
<MSTestSourceGenerationVersion>1.0.0-alpha.24555.1</MSTestSourceGenerationVersion>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Syncing versions

Suggested change
<MSTestSourceGenerationVersion>1.0.0-alpha.24555.1</MSTestSourceGenerationVersion>
<MSTestSourceGenerationVersion>1.0.0-alpha.24554.7</MSTestSourceGenerationVersion>

</PropertyGroup>
</Project>
2 changes: 0 additions & 2 deletions samples/Playground/DebuggerUtility.cs
Original file line number Diff line number Diff line change
Expand Up @@ -121,9 +121,7 @@ private static bool AttachVs(Process vs, int pid, bool enableLog = false)
IEnumMoniker? enumMoniker = null;
try
{
#pragma warning disable IL2050
int r = CreateBindCtx(0, out bindCtx);
#pragma warning restore IL2050
Marshal.ThrowExceptionForHR(r);
if (bindCtx == null)
{
Expand Down
2 changes: 1 addition & 1 deletion samples/Playground/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ public static async Task<int> Main(string[] args)
if (Environment.GetEnvironmentVariable("TESTSERVERMODE") != "1")
{
// To attach to the children
Microsoft.Testing.TestInfrastructure.DebuggerUtility.AttachCurrentProcessToParentVSProcess();
// Microsoft.Testing.TestInfrastructure.DebuggerUtility.AttachCurrentProcessToParentVSProcess();
ITestApplicationBuilder testApplicationBuilder = await TestApplication.CreateBuilderAsync(args);

// Test MSTest
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@
using Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution;
using Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Helpers;
using Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.ObjectModel;
#if NET8_0_OR_GREATER
using Microsoft.VisualStudio.TestPlatform.MSTestAdapter;
#endif
using Microsoft.VisualStudio.TestPlatform.ObjectModel;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Microsoft.VisualStudio.TestTools.UnitTesting.Internal;
Expand Down Expand Up @@ -305,8 +308,13 @@ private static bool DynamicDataAttached(UnitTestElement test, Lazy<TestMethodInf
TryProcessITestDataSourceTests(test, testMethodInfo.Value, tests);
}

[UnconditionalSuppressMessage("Aot", "IL3000:DoNotUseLocation", Justification = "Fixture tests are not supported in NativeAOT mode.")]

private static void AddFixtureTests(TestMethodInfo testMethodInfo, List<UnitTestElement> tests, HashSet<string> fixtureTests)
{
#if NET8_0_OR_GREATER
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be NETCOREAPP or NET7_0_OR_GREATER if we want to link it specifically to native aot.

ApplicationStateGuard.Ensure(System.Runtime.CompilerServices.RuntimeFeature.IsDynamicCodeSupported, "Fixture tests are not supported in NativeAOT mode.");
#endif
string assemblyName = testMethodInfo.Parent.Parent.Assembly.GetName().Name!;
string assemblyLocation = testMethodInfo.Parent.Parent.Assembly.Location;
string className = testMethodInfo.Parent.ClassType.Name;
Expand Down
2 changes: 2 additions & 0 deletions src/Adapter/MSTest.TestAdapter/Execution/TypeCache.cs
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,7 @@ private static bool TryGetUnescapedManagedTypeName(TestMethod testMethod, [NotNu
/// <param name="classType"> The class Type. </param>
/// <param name="testMethod"> The test Method. </param>
/// <returns> The <see cref="TestClassInfo"/>. </returns>
[UnconditionalSuppressMessage("Aot", "IL2070:DoNotUseGetConstructor", Justification = "We access all the types we need in metadata, so this is preserved and works.")]
private TestClassInfo CreateClassInfo(Type classType, TestMethod testMethod)
{
IEnumerable<ConstructorInfo> constructors = PlatformServiceProvider.Instance.ReflectionOperations.GetDeclaredConstructors(classType);
Expand Down Expand Up @@ -832,6 +833,7 @@ private MethodInfo GetMethodInfoForTestMethod(TestMethod testMethod, TestClassIn
return testMethodInfo;
}

[UnconditionalSuppressMessage("Aot", "IL2026:DoNotUseGetConstructor", Justification = "We access all the types we need in metadata, so this is preserved and works.")]
private static MethodInfo? GetMethodInfoUsingManagedNameHelper(TestMethod testMethod, TestClassInfo testClassInfo, bool discoverInternals)
{
MethodBase? methodBase = null;
Expand Down
62 changes: 34 additions & 28 deletions src/Adapter/MSTest.TestAdapter/Helpers/DataSerializationHelper.cs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using System.Collections.Concurrent;
using System.Diagnostics.CodeAnalysis;
using System.Runtime.Serialization.Json;
using System.Text;

Expand All @@ -23,6 +24,14 @@ internal static class DataSerializationHelper
/// </summary>
/// <param name="data">Data array to serialize.</param>
/// <returns>Serialized array.</returns>
[UnconditionalSuppressMessage(
"Trimming",
"IL2026:Members annotated with 'RequiresUnreferencedCodeAttribute' require dynamic access otherwise can break functionality when trimming application code",
Justification = "This should be safe as long as our generator mentions getting fields / properties of the target type. https://github.com/dotnet/runtime/issues/71350#issuecomment-1168140551. Not the best solution, maybe we can replace this with System.Text.Json, but the we need one generator calling the other.")]
[UnconditionalSuppressMessage(
"AOT",
"IL3050:Calling members annotated with 'RequiresDynamicCodeAttribute' may break functionality when AOT compiling.",
Justification = "This should be safe as long as our generator mentions getting fields / properties of the target type. https://github.com/dotnet/runtime/issues/71350#issuecomment-1168140551. Not the best solution, maybe we can replace this with System.Text.Json, but the we need one generator calling the other.")]
public static string?[]? Serialize(object?[]? data)
{
if (data == null)
Expand Down Expand Up @@ -51,14 +60,7 @@ internal static class DataSerializationHelper
DataContractJsonSerializer serializer = GetSerializer(type);

using var memoryStream = new MemoryStream();
// This should be safe as long as our generator mentions
// getting fields / properties of the target type. https://github.com/dotnet/runtime/issues/71350#issuecomment-1168140551
// Not the best solution, maybe we can replace this with System.Text.Json, but the we need one generator calling the other.
#pragma warning disable IL3050 // IL3050: Avoid calling members annotated with 'RequiresDynamicCodeAttribute' when publishing as Native AOT
#pragma warning disable IL2026 // IL2026: Members attributed with RequiresUnreferencedCode may break when trimming
serializer.WriteObject(memoryStream, data[i]);
#pragma warning restore IL3050 // IL3050: Avoid calling members annotated with 'RequiresDynamicCodeAttribute' when publishing as Native AOT
#pragma warning restore IL2026 // IL2026: Members attributed with RequiresUnreferencedCode may break when trimming
byte[] serializerData = memoryStream.ToArray();

serializedData[dataIndex] = Encoding.UTF8.GetString(serializerData, 0, serializerData.Length);
Expand All @@ -72,6 +74,14 @@ internal static class DataSerializationHelper
/// </summary>
/// <param name="serializedData">Serialized data array to deserialize.</param>
/// <returns>Deserialized array.</returns>
[UnconditionalSuppressMessage(
"Trimming",
"IL2026:Members annotated with 'RequiresUnreferencedCodeAttribute' require dynamic access otherwise can break functionality when trimming application code",
Justification = "This should be safe as long as our generator mentions getting fields / properties of the target type. https://github.com/dotnet/runtime/issues/71350#issuecomment-1168140551. Not the best solution, maybe we can replace this with System.Text.Json, but the we need one generator calling the other.")]
[UnconditionalSuppressMessage(
"AOT",
"IL3050:Calling members annotated with 'RequiresDynamicCodeAttribute' may break functionality when AOT compiling.",
Justification = "This should be safe as long as our generator mentions getting fields / properties of the target type. https://github.com/dotnet/runtime/issues/71350#issuecomment-1168140551. Not the best solution, maybe we can replace this with System.Text.Json, but the we need one generator calling the other.")]
public static object?[]? Deserialize(string?[]? serializedData)
{
if (serializedData == null || serializedData.Length % 2 != 0)
Expand All @@ -98,40 +108,36 @@ internal static class DataSerializationHelper

byte[] serializedDataBytes = Encoding.UTF8.GetBytes(serializedValue);
using var memoryStream = new MemoryStream(serializedDataBytes);
// This should be safe as long as our generator mentions
// getting fields / properties of the target type. https://github.com/dotnet/runtime/issues/71350#issuecomment-1168140551
// Not the best solution, maybe we can replace this with System.Text.Json, but the we need one generator calling the other.
#pragma warning disable IL3050 // IL3050: Avoid calling members annotated with 'RequiresDynamicCodeAttribute' when publishing as Native AOT
#pragma warning disable IL2026 // IL2026: Members attributed with RequiresUnreferencedCode may break when trimming
data[i] = serializer.ReadObject(memoryStream);
#pragma warning restore IL3050 // IL3050: Avoid calling members annotated with 'RequiresDynamicCodeAttribute' when publishing as Native AOT
#pragma warning restore IL2026 // IL2026: Members attributed with RequiresUnreferencedCode may break when trimming
}

return data;
}

[UnconditionalSuppressMessage(
"Trimming",
"IL2026:Members annotated with 'RequiresUnreferencedCodeAttribute' require dynamic access otherwise can break functionality when trimming application code",
Justification = "This should be safe as long as our generator mentions getting fields / properties of the target type. https://github.com/dotnet/runtime/issues/71350#issuecomment-1168140551. Not the best solution, maybe we can replace this with System.Text.Json, but the we need one generator calling the other.")]
[UnconditionalSuppressMessage(
"AOT",
"IL3050:Calling members annotated with 'RequiresDynamicCodeAttribute' may break functionality when AOT compiling.",
Justification = "This should be safe as long as our generator mentions getting fields / properties of the target type. https://github.com/dotnet/runtime/issues/71350#issuecomment-1168140551. Not the best solution, maybe we can replace this with System.Text.Json, but the we need one generator calling the other.")]

private static DataContractJsonSerializer GetSerializer(string assemblyQualifiedName)
=> SerializerCache.GetOrAdd(
assemblyQualifiedName,
// This should be safe as long as our generator mentions
// getting fields / properties of the target type. https://github.com/dotnet/runtime/issues/71350#issuecomment-1168140551
// Not the best solution, maybe we can replace this with System.Text.Json, but the we need one generator calling the other.
#pragma warning disable IL3050 // IL3050: Avoid calling members annotated with 'RequiresDynamicCodeAttribute' when publishing as Native AOT
#pragma warning disable IL2026 // IL2026: Members attributed with RequiresUnreferencedCode may break when trimming
_ => new DataContractJsonSerializer(PlatformServiceProvider.Instance.ReflectionOperations.GetType(assemblyQualifiedName) ?? typeof(object), SerializerSettings));
#pragma warning restore IL3050 // IL3050: Avoid calling members annotated with 'RequiresDynamicCodeAttribute' when publishing as Native AOT
#pragma warning restore IL2026 // IL2026: Members attributed with RequiresUnreferencedCode may break when trimming

[UnconditionalSuppressMessage(
"Trimming",
"IL2026:Members annotated with 'RequiresUnreferencedCodeAttribute' require dynamic access otherwise can break functionality when trimming application code",
Justification = "This should be safe as long as our generator mentions getting fields / properties of the target type. https://github.com/dotnet/runtime/issues/71350#issuecomment-1168140551. Not the best solution, maybe we can replace this with System.Text.Json, but the we need one generator calling the other.")]
[UnconditionalSuppressMessage(
"AOT",
"IL3050:Calling members annotated with 'RequiresDynamicCodeAttribute' may break functionality when AOT compiling.",
Justification = "This should be safe as long as our generator mentions getting fields / properties of the target type. https://github.com/dotnet/runtime/issues/71350#issuecomment-1168140551. Not the best solution, maybe we can replace this with System.Text.Json, but the we need one generator calling the other.")]
private static DataContractJsonSerializer GetSerializer(Type type)
=> SerializerCache.GetOrAdd(
type.AssemblyQualifiedName!,
// This should be safe as long as our generator mentions
// getting fields / properties of the target type. https://github.com/dotnet/runtime/issues/71350#issuecomment-1168140551
// Not the best solution, maybe we can replace this with System.Text.Json, but the we need one generator calling the other.
#pragma warning disable IL3050 // IL3050: Avoid calling members annotated with 'RequiresDynamicCodeAttribute' when publishing as Native AOT
#pragma warning disable IL2026 // IL2026: Members attributed with RequiresUnreferencedCode may break when trimming
_ => new DataContractJsonSerializer(type, SerializerSettings));
#pragma warning restore IL3050 // IL3050: Avoid calling members annotated with 'RequiresDynamicCodeAttribute' when publishing as Native AOT
#pragma warning restore IL2026 // IL2026: Members attributed with RequiresUnreferencedCode may break when trimming
}
2 changes: 1 addition & 1 deletion src/Adapter/MSTest.TestAdapter/MSTest.TestAdapter.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

<PropertyGroup>
<!-- Enable AOT analyzer warnings to make sure we don't call APIs that would fail when we use source generator mode together with NativeAOT. -->
<IsAotCompatible Condition="$([MSBuild]::IsTargetFrameworkCompatible('$(TargetFramework)', 'net7.0'))">$(EnableIotAnalyzers)</IsAotCompatible>
<IsAotCompatible Condition="$([MSBuild]::IsTargetFrameworkCompatible('$(TargetFramework)', 'net7.0'))">true</IsAotCompatible>
</PropertyGroup>

<PropertyGroup>
Expand Down
2 changes: 2 additions & 0 deletions src/Adapter/MSTest.TestAdapter/TestMethodFilter.cs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using System.Diagnostics.CodeAnalysis;
using System.Reflection;

using Microsoft.VisualStudio.TestPlatform.ObjectModel;
Expand Down Expand Up @@ -118,6 +119,7 @@ internal TestProperty PropertyProvider(string propertyName)
/// <param name="context">Discovery context.</param>
/// <param name="logger">The logger to log exception messages too.</param>
/// <returns>Filter expression.</returns>
[UnconditionalSuppressMessage("Aot", "IL2072:DoNotUseDynamicMembers", Justification = "Tested it, it works.")]
private ITestCaseFilterExpression? GetTestCaseFilterFromDiscoveryContext(IDiscoveryContext context, IMessageLogger logger)
{
try
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,17 @@ namespace Microsoft.VisualStudio.TestTools.UnitTesting;
public static class TestingPlatformBuilderHook
{
#pragma warning disable IDE0060 // Remove unused parameter
public static void AddExtensions(ITestApplicationBuilder testApplicationBuilder, string[] arguments) => testApplicationBuilder.AddMSTest(() => [Assembly.GetEntryAssembly()!]);
public static void AddExtensions(ITestApplicationBuilder testApplicationBuilder, string[] arguments)
{
#if NET8_0_OR_GREATER
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be NETCOREAPP or NET7_0_OR_GREATER if we want to link it specifically to native aot.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC it did not build with just netcoreapp, and net7 is out of life and rest of the platform used net8 so I used net8

if (!System.Runtime.CompilerServices.RuntimeFeature.IsDynamicCodeSupported)
{
// We don't have a reliable way to get reference to the entry dll when compiled as NativeAOT. So instead we do the same registration
// in source generator.
return;
}
#endif
testApplicationBuilder.AddMSTest(() => [Assembly.GetEntryAssembly()!]);
}
}
#endif
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

#if NETFRAMEWORK || NET
using System.Diagnostics;
using System.Diagnostics.CodeAnalysis;
using System.Reflection;
#if NETFRAMEWORK
using System.Runtime.InteropServices.WindowsRuntime;
Expand Down Expand Up @@ -291,16 +292,14 @@ private static
#endif
bool DoesFileExist(string filePath) => File.Exists(filePath);

[UnconditionalSuppressMessage("Aot", "IL2026:DoNotUseGetDefinedTypes", Justification = "The whole class is not used in source generator mode.")]

#if NETFRAMEWORK
protected virtual
#else
private static
#endif

// This whole class is not used in source generator mode.
#pragma warning disable IL2026 // Members attributed with RequiresUnreferencedCode may break when trimming
Assembly LoadAssemblyFrom(string path) => Assembly.LoadFrom(path);
#pragma warning restore IL2026 // Members attributed with RequiresUnreferencedCode may break when trimming

#if NETFRAMEWORK
protected virtual Assembly ReflectionOnlyLoadAssemblyFrom(string path) => Assembly.ReflectionOnlyLoadFrom(path);
Expand Down
Loading
Loading