Orleans 7.0 Serializer not working with POCOs originating from Nuget package

[Professional-amateur-dev]

I have a Orleans 7.0 project with and I'm having an issue with the new default serializer.

The old serializer in Orleans 3.6.5 worked fine and it was serializing all the models and Dtos in the several internal Nuget packages.

Now in Orleans 7.0 the project does not recognize the models in the Nuget packages and I get this error:

Orleans.Serialization.CodecNotFoundException: Could not find a copier for type InternalProject.Model.
   at Orleans.Serialization.Serializers.CodecProvider.ThrowCopierNotFound(Type type) in /_/src/Orleans.Serialization/Serializers/CodecProvider.cs:line 666
   at Orleans.Serialization.Serializers.CodecProvider.GetDeepCopier[T]() in /_/src/Orleans.Serialization/Serializers/CodecProvider.cs:line 300
   at Orleans.Serialization.ServiceCollectionExtensions.CopierHolder`1.get_Value() in /_/src/Orleans.Serialization/Hosting/ServiceCollectionExtensions.cs:line 203
   at Orleans.Serialization.GeneratedCodeHelpers.OrleansGeneratedCodeHelper.GetService[TService](Object caller, ICodecProvider codecProvider) in /_/src/Orleans.Serialization/GeneratedCodeHelpers/OrleansGeneratedCodeHelper.cs:lin
e 75
   at OrleansCodeGen.InternalProject.Model.Copier_SpecificModel..ctor(ICodecProvider codecProvider) in C:\Projects\InternalProject.Model\Orleans.CodeGenerator\Orleans.CodeGenerator.Orle
ansSerializationSourceGenerator\InternalProject.Model.orleans.g.cs:line 20736
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.ConstructorInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.Extensions.DependencyInjection.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ActivatorUtilities.CreateInstance(IServiceProvider provider, Type instanceType, Object[] parameters)
   at Microsoft.Extensions.DependencyInjection.ActivatorUtilities.GetServiceOrCreateInstance[T](IServiceProvider provider)
   at Orleans.Serialization.GeneratedCodeHelpers.OrleansGeneratedCodeHelper.GetService[TService](Object caller, ICodecProvider codecProvider) in /_/src/Orleans.Serialization/GeneratedCodeHelpers/OrleansGeneratedCodeHelper.cs:lin
e 72

Now for the interesting part: when I copy paste the exact same class in my main project I no longer get this error, and the class serializes perfectly fine.

I tried referencing the Nuget project directly in the main project(ProjectReference), the result is the same.

I only get rid of errors in two scenarios:

1. when I use the Json Serializer with IgnoreCycles (undesirable in my scenario, doesn't get rid of the underlying problem)

siloBuilder.Services.AddSerializer(sb =>
                    {
                        sb.AddJsonSerializer(
                            isSupported: type => type.Namespace.StartsWith("Nuget.Namespace"),
                            new JsonSerializerOptions()
                            {
                                ReferenceHandler = ReferenceHandler.IgnoreCycles
                            }
                        );
                    });

2. when I add the classes directly in the main project. Example class from the error:

[GenerateSerializer]
    public sealed class Class1: Class1Base
    {
        public Class1()
        {
            
        }

        public Class1(IRequest request, bool boolean, InternalClass internalClass) 
            : base(request, boolean)
        {
            InternalClass = internalClass;
        }

        [Id(0)]
        public InternalClass InternalClass{ get; set; }
        
    }

   [GenerateSerializer]
    public class Class1Base
    {
        public Class1Base()
        {
            
        }
        public Class1Base(IRequest request, bool boolean)
        {
            Request = request;
            Boolean= boolean;
        }

        [Id(0)]
        public IRequest Request { get; set; }
        [Id(1)]
        public bool Boolean{ get; set; }
    }

[david-obee]

I have found the same issue. I believe it stems from the fact that the code generation seems to only run on the assembly being compiled, and on dependencies explicitly linked for code generation to run on:

https://github.com/dotnet/orleans/blob/v7.1.0/src/Orleans.CodeGenerator/CodeGenerator.cs#L428

When determining which assemblies to examine (and hence to run code generation on), this starts at the compiling assembly, and the recursively looks at other assemblies that have been linked with the GenerateCodeForDeclaringAssemblyAttribute.

So in my case where I have types in another project in my solution, I was able to add

[assembly: GenerateCodeForDeclaringAssembly(typeof(SomeTypeInMyOtherAssembly))]

in my project, instruct the code generation to also generate serializers for types in that other assembly. I believe this should work in the same way for NuGet packages.

This isn't great in our case though. We have a library that sits on top of Orleans, and it means that consumers of that library have to instruct Orleans explicitly to run code generation on the library, rather than just using it and having it work automatically. I haven't yet found another way around that.

[Professional-amateur-dev]

Thanks for the code snippet, the error is now gone.

I now have another error:
Could not find a codec for type System.Text.Json.JsonElement.

The error is for all System.Objects I have in the project, example object:
[Id(0), Immutable] [JsonPropertyName("ext")] [DataMember(Name ="ext")] public object ObjectExtraInfo { get; set; }

Similar thing is happening with Interfaces, like in my example class in the question, which has a IRequest Interface.
For now I've replaced the interface and object with a class and it works (in this specific class), but that's not a real solution.

I need to have serializable Interfaces and System.Object since I have another ~1000 classes and they are used everywhere. Interfaces are required because we have a generic IRequest interface that is shared between multiple project, and now I have to always Map it to a Internal class and then map it again when I'm sending it to other services.
Objects are required because we don't always know what we will receive in that object, so it need to stay as System.Object.

Furthermore, you should add support to import all classes with GeneratedCodeAttribute in a specific assembly. Currently we need to manually add all the classes with that attribute, which is time consuming and unnecessary.

[adityamandaleeka]

Triage: we could consider providing codecs out of the box for JsonElement and the like.

As a workaround for now you could try serializing/deserializing your JSON data as strings.

[yoDon]

Just hit this. Looks like a huge problem for anyone using class libraries. Aborting our attempt to move from Orleans 3.x to Orleans 7. More evidence 7 is not ready for primetime.

[ReubenBond]

@yoDon I believe this is by design and there's nothing to fix or improve here except as noted above, we may include codecs for System.Text.Json & Newtonsoft internal types in case they are being sent directly over the wire.

Orleans 7 moves away from the automatic (i.e. magic) serialization approach and requires you to annotate classes & members explicitly. The auto approach was fragile to changes in your data model, often putting you in an unrecoverable situation, eg if you added or removed a field from a type. You can use JSON, Protocol Buffers, or another serialization library instead, if you prefer: https://learn.microsoft.com/en-us/dotnet/orleans/host/configuration-guide/serialization-configuration?pivots=orleans-7-0

There is no issue with serializing types from libraries.

Docs: https://learn.microsoft.com/en-us/dotnet/orleans/host/configuration-guide/serialization?pivots=orleans-7-0

If there's something we can help you with, please let us know. I am less available this week because I am on vacation, but I'll try to respond regardless

[yoDon]

@ReubenBond thanks for clarifying that the behavior @Professional-amateur-dev and @david-obee mentioned here is expected/by-design. Given that GenerateCodeForDeclaringAssembly doesn't seem to be documented anywhere, I hope it's ok I've created a couple separate more granular issues to capture this documentation issue and a couple other not-obvious-to-me issues I encountered while porting a non-trivial Orleans 3.x project to Orleans 7.

I'm a huge fan of Orleans, and have been for many years. Hopefully if there are misconceptions in any of the issues below those misconceptions can at least be useful insights for whomever is building docs to help devs successfully adopt Orleans.

• GenerateCodeForDeclaringAssembly is critical and undocumented #8520
• [GenerateSerializer] fails silently #8521
• IPersistentState<> should generate compile-time errors not runtime errors #8522
• Orleans serializer code-fix questions #8523

[onionhammer]

Any update on this? seems like JsonElement is a common enough type for use in data transferring that it should be supported OOTB

[ilya-girman]

I have same problem of "Orleans.Serialization.CodecNotFoundException : Could not find a codec" but in unit tests.
It seems like It's impossible to write xunit tests with Microsoft.Orleans.TestingHost if in main project there are classes without GenerateSerializer attribute.
I have project with common types: DomainObjects. Also I have let's say AllGrainsProject and TestsGrainsPoject (XUnit). Dependencies are follows: TestsGrainsPoject => AllGrainsProject =>DomainObjects

AllGrainsProject works fine by the help of
siloBuilder.ConfigureServices(services => { services.AddSerializer(serializerBuilder => { serializerBuilder.AddJsonSerializer(isSupported: type => type.Namespace.StartWith("DomainObjects");

But the same trick with TestClusterBuilder from Microsoft.Orleans.TestingHost doesn't work!
Approaches with [assembly: GenerateCodeForDeclaringAssembly(typeof(DomainObjects.ACD.RecordListLine))] and IgnoreCycles in XUnit project didn't help at all!

[ReubenBond]

@ilya-girman do you have Microsoft.Orleans.Sdk installed in your test project?

[remcoros]

@ilya-girman @ReubenBond

I fixed this by adding 'AddSerializer' to both the silo and client configuration of the test cluster:

builder.AddSiloBuilderConfigurator<SiloBuilderConfigurator>();
builder.AddClientBuilderConfigurator<ClientBuilderConfigurator>();

    private class SiloBuilderConfigurator : ISiloConfigurator
    {
        public void Configure(ISiloBuilder siloBuilder)
        {
            siloBuilder.Services.AddSerializer(serializerBuilder =>
            {
                serializerBuilder.AddNewtonsoftJsonSerializer(isSupported: type => true);
            });
        }
    }

    private class ClientBuilderConfigurator : IClientBuilderConfigurator
    {
        public void Configure(IConfiguration configuration, IClientBuilder clientBuilder)
        {
            clientBuilder.Services.AddSerializer(serializerBuilder =>
            {
                serializerBuilder.AddNewtonsoftJsonSerializer(isSupported: type => true);
            });
        }
    }

[jbusuttil83]

@ReubenBond I have grain in my projects that accept CancellationToken in the grain interface methods. What do you suggest for serializing these with the new Orleans version?

[ReubenBond]

@jbusuttil83 once #9127 is merged, that scenario will work. Serialization of a cancellation token would not be enough.