refactor(finalizer): Rework finalizer in controllers (#625)

This also adds integration tests for the system. (and some todos). BREAKING CHANGE: Finalizers are registered with an identifier now. The identifier is generated by the KubeOps.Generator when used. Finalizers are attached via EntityFinalizerAttacher<> delegates that attach the finalizer to an entity.
buehler · Oct 3, 2023 · 5fc23d3 · 5fc23d3
1 parent 3a95bd5
commit 5fc23d3
Show file tree

Hide file tree

Showing 40 changed files with 1,134 additions and 71 deletions.
diff --git a/.config/dotnet-tools.json b/.config/dotnet-tools.json
@@ -7,12 +7,6 @@
       "commands": [
         "docfx"
       ]
-    },
-    "kubeops.cli": {
-      "version": "8.0.0-pre.8",
-      "commands": [
-        "kubeops"
-      ]
     }
   }
-}
+}
diff --git a/README.md b/README.md
@@ -7,23 +7,22 @@
 This is the repository of "KubeOps" - The dotnet Kubernetes Operator SDK.
 
 The documentation is provided in the code itself (description of the methods and classes)
-and each package contains a README with further information/documentation.
-
-Also, there is a `docfx` site that provides further documentation and examples.
-You can find it [here](https://buehler.github.io/dotnet-operator-sdk/).
+and each package contains a README with further information/documentation. For a more
+detailed documentation, head to the [GitHub Pages](https://buehler.github.io/dotnet-operator-sdk/).
 
 ## Packages
 
 The following packages exist:
 
-| Package                                                      | Description                                         | Version                                                                                                               | Pre Version                                                                                                                                                     |
-|--------------------------------------------------------------|-----------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| [KubeOps.Abstractions](./src/KubeOps.Abstractions/README.md) | Contains abstractions, attributes, etc. for the SDK | [![Nuget](https://img.shields.io/nuget/v/KubeOps.Abstractions)](https://www.nuget.org/packages/KubeOps.Abstractions/) | [![Nuget](https://img.shields.io/nuget/vpre/KubeOps.Abstractions?label=nuget%20prerelease)](https://www.nuget.org/packages/KubeOps.Abstractions/absoluteLatest) |
-| [KubeOps.Cli](./src/KubeOps.Cli/README.md)                   | CLI Dotnet Tool to generate stuff                   | [![Nuget](https://img.shields.io/nuget/v/KubeOps.Cli)](https://www.nuget.org/packages/KubeOps.Cli/)                   | [![Nuget](https://img.shields.io/nuget/vpre/KubeOps.Cli?label=nuget%20prerelease)](https://www.nuget.org/packages/KubeOps.Cli/absoluteLatest)                   |
-| [KubeOps.Generator](./src/KubeOps.Generator/README.md)       | Source Generator for the SDK                        | [![Nuget](https://img.shields.io/nuget/v/KubeOps.Generator)](https://www.nuget.org/packages/KubeOps.Generator/)       | [![Nuget](https://img.shields.io/nuget/vpre/KubeOps.Generator?label=nuget%20prerelease)](https://www.nuget.org/packages/KubeOps.Generator/absoluteLatest)       |
-| [KubeOps.Operator](./src/KubeOps.Operator/README.md)         | Main SDK entrypoint to create an operator           | [![Nuget](https://img.shields.io/nuget/v/KubeOps.Operator)](https://www.nuget.org/packages/KubeOps.Operator/)         | [![Nuget](https://img.shields.io/nuget/vpre/KubeOps.Operator?label=nuget%20prerelease)](https://www.nuget.org/packages/KubeOps.Operator/absoluteLatest)         |
-| [KubeOps.Operator.Web](./src/KubeOps.Operator.Web/README.md) | Web part of the operator (for webhooks)             | [![Nuget](https://img.shields.io/nuget/v/KubeOps.Operator.Web)](https://www.nuget.org/packages/KubeOps.Operator.Web/) | [![Nuget](https://img.shields.io/nuget/vpre/KubeOps.Operator.Web?label=nuget%20prerelease)](https://www.nuget.org/packages/KubeOps.Operator.Web/absoluteLatest) |
-| [KubeOps.Transpiler](./src/KubeOps.Transpiler/README.md)     | Transpilation helpers for CRDs and RBAC elements    | [![Nuget](https://img.shields.io/nuget/v/KubeOps.Transpiler)](https://www.nuget.org/packages/KubeOps.Transpiler/)     | [![Nuget](https://img.shields.io/nuget/vpre/KubeOps.Transpiler?label=nuget%20prerelease)](https://www.nuget.org/packages/KubeOps.Transpiler/absoluteLatest)     |
+| Package                                                              | Description                                            | Framework Support                           | Latest Version                                                                                                                                                          |
+|----------------------------------------------------------------------|--------------------------------------------------------|---------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| [KubeOps.Abstractions](./src/KubeOps.Abstractions/README.md)         | Contains abstractions, attributes, etc. for the SDK    | netstandard2.0 netstandard2.1 net6.0 net7.0 | [![Nuget](https://img.shields.io/nuget/vpre/KubeOps.Abstractions?label=nuget%20prerelease)](https://www.nuget.org/packages/KubeOps.Abstractions/absoluteLatest)         |
+| [KubeOps.Cli](./src/KubeOps.Cli/README.md)                           | CLI Dotnet Tool to generate stuff                      | net7.0                                      | [![Nuget](https://img.shields.io/nuget/vpre/KubeOps.Cli?label=nuget%20prerelease)](https://www.nuget.org/packages/KubeOps.Cli/absoluteLatest)                           |
+| [KubeOps.Generator](./src/KubeOps.Generator/README.md)               | Source Generator for the SDK                           | netstandard2.0                              | [![Nuget](https://img.shields.io/nuget/vpre/KubeOps.Generator?label=nuget%20prerelease)](https://www.nuget.org/packages/KubeOps.Generator/absoluteLatest)               |
+| [KubeOps.KubernetesClient](./src/KubeOps.KubernetesClient/README.md) | Extended client to communicate with the Kubernetes API | net6.0 net7.0                               | [![Nuget](https://img.shields.io/nuget/vpre/KubeOps.KubernetesClient?label=nuget%20prerelease)](https://www.nuget.org/packages/KubeOps.KubernetesClient/absoluteLatest) |
+| [KubeOps.Operator](./src/KubeOps.Operator/README.md)                 | Main SDK entrypoint to create an operator              | net6.0 net7.0                               | [![Nuget](https://img.shields.io/nuget/vpre/KubeOps.Operator?label=nuget%20prerelease)](https://www.nuget.org/packages/KubeOps.Operator/absoluteLatest)                 |
+| [KubeOps.Operator.Web](./src/KubeOps.Operator.Web/README.md)         | Web part of the operator (for webhooks)                | net6.0 net7.0                               | [![Nuget](https://img.shields.io/nuget/vpre/KubeOps.Operator.Web?label=nuget%20prerelease)](https://www.nuget.org/packages/KubeOps.Operator.Web/absoluteLatest)         |
+| [KubeOps.Transpiler](./src/KubeOps.Transpiler/README.md)             | Transpilation helpers for CRDs and RBAC elements       | netstandard2.0 netstandard2.1 net6.0 net7.0 | [![Nuget](https://img.shields.io/nuget/vpre/KubeOps.Transpiler?label=nuget%20prerelease)](https://www.nuget.org/packages/KubeOps.Transpiler/absoluteLatest)             |
 
 ## Contribution
 

diff --git a/examples/Operator/Controller/V1TestEntityController.cs b/examples/Operator/Controller/V1TestEntityController.cs
@@ -1,26 +1,46 @@
 using KubeOps.Abstractions.Controller;
+using KubeOps.Abstractions.Finalizer;
 using KubeOps.Abstractions.Rbac;
+using KubeOps.KubernetesClient;
 
 using Microsoft.Extensions.Logging;
 
 using Operator.Entities;
+using Operator.Finalizer;
 
 namespace Operator.Controller;
 
 [EntityRbac(typeof(V1TestEntity), Verbs = RbacVerb.All)]
 public class V1TestEntityController : IEntityController<V1TestEntity>
 {
     private readonly ILogger<V1TestEntityController> _logger;
+    private readonly IKubernetesClient<V1TestEntity> _client;
+    private readonly EntityFinalizerAttacher<FinalizerOne, V1TestEntity> _finalizer1;
+    private readonly EntityFinalizerAttacher<FinalizerTwo, V1TestEntity> _finalizer2;
 
-    public V1TestEntityController(ILogger<V1TestEntityController> logger)
+    public V1TestEntityController(
+        ILogger<V1TestEntityController> logger,
+        IKubernetesClient<V1TestEntity> client,
+        EntityFinalizerAttacher<FinalizerOne, V1TestEntity> finalizer1,
+        EntityFinalizerAttacher<FinalizerTwo, V1TestEntity> finalizer2)
     {
         _logger = logger;
+        _client = client;
+        _finalizer1 = finalizer1;
+        _finalizer2 = finalizer2;
     }
 
-    public Task ReconcileAsync(V1TestEntity entity)
+    public async Task ReconcileAsync(V1TestEntity entity)
     {
         _logger.LogInformation("Reconciling entity {Entity}.", entity);
-        return Task.CompletedTask;
+
+        entity = await _finalizer1(entity);
+        entity = await _finalizer2(entity);
+
+        entity.Status.Status = "Reconciling";
+        entity = await _client.UpdateStatus(entity);
+        entity.Status.Status = "Reconciled";
+        await _client.UpdateStatus(entity);
     }
 
     public Task DeletedAsync(V1TestEntity entity)

diff --git a/examples/Operator/Entities/V1TestEntity.cs b/examples/Operator/Entities/V1TestEntity.cs
@@ -1,20 +1,11 @@
-using k8s;
-using k8s.Models;
+using k8s.Models;
+
+using KubeOps.Abstractions.Entities;
 
 namespace Operator.Entities;
 
 [KubernetesEntity(Group = "testing.dev", ApiVersion = "v1", Kind = "TestEntity")]
-public class V1TestEntity : IKubernetesObject<V1ObjectMeta>, ISpec<V1TestEntitySpec>, IStatus<V1TestEntityStatus>
+public class V1TestEntity : CustomKubernetesEntity<V1TestEntitySpec, V1TestEntityStatus>
 {
-    public required string ApiVersion { get; set; }
-
-    public required string Kind { get; set; }
-
-    public V1ObjectMeta Metadata { get; set; } = new();
-
-    public V1TestEntitySpec Spec { get; set; } = new();
-
-    public V1TestEntityStatus Status { get; set; } = new();
-
     public override string ToString() => $"Test Entity ({Metadata.Name}): {Spec.Username} ({Spec.Email})";
 }
diff --git a/examples/Operator/Finalizer/FinalizerOne.cs b/examples/Operator/Finalizer/FinalizerOne.cs
@@ -0,0 +1,13 @@
+using KubeOps.Abstractions.Finalizer;
+
+using Operator.Entities;
+
+namespace Operator.Finalizer;
+
+public class FinalizerOne : IEntityFinalizer<V1TestEntity>
+{
+    public Task FinalizeAsync(V1TestEntity entity)
+    {
+        return Task.CompletedTask;
+    }
+}
diff --git a/examples/Operator/Finalizer/FinalizerTwo.cs b/examples/Operator/Finalizer/FinalizerTwo.cs
@@ -0,0 +1,13 @@
+using KubeOps.Abstractions.Finalizer;
+
+using Operator.Entities;
+
+namespace Operator.Finalizer;
+
+public class FinalizerTwo : IEntityFinalizer<V1TestEntity>
+{
+    public Task FinalizeAsync(V1TestEntity entity)
+    {
+        return Task.CompletedTask;
+    }
+}
diff --git a/examples/Operator/Program.cs b/examples/Operator/Program.cs
@@ -9,7 +9,7 @@
 
 builder.Services
     .AddKubernetesOperator()
-    .RegisterResources();
+    .RegisterComponents();
 
 using var host = builder.Build();
 await host.RunAsync();
diff --git a/examples/Operator/Properties/launchSettings.json b/examples/Operator/Properties/launchSettings.json
@@ -0,0 +1,8 @@
+{
+  "$schema": "http://json.schemastore.org/launchsettings.json",
+  "profiles": {
+    "Operator": {
+      "commandName": "Project"
+    }
+  }
+}
diff --git a/examples/Operator/todos.txt b/examples/Operator/todos.txt
@@ -1,10 +1,15 @@
 todo:
-- finalizer
 - events
 - leadership election
 - requeue
 - build targets
 - other CLI commands
+- error handling
 - web: webhooks
+- docs
 - cache?
 - try .net 8 AOT?
+- client:
+        // TODO: make all sync calls as well.
+        // TODO: test list / get call.
+        // TODO: update list call
diff --git a/renovate.json b/renovate.json
@@ -3,6 +3,6 @@
   "extends": ["github>buehler/renovate-config", ":disableMajorUpdates"],
   "ignorePaths": ["_old/"],
   "dotnet": {
-    "ignoreDeps": ["StyleCop.Analyzers", "KubeOps.Cli"]
+    "ignoreDeps": ["StyleCop.Analyzers"]
   }
 }
diff --git a/src/KubeOps.Abstractions/Builder/IOperatorBuilder.cs b/src/KubeOps.Abstractions/Builder/IOperatorBuilder.cs
@@ -3,6 +3,7 @@
 
 using KubeOps.Abstractions.Controller;
 using KubeOps.Abstractions.Entities;
+using KubeOps.Abstractions.Finalizer;
 
 using Microsoft.Extensions.DependencyInjection;
 
@@ -57,4 +58,22 @@ IOperatorBuilder AddController<TImplementation, TEntity>()
     IOperatorBuilder AddControllerWithEntity<TImplementation, TEntity>(EntityMetadata metadata)
         where TImplementation : class, IEntityController<TEntity>
         where TEntity : IKubernetesObject<V1ObjectMeta>;
+
+    /// <summary>
+    /// Add a finalizer implementation for a specific entity.
+    /// This adds the implementation as a transient service and registers
+    /// the finalizer with the provided identifier. Then an
+    /// <see cref="EntityFinalizerAttacher{TImplementation,TEntity}"/> is registered to
+    /// provide a delegate for attaching the finalizer to an entity.
+    /// </summary>
+    /// <param name="identifier">
+    /// The identifier for the finalizer.
+    /// This string is added to the Kubernetes entity as a finalizer.
+    /// </param>
+    /// <typeparam name="TImplementation">Type of the finalizer implementation.</typeparam>
+    /// <typeparam name="TEntity">Type of the Kubernetes entity.</typeparam>
+    /// <returns>The builder for chaining.</returns>
+    IOperatorBuilder AddFinalizer<TImplementation, TEntity>(string identifier)
+        where TImplementation : class, IEntityFinalizer<TEntity>
+        where TEntity : IKubernetesObject<V1ObjectMeta>;
 }
diff --git a/src/KubeOps.Abstractions/Entities/CustomKubernetesEntity{TSpec,TStatus}.cs b/src/KubeOps.Abstractions/Entities/CustomKubernetesEntity{TSpec,TStatus}.cs
@@ -1,4 +1,6 @@
-using k8s;
+using System.Text.Json.Serialization;
+
+using k8s;
 
 namespace KubeOps.Abstractions.Entities;
 
@@ -17,5 +19,6 @@ public abstract class CustomKubernetesEntity<TSpec, TStatus> : CustomKubernetesE
     /// <summary>
     /// Status object for the entity.
     /// </summary>
+    // [JsonPropertyName("status")]
     public TStatus Status { get; set; } = new();
 }
diff --git a/src/KubeOps.Abstractions/Finalizer/EntityFinalizerAttacher.cs b/src/KubeOps.Abstractions/Finalizer/EntityFinalizerAttacher.cs
@@ -0,0 +1,41 @@
+using k8s;
+using k8s.Models;
+
+namespace KubeOps.Abstractions.Finalizer;
+
+/// <summary>
+/// <para>
+/// Injectable delegate for finalizers. This delegate is used to attach a finalizer
+/// with its identifier to an entity. When injected, simply call the delegate with
+/// the entity to attach the finalizer.
+/// </para>
+/// <para>
+/// As with other (possibly) mutating calls, use the returned entity for further
+/// modification and Kubernetes client interactions, since the resource version
+/// is updated each time the entity is modified.
+/// </para>
+/// </summary>
+/// <typeparam name="TImplementation">The type of the entity finalizer.</typeparam>
+/// <typeparam name="TEntity">The type of the Kubernetes entity.</typeparam>
+/// <param name="entity">The instance of the entity, that the finalizer is attached if needed.</param>
+/// <returns>A <see cref="Task"/> that resolves when the finalizer was attached.</returns>
+/// <example>
+/// <code>
+/// [EntityRbac(typeof(V1TestEntity), Verbs = RbacVerb.All)]
+/// public class V1TestEntityController : IEntityController&lt;V1TestEntity&gt;
+/// {
+///     private readonly EntityFinalizerAttacher&lt;FinalizerOne, V1TestEntity&gt; _finalizer1;
+///
+///     public V1TestEntityController(
+///         EntityFinalizerAttacher&lt;FinalizerOne, V1TestEntity&gt; finalizer1) => _finalizer1 = finalizer1;
+///
+///     public async Task ReconcileAsync(V1TestEntity entity)
+///     {
+///         entity = await _finalizer1(entity);
+///     }
+/// }
+/// </code>
+/// </example>
+public delegate Task<TEntity> EntityFinalizerAttacher<TImplementation, TEntity>(TEntity entity)
+    where TImplementation : IEntityFinalizer<TEntity>
+    where TEntity : IKubernetesObject<V1ObjectMeta>;
diff --git a/src/KubeOps.Abstractions/Finalizer/IEntityFinalizer{TEntity}.cs b/src/KubeOps.Abstractions/Finalizer/IEntityFinalizer{TEntity}.cs
@@ -0,0 +1,19 @@
+using k8s;
+using k8s.Models;
+
+namespace KubeOps.Abstractions.Finalizer;
+
+/// <summary>
+/// Finalizer for an entity.
+/// </summary>
+/// <typeparam name="TEntity">The type of the entity.</typeparam>
+public interface IEntityFinalizer<in TEntity>
+    where TEntity : IKubernetesObject<V1ObjectMeta>
+{
+    /// <summary>
+    /// Finalize an entity that is pending for deletion.
+    /// </summary>
+    /// <param name="entity">The kubernetes entity that needs to be finalized.</param>
+    /// <returns>A task that resolves when the operation is done.</returns>
+    Task FinalizeAsync(TEntity entity);
+}
diff --git a/src/KubeOps.Cli/Properties/launchSettings.json b/src/KubeOps.Cli/Properties/launchSettings.json
@@ -0,0 +1,30 @@
+{
+  "$schema": "http://json.schemastore.org/launchsettings.json",
+  "profiles": {
+    "CLI Generate RBAC": {
+      "commandName": "Project",
+      "workingDirectory": "$(ProjectDir)",
+      "commandLineArgs": "gen rbac ../../examples/Operator/Operator.csproj"
+    },
+    "CLI Generate Cert": {
+      "commandName": "Project",
+      "workingDirectory": "$(ProjectDir)",
+      "commandLineArgs": "g cert test-operator"
+    },
+    "CLI Generate CRDs": {
+      "commandName": "Project",
+      "workingDirectory": "$(ProjectDir)",
+      "commandLineArgs": "g c ../../examples/Operator/Operator.csproj"
+    },
+    "CLI Install": {
+      "commandName": "Project",
+      "workingDirectory": "$(ProjectDir)",
+      "commandLineArgs": "i ../../examples/Operator/Operator.csproj -f"
+    },
+    "CLI Uninstall": {
+      "commandName": "Project",
+      "workingDirectory": "$(ProjectDir)",
+      "commandLineArgs": "u ../../examples/Operator/Operator.csproj -f"
+    }
+  }
+}
diff --git a/src/KubeOps.Generator/Generators/ControllerRegistrationGenerator.cs b/src/KubeOps.Generator/Generators/ControllerRegistrationGenerator.cs
@@ -12,7 +12,7 @@
 namespace KubeOps.Generator.Generators;
 
 [Generator]
-public class ControllerRegistrationGenerator : ISourceGenerator
+internal class ControllerRegistrationGenerator : ISourceGenerator
 {
     private readonly EntityControllerSyntaxReceiver _ctrlReceiver = new();
     private readonly KubernetesEntitySyntaxReceiver _entityReceiver = new();

diff --git a/src/KubeOps.Generator/Generators/EntityDefinitionGenerator.cs b/src/KubeOps.Generator/Generators/EntityDefinitionGenerator.cs
@@ -12,7 +12,7 @@
 namespace KubeOps.Generator.Generators;
 
 [Generator]
-public class EntityDefinitionGenerator : ISourceGenerator
+internal class EntityDefinitionGenerator : ISourceGenerator
 {
     public void Initialize(GeneratorInitializationContext context)
     {