Knative Serving use of CD Events, Use Cases and PoC

[salaboy]

To refine and improve the CD Events vocabulary and show its potential on integrating different technologies and covering different use cases I want to propose a new set of use cases covering Knative Serving, CD Events and Tekton Pipelines.

Knative Serving is closely related with the Continuous Deployment pipeline events

Use Case 1:

• A Tekton Pipeline builds a service and produce a new artifact (container image). A CD Event is emitted.
• Knative Serving is capable of understanding this event and:
  • check if there is a service that can be updated with the new artifact, and perform the update
  • If there is no service and it is configured correctly, it can create a new service with the newly produced artifact

Use Case 2:

• A Knative Service is deployed and a CD Event is emitted
• A Tekton pipeline is triggered to run a set of integration tests against the newly deployed service

While these use cases can be implemented by imperatively interacting with Knative from Tekton or by creating a request from Knative to Tekton, the CD events approach decouple the interactions between these systems promoting wider ecosystem integration possibilities and reducing the dependencies between the different parties. At the same time, it helps us to improve the vocabulary with working examples using real-life projects and implementations.

These use cases can be extended by adding Keptn to have more control on what actions to perform on Knative Serving CD Events.

An ongoing PoC can be found here thanks to @ishankhare07 who is doing an amazing work 🤘

[salaboy]

The proposal includes consuming events from the Continuous Integration Pipeline Events such as cd.artifact.packaged.v1 and emitting events from the Continuous Deployment Pipeline Events such as cd.service.created.v1 or cd.service.upgraded.v1

As mentioned in the issue description, I think that we would benefit by implementing the described use cases with Tekton and then expanding to other projects.

This PoC will allow us to evolve the definition of the categories in the vocabulary to expand the type of use cases that can be implemented using the current CD Events SDK.

I am currently concerned about the semantics of the events that we have described in the vocabulary, as I believe that we need to expand the vocabulary to cover both inbound and outbound events that make sense, no matter which technology we use. For example:

• In Tekton, if a pipeline runs, we know which event to emit, but it is not clear what event, or what the semantics of that event should be if we want to trigger a pipeline to run.
• In Knative Serving, when a new service is deployed, we can easily emit events, but what event from the spec should we listen to perform a deployment?

For both examples, if we let users do the mapping, which is the most flexible approach, we push implementations to build these custom components. I have a gut feeling that tells me that what we need to provide is a way to expose which operations can be performed (running a pipeline, deploying a service, etc) and then how cd.events maps to those operations. I have the feeling that all incoming events to projects will require this level of mappings and custom coding these mappings might be to much for implementations to adopt CD Events if we don't provide those helpers.

[erkist]

Seems like a very concrete and intuitively valuable use case. I am in favor of letting this use case help define events and event content.

[afrittoli]

@salaboy @ishakhare07 thanks for this use case.

About outgoing / incoming events, I think the discussion is related to the declarative vs. imperative events discussion we had in the past. IIUC we agreed that we want our events to be declarative, and let the receiver side decide what to do with them.
I believe use cases and proof of concepts is where we:

• validate that there is enough information in the declarative events is enough for the receiving side to do something meaningful, in a number of use cases that we identify and document
• show our users what we believe are best practices in consuming events

I would not go as far as adding anything to the spec though that tells implementors what to do when they receive an event.

[afrittoli]

About the events required for this use case, in the spec today we have:

Service Deployed: a new instance of the Service has been deployed
Service Upgraded: an existing instance of a Service has been upgraded to a new version
Service Rolledback: an existing instance of a Service has been rolledback to a previous version
Service Removed: an existing instance of a Service has been terminated an it is not longer present in an environment

In this issues you're proposing:

Service Created
Service Upgraded
Service Removed
Service Published

What is the relationship with existing events? Are they mean to replace the original ones, or add to them?
Could you create an issue or a PR to the spec (cdevents/spec), with more context and documentation for the events?

Thank you!

[ishankhare07]

@afrittoli I think I can clarify a bit about some of the existing/proposed events and their relationships.
Talking about

1. Service Upgraded
2. Service Removed

I think the idea here was to just add a few more details to the existing events while keeping whats already there. These mostly go to the event's extensions fields.

Coming to the other ones, here's what our initial thoughts were:

1. Service Created: used when a Knative service appears for the first time.
2. Service Deployed: used when the above Knative service is ready.
3. Service Published: while a more concrete use-case for this can be discussed but we thought we should keep it around for a scenario like
   • When a new revision is created / becomes active.
   • When we are able to detect a URL for a particular Knative service

Would like to know @salaboy 's thought on that as well

[salaboy]

@ishankhare07 @afrittoli good questions

We created the vocabulary without having any real example in mind and we will need to start refining the events, names and semantics as we go.

@ishankhare07 I think that we can create a separate issue for adding Service Published to the vocabulary and in the meantime we use Deployed, Upgraded and Removed which are already defined.

@afrittoli I think that we need Service Published to let other parties know when a Service is accessible by users (in this case outside the cluster). I believe that by implementing this PoC at least we have a justfication for each of the events in the vocabulary, and I wouldn't be too strict about modifying what is in there, as we defined them without concrete use cases in mind. I will keep an eye to make sure that we don't include events or change the vocabulary with system or use case specific events that doesn't apply to other scenarios/technologies. Does this sound good to you?

[ishankhare07]

thanks for the clarification @salaboy

Also to reflecting on your suggestion and to add a bit more context to it, I'll add these following two flowcharts

1. First Flow, we have this flow implemented partially where we trigger the cd.artifact.published.v1 the POC we have created so far. We will be enhancing this with tekton components as we go along.
   
2. Second flow
   

[salaboy]

@afrittoli can you check these diagrams to validate that they make sense? We will move ahead trying to implement this and I've opened an issue for adding just one new event type.