Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flagger addon #440

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions docs/addons/flagger.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
## Flagger Add-On

[Flagger](https://flagger.app/) is a progressive delivery tool that automates the release process for applications running on Kubernetes. It reduces the risk of introducing a new software version in production by gradually shifting traffic to the new version while measuring metrics and running conformance tests. The Flagger add-on provisions the necessary Helm chart, and namespace to allow support for flagger in an EKS workload.

## Usage

```typescript
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import * as blueprints from '@aws-quickstart/eks-blueprints';

const app = new cdk.App();

const addOn = new blueprints.addons.Flagger();

const blueprint = blueprints.EksBlueprint.builder()
.addOns(addOn)
.build(app, 'my-stack-name');
```

## Functionality
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you want to describe supported configuration options?


1. Creates the `flagger` namespace.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this parameter is optional and may provided by the user in the namespace field of your addon props.

Copy link
Contributor Author

@Eli1123 Eli1123 Jul 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed it to this: 1. Creates the 'flagger' namespace. This parameter is optional and may be provided by the user in the namespace field of your addon props.

2. Deploys the `flagger` Helm chart into the cluster.
3. Supports [standard helm configuration options](./index.md#standard-helm-add-on-configuration-options)
19 changes: 10 additions & 9 deletions docs/addons/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,30 +12,31 @@ The framework currently supports the following add-ons.
|------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------|
| [`AppMeshAddOn`](./app-mesh) | Adds an AppMesh controller and CRDs (pending validation on the latest version of CDK) |
| [`ArgoCDAddOn`](./argo-cd) | Provisions Argo CD into your cluster. |
| [`AWS for Fluent Bit`](./aws-for-fluent-bit) | Provisions Fluent Bit into your cluster for log aggregation and consumption. |
| [`AWS for Fluent Bit`](./aws-for-fluent-bit) | Provisions Fluent Bit into your cluster for log aggregation and consumption. |
| [`AWS Load Balancer Controller`](./aws-load-balancer-controller) | Provisions the AWS Load Balancer Controller into your cluster. |
| [`AWS Node Termination Handler`](./aws-node-termination-handler) | Provisions Node Termination Handler into your cluster. |
| [`AWS Node Termination Handler`](./aws-node-termination-handler) | Provisions Node Termination Handler into your cluster. |
| [`CalicoAddOn`](./calico) | Adds the Calico 1.7.1 CNI/Network policy engine |
| [`ClusterAutoscalerAddOn`](./cluster-autoscaler) | Adds the standard cluster autoscaler |
| [`ContainerInsightsAddOn`](./container-insights) | Adds Container Insights support integrating monitoring with CloudWatch |
| [`CoreDnsAddOn`](./coredns.md) | Adds CoreDNS Amazon EKS add-on. CoreDNS is a flexible, extensible DNS server that can serve as the Kubernetes cluster DNS |
| [`DatadogAddOn`](./datadog.md) | Adds [Datadog](https://www.datadoghq.com/) Amazon EKS add-on. Datadog is the monitoring and security platform for cloud applications. |
| [`Dynatrace`](https://github.com/dynatrace-oss/dynatrace-eks-blueprints-addon) | Adds the [Dynatrace](https://www.dynatrace.com/) [OneAgent Operator](https://github.com/Dynatrace/dynatrace-oneagent-operator) |
| [`DatadogAddOn`](./datadog.md) | Adds [Datadog](https://www.datadoghq.com/)| Amazon EKS add-on. Datadog is the monitoring and security platform for cloud applications. |
| [`Dynatrace`](https://github.com/dynatrace-oss/dynatrace-eks-blueprints-addon)| Adds the [Dynatrace](https://www.dynatrace.com/) [OneAgent Operator](https://github.com/Dynatrace/dynatrace-oneagent-operator) |
| [`EbsCsiDriverAddOn`](./ebs-csi-driver.md) | Adds EBS CSI Driver Amazon EKS add-on. This driver manages the lifecycle of Amazon EBS volumes for persistent storage |
| [`EfsCsiDriverAddOn`](./efs-csi-driver.md) | Adds EFS CSI Driver Amazon EKS add-on. This driver manages the lifecycle of Amazon EFS volumes for persistent storage |
| [`ExternalDnsAddOn`](./external-dns) | Adds [External DNS](https://github.com/kubernetes-sigs/external-dns) support for AWS to the cluster, integrating with Amazon Route 53 |
| [`Keptn`](https://github.com/keptn-sandbox/keptn-eks-blueprints-addon) | [Keptn](https://keptn.sh/) Control Plane and Execution Plane AddOn |
| [`FlaggerAddOn`](./flagger) | Adds support for Flagger progressive delivery and adds CRDs for Canary |
shapirov103 marked this conversation as resolved.
Show resolved Hide resolved
| [`Keptn`](https://github.com/keptn-sandbox/keptn-eks-blueprints-addon) | [Keptn](https://keptn.sh/) Control Plane and Execution Plane AddOn |
| [`KubecostAddOn`](./kubecost.md) | Adds [Kubecost](https://kubecost.com) cost analyzer to the EKS cluster |
| [`KubeviousAddOn`](./kubevious.md) | Adds [Kubevious](https://github.com/kubevious/kubevious) open source Kubernetes dashboard to an EKS cluster | |
| [`KubeviousAddOn`](./kubevious.md) | Adds [Kubevious](https://github.com/kubevious/kubevious) open source Kubernetes dashboard to an EKS cluster |
| [`KarpenterAddOn`](./karpenter.md) | Adds [Karpenter](https://github.com/awslabs/karpenter) support for Amazon EKS. |
| [`KubeProxyAddOn`](./kube-proxy.md) | Adds kube-proxy Amazon EKS add-on. Kube-proxy maintains network rules on each Amazon EC2 node |
| [`MetricsServerAddOn`](./metrics-server) | Adds metrics server (pre-req for HPA and other monitoring tools) |
| [`NewRelicAddOn`](./newrelic.md) | Adds [New Relic](https://newrelic.com/) and [Pixie](https://pixielabs.ai/) observability for Amazon EKS.|
| [`NewRelicAddOn`](./newrelic.md) | Adds [New Relic](https://newrelic.com/) and [Pixie](https://pixielabs.ai/) observability for Amazon EKS. |
| [`NginxAddOn`](./nginx.md) | Adds NGINX ingress controller |
| [`OpaGatekeeperAddOn (Currently Not Supported, In Progress)`](./opa-gatekeeper.md) | Adds OPA Gatekeeper |
| [`OpaGatekeeperAddOn (Currently Not Supported, In Progress)`](./opa-gatekeeper.md)| Adds OPA Gatekeeper |
| [`PixieAddOn`](./pixie.md) | Adds [Pixie](https://px.dev) to the EKS Cluster. Pixie provides auto-telemetry for requests, metrics, application profiles, and more. |
| [`SecretsStoreAddOn`](./secrets-store.md) | Adds AWS Secrets Manager and Config Provider for Secret Store CSI Driver to the EKS Cluster |
| [`Snyk`](https://github.com/snyk-partners/snyk-monitor-eks-blueprints-addon) | Adds the [Snyk Monitor](https://github.com/snyk/kubernetes-monitor) to the EKS Cluster |
| [`Snyk`](https://github.com/snyk-partners/snyk-monitor-eks-blueprints-addon) | Adds the [Snyk Monitor](https://github.com/snyk/kubernetes-monitor) to the EKS Cluster |
| [`SSMAgentAddOn`](./ssm-agent.md) | Adds [Amazon SSM Agent](https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent.html) to worker nodes |
| [`VpcCniAddOn`](./vpc-cni.md) | Adds the Amazon VPC CNI Amazon EKS addon to support native VPC networking for Amazon EKS |
| [`VeleroAddOn`](./velero.md) | Adds [Velero](https://velero.io/) to the EKS Cluster |
Expand Down
9 changes: 5 additions & 4 deletions examples/blueprint-construct/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { Construct } from "constructs";
import * as blueprints from '../../lib';
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix the GitHub actions warnings.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure what you mean by that? Are you saying I should have it instead of ../../lib be the import * as blueprints from '@aws-quickstart/eks-blueprints'?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

import { HelmAddOn } from '../../lib';
import * as team from '../teams';
import {FlaggerAddOn} from '../../lib/addons/flagger'

const burnhamManifestDir = './examples/teams/team-burnham/';
const rikerManifestDir = './examples/teams/team-riker/';
Expand All @@ -20,7 +21,7 @@ export interface BlueprintConstructProps {
export default class BlueprintConstruct {
constructor(scope: Construct, props: cdk.StackProps) {

HelmAddOn.validateHelmVersions = true;
//HelmAddOn.validateHelmVersions = true;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please fix the three github actions warning below like 'teams' is assigned a value but never used

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I revert this file back to what it was before since I was using it for testing purposes?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, please revert.


// TODO: fix IAM user provisioning for admin user
// Setup platform team.
Expand Down Expand Up @@ -132,9 +133,9 @@ export default class BlueprintConstruct {
});

blueprints.EksBlueprint.builder()
.addOns(...addOns)
.clusterProvider(clusterProvider)
.teams(...teams)
.addOns(new FlaggerAddOn())//...addOns)
//.clusterProvider(clusterProvider)
.teams()//...teams)
.enableControlPlaneLogTypes(blueprints.ControlPlaneLogType.API)
.build(scope, blueprintID, props);
}
Expand Down
21 changes: 21 additions & 0 deletions lib/addons/flagger/hpa.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
apiVersion: autoscaling/v2beta2
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this file for ? why is it in the source control?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was in the file example I got this from and figured an example api version in the file wouldn't hurt the test I was using it for. And I have not deleted those temporary yaml files yet, I can do that now.

kind: HorizontalPodAutoscaler
metadata:
name: podinfo
namespace: test
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: flagger-loadtester
minReplicas: 2
maxReplicas: 4
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
# scale up if usage is above
# 99% of the requested CPU (100m)
averageUtilization: 99
68 changes: 68 additions & 0 deletions lib/addons/flagger/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
import 'source-map-support/register';
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't need this import. Why was it added?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure might have been from a quick fix I did but it was not faded so I assumed it was doing something. I have deleted it and see it did not mess anything up.

import * as blueprints from '../../../lib';
import { Construct } from 'constructs';
import { Values } from "../../spi";
import merge from "ts-deepmerge";

/**
shapirov103 marked this conversation as resolved.
Show resolved Hide resolved
* User provided options for the FlaggerAddonProps values.
*/
export interface FlaggerAddOnProps extends blueprints.HelmAddOnUserProps {//this is the root level
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please remove the comment // this is the root level
it is evident

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was a personal note I forgot to remove, thanks for the catch.

prometheusInstall?: boolean;
Copy link
Collaborator

@shapirov103 shapirov103 Jul 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense.

meshProvider?: MeshProviderOptions;
}

/**
shapirov103 marked this conversation as resolved.
Show resolved Hide resolved
* All the meshProvider values that can be chosen by the user.
*/
export const enum MeshProviderOptions { //could use a better name later
KUBERNETES = 'kubernetes',
ISTIO = 'istio',
LINKERD = 'linkerd',
APPMESH = 'appmesh',
CONTOUR = 'contour',
NGINX = 'nginx',
GLOO = 'gloo',
SKIPPER = 'skipper',
TRAEFIK = 'traefik',
OSM = 'osm'
}

/**
shapirov103 marked this conversation as resolved.
Show resolved Hide resolved
* defaultProps makes the flagger namespace and chart.
*/
export const defaultProps: blueprints.HelmAddOnProps & FlaggerAddOnProps = {
name: "flagger",
namespace: "flagger",
chart: "flagger",
version: "1.22.0",
release: "flagger",
repository: "https://flagger.app"
};

/**
* This creates and deploys a cluster with the prometheus and mesh provider settings set unless the user specifies their own values for them.
*/
export class FlaggerAddOn extends blueprints.HelmAddOn {

readonly options: FlaggerAddOnProps;
Copy link
Collaborator

@shapirov103 shapirov103 Jul 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line break here.
HelmAddOn{Space}{
line breaks after class declaration, after member declaration, method. Inside methods if there are distinct paragraphs like {init block}, {call this API block}, {if/then/else} block, use line breaks to segregate.


constructor(props?: FlaggerAddOnProps) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line break here.

super({ ...defaultProps, ...props });
this.options = this.props as FlaggerAddOnProps;
}

deploy(clusterInfo: blueprints.ClusterInfo): Promise<Construct> {

let values: Values = {
prometheus: {
install: true
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are hardcoding the option that we allows customers to pass, ignoring their setting.

},
meshProvider: MeshProviderOptions.KUBERNETES
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't hardcode there values.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the way this worked was it set values to these, then if the user put in their own values values = merge(values, this.props.values ?? {}); would override the hardcoded values I put? I hardcoded them since they are the default if the user did not put in anything was the idea. Or should I not have anything by default?

};

values = merge(values, this.props.values ?? {});
const chart = this.addHelmChart(clusterInfo, values);
return Promise.resolve(chart);
}
}
78 changes: 78 additions & 0 deletions lib/addons/flagger/load-tester.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
apiVersion: apps/v1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this file

kind: Deployment
metadata:
name: flagger-loadtester
labels:
app: flagger-loadtester
namespace: test
spec:
replicas: 3
minReadySeconds: 3
revisionHistoryLimit: 5
progressDeadlineSeconds: 60
strategy:
rollingUpdate:
maxUnavailable: 0
type: RollingUpdate
selector:
matchLabels:
app: flagger-loadtester
template:
metadata:
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "9797"
labels:
app: flagger-loadtester
spec:
containers:
- name: flagger-loadtester
image: ghcr.io/stefanprodan/podinfo:6.1.6
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 9898
protocol: TCP
- name: http-metrics
containerPort: 9797
protocol: TCP
- name: grpc
containerPort: 9999
protocol: TCP
command:
- ./podinfo
- --port=9898
- --port-metrics=9797
- --grpc-port=9999
- --grpc-service-name=podinfo
- --level=info
- --random-delay=false
- --random-error=false
env:
- name: PODINFO_UI_COLOR
value: "#34577c"
livenessProbe:
exec:
command:
- podcli
- check
- http
- localhost:9898/healthz
initialDelaySeconds: 5
timeoutSeconds: 5
readinessProbe:
exec:
command:
- podcli
- check
- http
- localhost:9898/readyz
initialDelaySeconds: 5
timeoutSeconds: 5
resources:
limits:
cpu: 2000m
memory: 512Mi
requests:
cpu: 100m
memory: 64Mi
56 changes: 56 additions & 0 deletions lib/addons/flagger/podinfo-canary.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
apiVersion: flagger.app/v1beta1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this file

kind: Canary
metadata:
name: podinfo
namespace: test
spec:
# service mesh provider can be: kubernetes, istio, appmesh, nginx, gloo
provider: kubernetes
# deployment reference
targetRef:
apiVersion: apps/v1
kind: Deployment
name: flagger-loadtester
# the maximum time in seconds for the canary deployment
# to make progress before rollback (default 600s)
progressDeadlineSeconds: 60
service:
port: 9898
portDiscovery: true
analysis:
# schedule interval (default 60s)
interval: 30s
# max number of failed checks before rollback
threshold: 2
# number of checks to run before rollback
iterations: 10
# Prometheus checks based on
# http_request_duration_seconds histogram
metrics:
- name: request-success-rate
# minimum req success rate (non 5xx responses)
# percentage (0-100)
thresholdRange:
min: 99
interval: 1m
- name: request-duration
# maximum req duration P99
# milliseconds
thresholdRange:
max: 500
interval: 30s
# acceptance/load testing hooks
webhooks:
- name: smoke-test
type: pre-rollout
url: http://flagger-loadtester.test/
timeout: 15s
metadata:
type: bash
cmd: "curl -sd 'anon' http://podinfo-canary.test:9898/token | grep token"
- name: load-test
url: http://flagger-loadtester.test/
timeout: 5s
metadata:
type: cmd
cmd: "hey -z 1m -q 10 -c 2 http://podinfo-canary.test:9898/"
2 changes: 1 addition & 1 deletion lib/addons/index.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

export * from './appmesh';
export * from './argocd';
export * from './aws-for-fluent-bit';
Expand Down Expand Up @@ -31,6 +30,7 @@ export * from './ebs-csi-driver';
export * from './efs-csi-driver';
export * from './istio-base';
export * from './istio-control-plane';
//export * from './flagger';

export class Constants {
public static readonly BLUEPRINTS_ADDON = "blueprints-addon";
Expand Down
11 changes: 11 additions & 0 deletions lib/teams/application-team/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import { ArnPrincipal } from 'aws-cdk-lib/aws-iam';
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this file here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops when I was trying to figure stuff out for patterns and add a team for it I think I put this into the wrong program when I swapped between them.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is still in the code, why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is gone now, guess I got distracted by other issues and never got around to deleting it. I have done it now though.

import { ApplicationTeam } from '../../../lib/teams';

export class TeamApplication extends ApplicationTeam {
constructor(name: string, accountID: string) {
super({
name: name,
users: [new ArnPrincipal(`arn:aws:iam::${accountID}:user/application`)]
});
}
}
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,12 +38,12 @@
"dot-object": "^2.1.4",
"js-yaml": "4.1.0",
"lint": "^1.1.2",
"lodash":"4.17.21",
"lodash": "4.17.21",
"sync-request": "6.1.0",
"ts-deepmerge": "^2.0.4",
"ts-md5": "^1.2.11",
"tslog": "^3.3.3",
"yaml": "^2.1.1",
"zod": "^3.17.3"
}
}
}