Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First KEP version for review #1

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
266 changes: 266 additions & 0 deletions keps/sig-node/3702-ima-namespace-support/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,266 @@
# KEP-3702: IMA namespace support inside containers

- [KEP-XXXXX: IMA namespace support inside containers](#kep-xxxxx-ima-namespace-support-inside-containers)
- [Release Signoff Checklist](#release-signoff-checklist)
- [Summary](#summary)
- [Motivation](#motivation)
- [Goals](#goals)
- [Non-Goals](#non-goals)
- [Proposal](#proposal)
- [User Stories (Optional)](#user-stories-optional)
- [Story 1](#story-1)
- [Story 2](#story-2)
- [Story 3](#story-3)
- [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional)
- [Risks and Mitigations](#risks-and-mitigations)
- [Design Details](#design-details)
- [Linux kernel](#linux-kernel)
- [Runtime specification](#runtime-specification)
- [CRI API](#cri-api)
- [Kubernetes pod resource](#kubernetes-pod-resource)
- [Monitoring and alerting](#monitoring-and-alerting)
- [Test Plan](#test-plan)
- [Graduation Criteria](#graduation-criteria)
- [GA](#ga)
- [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy)
- [Version Skew Strategy](#version-skew-strategy)
- [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire)
- [Implementation History](#implementation-history)
- [Drawbacks](#drawbacks)
- [Alternatives](#alternatives)
- [Infrastructure Needed (Optional)](#infrastructure-needed-optional)

<!-- /toc -->

## Release Signoff Checklist

<!--
**ACTION REQUIRED:** In order to merge code into a release, there must be an
issue in [kubernetes/enhancements] referencing this KEP and targeting a release
milestone **before the [Enhancement Freeze](https://git.k8s.io/sig-release/releases)
of the targeted release**.

For enhancements that make changes to code or processes/procedures in core
Kubernetes—i.e., [kubernetes/kubernetes], we require the following Release

Signoff checklist to be completed.


Check these off as they are completed for the Release Team to track. These
checklist items _must_ be updated for the enhancement to be released.
-->

Items marked with (R) are required *prior to targeting to a milestone / release*.

<!--
**Note:** This checklist is iterative and should be reviewed and updated every time this enhancement is being considered for a milestone.
-->



[kubernetes.io]: https://kubernetes.io/

[kubernetes/enhancements]: https://git.k8s.io/enhancements

[kubernetes/kubernetes]: https://git.k8s.io/kubernetes

[kubernetes/website]: https://git.k8s.io/website


## Summary

IMA namespaces allow to check the file integrity. This proposal adds regular file integrity inside containers deployed in kubernetes.


## Motivation

File integrity is a way to improve security in systems allowing to:

* Detect illicit activity
* Detect unintended changes
* Verify the status and health of the system
* Comply with access rules

This can be achieved using IMA (Integrity Measurement Architecture) and EVM (Extended Verification Module), which can use a TPM chip as Hardware Root of Trust for high security environments.

### Goals

* Check integrity of regular files inside containers, and hence check POD integrity
* Using remote attestation mechanism, check the integrity of a given POD or all PODS in the cluster deployed with IMA namespace enabled periodically
* Alert about corrupted or compromised pods

### Non-Goals
.


## Proposal
<!-- We propose to enable IMA linux namespaces in pods.
Since IMA namespaces can be created when a container is launched, we can provide transparent integrity verification on any linux container.
IMA and EVM can use a TPM chip as a hardware root of trust. Hence we can verify images against a set of golden hash values, as well as avoiding any further changes to the overlayfs to intercept calls and check the integrity of files. -->

### User Stories (Optional)

#### Story 1
As a cluster admin, I want to detect undesired file changes, so that I can take out pods that have been compromised.

An intruder perform some malicious changes inside a certain pod's files. The system should be able to detect those changed and alert about the inconsistent pod. The remote attestation framework could keep this pod running or make a copy (for forensic analysis) and delete it.

#### Story 2
As a cluster admin, I want to deploy a only proven and certified pods, so that I can comply with internal policies as well as security regulations.

Let's say that we are working in a high security environment where only approved images can be deployed. In this scenario we can make sure that the pod deploy used an imaged that hasn't been tampered.

#### Story 3
As a cluster admin, I want to deny access to certain files inside the pod, so that a potential intruder can't access sensitive information.

In some cases, we should not even allow root to modify certain files inside the container.

### Notes/Constraints/Caveats (Optional)
We need to enable IMA in the kernel and container runtimes (runC, CRI-O, docker, containerd, etc.).

### Risks and Mitigations



## Design Details


<!--
This section should contain enough information that the specifics of your
change are understandable. This may include API specs (though not always
required) or even code snippets. If there's any ambiguity about HOW your
proposal will be implemented, this is the place to discuss them.
-->

In order for the this feature to work, the nodes where the pods will be deploy should have IMA enabled and a recent kernel that supports IMA namespaces (WIP, it should be merged soon). The nodes should also have a TPM chip. We could use nodAffinity of labels and annotations in nodes in order to select where to deploy the pods.

The linux kernel IMA namespace support is based on user namespaces. Therefore, the container runtime should first create a user namespace and then create an IMA namespaces. In order to use IMA namespaces it is necessary to enable user namespaces as well.

Should we enable IMA namespaces by default when enabling user namespaces?

There will be a CRI API change which will allow the pod to use IMA namespaces and specify the namespace policy.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This text doesn't seem consistent with the CRI change proposed (no way to select a policy). Is this something you plan to add later?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will make the change to move IMA to Pod.SecurityContext. One thing that I am concerned about is a mixed cloud scenario, with some hosts or guests running Windows or machines without hardware TPM. In that case, we want to make sure that the pods are deployed to the machines that have proper HW support. That's why I mentioned pinning the pods or using node affinity. These are mechanisms available in K8S. Should I expand this reasoning?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant the policy. Here it says we can specify the namespace policy in the CRI, but the change seems to be a bool, so... how can the policy be specified?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, my bad. That was a left over. I will remove any reference about policies.


### Linux kernel

IMA is only available in Linux hosts and Linux containers. Unfortunately, IMA is not a separate namespace, which is needed in order to isolate it and be used inside containers. Upcoming kernel patches should add support for IMA namespaces.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems confusing, you say it is not a separate namespace however the KEP talks all the time about the IMA namespace.

I guess you refer to the fact that won't be unshared? Not sure that is relevant here, but if you need to go into those details, please don't say it is not a namespace here and that it is a namespace in the rest of the KEP

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am referring to the fact that currently the Linux kernel doesn't support IMA namespaces. I guess the patches will be merged soon.

For now, IMA only works at host level, but newer kernels will have support for IMA namespaces. Regarding shared or unshared, I guess that IMA namespaces will be shared among the containers in the same pod, as it happens now with user namespaces.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently IMA namespace cannot "live" without any user namespace. IOW one user_ns -- one IMA namespace. Is there any initial user namespace in Kuber by defaut? @asierHuawei @rata

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, kubernetes 1.25 supports userns for stateless pods (this is alpha). I'll rework this to depend on idmap mounts for stateless pods soon (will need Linux >=6.3) and then add support for stateful pods

Copy link

@DenisSemakin DenisSemakin Jan 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rata Hm... sounds good, good news. From our side we'll try to implement and prepare all necessary low-level environment (e.g. patches for kernel) until Linux 6.3 is not released

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DenisSemakin cool. Do you need more patches to the kernel besides the ima ns patchset?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rata If we're talking about IMA namespaces as a standalone feature then the answer is... I guess it's "No". All patches provided by Stefan cover the basic functionality. But there's still opened question I see here: "will it be enough those patches for our purpose?"...
In case of remote attestation and pod (container) verification the answer is definitely "Yes". Here is a link to which patches we would like to add for this: https://lore.kernel.org/linux-integrity/[email protected]/



### Runtime specification

There is an ongoing discussion regarding the runtime changes.

https://github.com/opencontainers/runtime-spec/pull/1164


### CRI API

We propose to add the following message.



```protobuf

message LinuxSandboxSecurityContext {
NamespaceOption namespace_options = 1;
SELinuxOption selinux_options = 2;
Int64Value run_as_user = 3;
Int64Value run_as_group = 8;
bool readonly_rootfs = 4;
repeated int64 supplemental_groups = 5;
bool privileged = 6;
SecurityProfile seccomp = 9;
SecurityProfile apparmor = 10;
string seccomp_profile_path = 7 [deprecated=true];
// new field
bool ima = 11;
}

```
### Kubernetes pod resource
We propose the following change in the podTemplate to enable IMA namespaces.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pod.spec.

I think this and some others I've already mentioned in my previous review.


```yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
securityContext:
# New field

ima: true
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
```

The item pod.spec.ima.policy will automatically enable IMA for all the container in the pod with a given policy. Since all containers in a pod share the same namespaces, we need to have this policy in advance when creating the pod and its infrastructure container.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please specify (not just an example pod.spec) which fields are added and which type they will be



### Monitoring and alerting

This features will integrate with a future remote attestation procedure, which will monitor pods and in case of a violation take some actions like pod revocation, alerting, etc.

### Test Plan



Which unit tests should we include?



### Graduation Criteria


#### GA


### Upgrade / Downgrade Strategy


### Version Skew Strategy

## Production Readiness Review Questionnaire



Not applicable because this is a policy KEP.



## Implementation History

<!--
Major milestones in the lifecycle of a KEP should be tracked in this section.
Major milestones might include:
- the `Summary` and `Motivation` sections being merged, signaling SIG acceptance
- the `Proposal` section being merged, signaling agreement on a proposed design
- the date implementation started
- the first Kubernetes release where an initial version of the KEP was available
- the version of Kubernetes where the KEP graduated to general availability
- when the KEP was retired or superseded
-->

## Drawbacks

<!--
Why should this KEP _not_ be implemented?
-->

## Alternatives

<!--
What other approaches did you consider, and why did you rule them out? These do
not need to be as detailed as the proposal, but should include enough
information to express the idea and why it was not acceptable.
-->

## Infrastructure Needed (Optional)

<!--
Use this section if you need things from the project/SIG. Examples include a
new subproject, repos requested, or GitHub details. Listing these here allows a
SIG to get the process for these resources started right away.
-->
1 change: 1 addition & 0 deletions out
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSghO9XxHwLdURMMMGFJC/Ew24rbqHyejKOuTDXI4knRZwLY6ckl01Ixp39/3m4LEeuh7Qpl/HjvkwETyrUUNoEGKP8+I9xapOzRyCJaG2SpU1KMsb6FhdTOCCZeSfoA2fofxArhabrn4IXfKo8rS356DgZnDZjo46+cV5fbkZtzgAa4wiEOcCyxDMuEo7wkP6BITTgOuQgqitSNZRckRoUonxV9rVpQ+PdIyW3QRw+WLyYqahoNwIEJWQnhT/DyhzKjyxG7fVWJWiaQN4j42Ly9CdKhzJ8k68UZUVjm06kPNoGB1M1KsltVOF4C+o1JCXQKKYVWeEvcQafvocijgx root@mscphis01197
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems commited by mistake