First KEP version for review #1

asierHuawei · 2022-12-12T14:52:32Z

One-line PR description:

Issue link:

Other comments:

rata

@asierHuawei Thanks! Left some comments, let me know what you think :)

rata · 2022-12-28T17:08:46Z

keps/sig-node/asier/README.md

+To get started with this template:
+
+
+


Spacing seems broken (compared to the kep template). It is not a problem now, but something to fix before opening the upstream PR

rata · 2022-12-28T17:09:41Z

keps/sig-node/asier/README.md

+
+
+
+IMA namespaces allow to check the file integrity. This proposal adds file integrity inside containers deployed in kubernetes.


"the file" --> which file? The rootfs?

This is first of all about regular files I suppose @asierHuawei

Yes, regular files inside the container. I will change the wording to be more explicit.

rata · 2022-12-28T17:12:41Z

keps/sig-node/asier/README.md

+
+
+
+* Allow IMA to work inside a container with remote attestation


For an initial stage this can be fine. But more concrete goals derived by what allowing IMA namespace inside containers enables will probably be needed at some point

I would say that IMA in container allows to control the POD integrity and using the mechanism of remote attestation one can definitely check the integrity at any given POD and any time.
@asierHuawei What you think?

@DenisSemakin That's right, I will make it more explicit.

rata · 2022-12-28T17:13:56Z

keps/sig-node/asier/README.md

+
+
+
+### Non-Goals


Non-goals are usually used to put out of scope some things reviewers might wonder. It helps to clarify the scope. It is okay now to not have any, but I guess at some point we will add something here too (at least after conversations when people assume this will tackle X).

Maybe something to leave out of scope is IMA support for volumes and just the rootfs is used now?

I will leave it empty for now. As I get more feedback from the community, I can start filling it in.

rata · 2022-12-28T17:18:33Z

keps/sig-node/asier/README.md

+
+### Runtime specification
+
+In order to run a container we need a bundle that contains a config.json file with all the configuration. There is a Linux specific configuration section where the namespaces are listed. According to the standard, the following namespace types SHOULD be supported.


You shouldn't explain the runtime-spec change here. At most mention that a PR is merged/open for this, but not go into the details here in the KEP. Lot of people will comment things that will be lost with the upstream discussion for runtime-spec.

Let's keep the runtime-spec discussion in the runtime-spec PR and remove most of this from here.

Thanks, I will link to a PR in runC that shows how it can be achieved.

rata · 2022-12-28T17:22:57Z

keps/sig-node/asier/README.md

+
+
+
+This KEP is a policy KEP, not a feature KEP. It will start as GA.


Where does this reasoning come from? I don't think this is ok

This was a left over from the template. I removed it, thanks for noticing.

rata · 2022-12-28T17:25:22Z

keps/sig-node/asier/README.md

+#### Story 3
+
+
+As a cluster admin, I want to deny access to certain files inside the pod, so that a potential intruder can't access sensitive information.


What would be a way to deny access used by the IMA namespace? Why is this better than using the permission bits to deny access?

I will add more specific info. Thanks for the feedback.

rata · 2022-12-28T17:26:34Z

keps/sig-node/asier/README.md

+#### Story 2
+
+
+As a cluster admin, I want to deploy a only proven and certified pods, so that I can comply with internal policies as well as security regulations.


It doesn't seem the current proposal has any way to only allow certified pods. Can you comment on where this would be expanded to support this?

I will change the wording.

rata · 2022-12-28T17:28:27Z

keps/sig-node/asier/README.md

+
+#### Story 1
+
+As a cluster admin, I want to detect undesired file changes, so that I can take out pods that have been compromised.


Can you comment at a high-level how the undesired file change can bubble-up so anyone knows and takes out pods? Or will this be automatic?

rata · 2022-12-28T17:39:13Z

keps/sig-node/asier/README.md

+IMA and EVM can use a TPM chip as a hardware root of trust. Hence we can verify images against a set of golden hash values, as well as avoiding any further changes to the overlayfs to intercept calls and check the integrity of files.
+
+
+### User Stories (Optional)


The current user-story seem more aspirational for me (maybe I'm missing something), that is why I posted some questions to them (to see if they are really aspirational or do we have a high-level idea on how that can be accomplished). If we don't really know how that can be accomplished and they seem mostly aspirational, let's just remove them for now.

What actual things do you expect to be possible? I think this section should be that, not something aspirational. Something this will allow.

It is usually helpful to have this section completed early, but if you still don't know, leave it empty.

Maybe things that can be done is something like this? No idea, though:

As a cluster admin, I want to guarantee the rootfs files are unchanged and verified when a container access it, so I can have a high guarantee that files were not tampered.

Maybe more can be created after the measure mode, some others after the appraisal mode too (if we plan to support it)?

But let's keep this section concrete and real, or not fill them yet if we need to play a little more with the feature to understand what will be possible.

Thanks for the tip. I will leave it empty for now.

rata · 2022-12-29T15:20:14Z

keps/sig-node/3702-ima-namespace-support/README.md

+
+There is an ongoing discussion regarding the runtime changes.
+
+https://github.com/opencontainers/runc/pull/3639


That is runc, an implementation of the specification. The spec PR seems to be this one: opencontainers/runtime-spec#1164

rata · 2022-12-29T15:23:15Z

keps/sig-node/3702-ima-namespace-support/README.md

+
+Since IMA namespaces can be created when a container is launched, we can provide transparent integrity verification on any linux container.
+
+IMA and EVM can use a TPM chip as a hardware root of trust. Hence we can verify images against a set of golden hash values, as well as avoiding any further changes to the overlayfs to intercept calls and check the integrity of files. -->


This was moved inside a comment, so not visible

Following your suggestion, I decided to remove it. Maybe I should remove it completely instead of commenting it out?

rata · 2022-12-29T15:24:47Z

keps/sig-node/3702-ima-namespace-support/README.md

+Should we enable IMA namespaces by default when enabling user namespaces?
+
+
+There will be a CRI API change which will allow the pod to use IMA namespaces and specify the namespace policy.


This text doesn't seem consistent with the CRI change proposed (no way to select a policy). Is this something you plan to add later?

I will make the change to move IMA to Pod.SecurityContext. One thing that I am concerned about is a mixed cloud scenario, with some hosts or guests running Windows or machines without hardware TPM. In that case, we want to make sure that the pods are deployed to the machines that have proper HW support. That's why I mentioned pinning the pods or using node affinity. These are mechanisms available in K8S. Should I expand this reasoning?

I meant the policy. Here it says we can specify the namespace policy in the CRI, but the change seems to be a bool, so... how can the policy be specified?

Yes, my bad. That was a left over. I will remove any reference about policies.

rata · 2022-12-29T15:26:52Z

keps/sig-node/3702-ima-namespace-support/README.md

+
+
+
+IMA is only available in Linux hosts and Linux containers. Unfortunately, IMA is not a separate namespace, which is needed in order to isolate it and be used inside containers. Upcoming kernel patches should add support for IMA namespaces.


This seems confusing, you say it is not a separate namespace however the KEP talks all the time about the IMA namespace.

I guess you refer to the fact that won't be unshared? Not sure that is relevant here, but if you need to go into those details, please don't say it is not a namespace here and that it is a namespace in the rest of the KEP

I am referring to the fact that currently the Linux kernel doesn't support IMA namespaces. I guess the patches will be merged soon.

For now, IMA only works at host level, but newer kernels will have support for IMA namespaces. Regarding shared or unshared, I guess that IMA namespaces will be shared among the containers in the same pod, as it happens now with user namespaces.

Currently IMA namespace cannot "live" without any user namespace. IOW one user_ns -- one IMA namespace. Is there any initial user namespace in Kuber by defaut? @asierHuawei @rata

Yes, kubernetes 1.25 supports userns for stateless pods (this is alpha). I'll rework this to depend on idmap mounts for stateless pods soon (will need Linux >=6.3) and then add support for stateful pods

@rata Hm... sounds good, good news. From our side we'll try to implement and prepare all necessary low-level environment (e.g. patches for kernel) until Linux 6.3 is not released

@DenisSemakin cool. Do you need more patches to the kernel besides the ima ns patchset?

@rata If we're talking about IMA namespaces as a standalone feature then the answer is... I guess it's "No". All patches provided by Stefan cover the basic functionality. But there's still opened question I see here: "will it be enough those patches for our purpose?"...
In case of remote attestation and pod (container) verification the answer is definitely "Yes". Here is a link to which patches we would like to add for this: https://lore.kernel.org/linux-integrity/[email protected]/

rata · 2022-12-29T15:27:29Z

keps/sig-node/3702-ima-namespace-support/README.md

+
+```
+### Kubernetes pod resource
+We propose the following change in the podTemplate to enable IMA namespaces.


pod.spec.

I think this and some others I've already mentioned in my previous review.

rata · 2022-12-29T15:28:31Z

keps/sig-node/3702-ima-namespace-support/README.md

+ - containerPort: 80
+```
+
+The item pod.spec.ima.policy will automatically enable IMA for all the container in the pod with a given policy. Since all containers in a pod share the same namespaces, we need to have this policy in advance when creating the pod and its infrastructure container.


Please specify (not just an example pod.spec) which fields are added and which type they will be

rata · 2023-01-11T16:34:44Z

out

@@ -0,0 +1 @@
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSghO9XxHwLdURMMMGFJC/Ew24rbqHyejKOuTDXI4knRZwLY6ckl01Ixp39/3m4LEeuh7Qpl/HjvkwETyrUUNoEGKP8+I9xapOzRyCJaG2SpU1KMsb6FhdTOCCZeSfoA2fofxArhabrn4IXfKo8rS356DgZnDZjo46+cV5fbkZtzgAa4wiEOcCyxDMuEo7wkP6BITTgOuQgqitSNZRckRoUonxV9rVpQ+PdIyW3QRw+WLyYqahoNwIEJWQnhT/DyhzKjyxG7fVWJWiaQN4j42Ly9CdKhzJ8k68UZUVjm06kPNoGB1M1KsltVOF4C+o1JCXQKKYVWeEvcQafvocijgx root@mscphis01197


This seems commited by mistake

asierHuawei added 2 commits December 22, 2022 21:35

First KEP version for review

cf433b2

Initial draft for IMA namespace support in kubernetes

b1bffa2

rata reviewed Dec 28, 2022

View reviewed changes

Changed the wording in a number of points according to feedback

e81003a

asierHuawei force-pushed the asier branch from 4642125 to e81003a Compare December 29, 2022 14:45

rata reviewed Dec 29, 2022

View reviewed changes

Incorporated feedback from @rata

b05513b

asierHuawei force-pushed the asier branch from 4add0ed to b05513b Compare January 11, 2023 13:52

rata reviewed Jan 11, 2023

View reviewed changes

Removed old left overs about IMA policies

c5eb082

asierHuawei force-pushed the asier branch from f861509 to c5eb082 Compare January 12, 2023 09:57

Delete out

fbc5b76




		IMA namespaces allow to check the file integrity. This proposal adds file integrity inside containers deployed in kubernetes.




		* Allow IMA to work inside a container with remote attestation


		### Runtime specification

		In order to run a container we need a bundle that contains a config.json file with all the configuration. There is a Linux specific configuration section where the namespaces are listed. According to the standard, the following namespace types SHOULD be supported.




		This KEP is a policy KEP, not a feature KEP. It will start as GA.

		#### Story 3


		As a cluster admin, I want to deny access to certain files inside the pod, so that a potential intruder can't access sensitive information.

		#### Story 2


		As a cluster admin, I want to deploy a only proven and certified pods, so that I can comply with internal policies as well as security regulations.


		#### Story 1

		As a cluster admin, I want to detect undesired file changes, so that I can take out pods that have been compromised.

		IMA and EVM can use a TPM chip as a hardware root of trust. Hence we can verify images against a set of golden hash values, as well as avoiding any further changes to the overlayfs to intercept calls and check the integrity of files.


		### User Stories (Optional)


		There is an ongoing discussion regarding the runtime changes.

		https://github.com/opencontainers/runc/pull/3639


		Since IMA namespaces can be created when a container is launched, we can provide transparent integrity verification on any linux container.

		IMA and EVM can use a TPM chip as a hardware root of trust. Hence we can verify images against a set of golden hash values, as well as avoiding any further changes to the overlayfs to intercept calls and check the integrity of files. -->

		Should we enable IMA namespaces by default when enabling user namespaces?


		There will be a CRI API change which will allow the pod to use IMA namespaces and specify the namespace policy.




		IMA is only available in Linux hosts and Linux containers. Unfortunately, IMA is not a separate namespace, which is needed in order to isolate it and be used inside containers. Upcoming kernel patches should add support for IMA namespaces.

		@@ -0,0 +1 @@
		ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSghO9XxHwLdURMMMGFJC/Ew24rbqHyejKOuTDXI4knRZwLY6ckl01Ixp39/3m4LEeuh7Qpl/HjvkwETyrUUNoEGKP8+I9xapOzRyCJaG2SpU1KMsb6FhdTOCCZeSfoA2fofxArhabrn4IXfKo8rS356DgZnDZjo46+cV5fbkZtzgAa4wiEOcCyxDMuEo7wkP6BITTgOuQgqitSNZRckRoUonxV9rVpQ+PdIyW3QRw+WLyYqahoNwIEJWQnhT/DyhzKjyxG7fVWJWiaQN4j42Ly9CdKhzJ8k68UZUVjm06kPNoGB1M1KsltVOF4C+o1JCXQKKYVWeEvcQafvocijgx root@mscphis01197

First KEP version for review #1

Are you sure you want to change the base?

First KEP version for review #1

Conversation

asierHuawei commented Dec 12, 2022

rata left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DenisSemakin Jan 12, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DenisSemakin Jan 12, 2023 •

edited

Loading