Build out-of-tree kernel modules with kmod kit

[arnaldo2792]

I'd like to compile and load out-of-tree kernel modules from orchestrated containers, using the kmod kit.

[arnaldo2792]

In order to build an out-of-tree kernel module for Bottlerocket, you need to use the kmod kit for the variant version you are using. This is an example of how to build the falco driver using a container (Note: this is an example script and should serve as reference):

FROM rust AS tuftool
RUN cargo install tuftool

FROM fedora:33 AS builder
WORKDIR /tmp
COPY --from=tuftool /usr/local/cargo/bin/tuftool /usr/local/bin/tuftool

# Install dependencies and download sources
RUN \
  ulimit -n 1024; dnf -y install \
  bc bzip2 cmake3 curl diffutils dwarves elfutils-devel \
  findutils gcc gcc-c++ git kmod make tar ncurses-devel \
  patch xz && \
  git clone https://github.com/falcosecurity/falco.git

# Download root.json to fetch artifacts from tuf repo
RUN curl -O "https://cache.bottlerocket.aws/root.json" && \
    echo "90393204232a1ad6b0a45528b1f7df1a3e37493b1e05b1c149f081849a292c8dafb4ea5f7ee17bcc664e35f66e37e4cfa4aae9de7a2a28aa31ae6ac3d9bea4d5  root.json" | sha512sum -c

FROM builder AS driver
ARG VARIANT="<VARIANT>"
ARG ARCH="<ARCH>"
ARG VERSION="<VERSION>"
ARG KIT="${VARIANT}-${ARCH}-kmod-kit-v${VERSION}"
ARG KERNELDIR="/tmp/${KIT}/kernel-devel"
ARG CROSS_COMPILE="${ARCH}-bottlerocket-linux-musl-"
ARG INSTALL_MOD_STRIP=1
RUN tuftool download . --root ./root.json \
      --target-name $KIT.tar.xz \
      --metadata-url "https://updates.bottlerocket.aws/2020-07-07/$VARIANT/$ARCH/" \
      --targets-url "https://updates.bottlerocket.aws/targets/"
RUN tar xf ${KIT}.tar.xz

RUN \
  export PATH="/tmp/${KIT}/toolchain/usr/bin:${PATH}" && \
  mkdir -p falco/build && \
  cd falco/build && \
  cmake3 -DUSE_BUNDLED_DEPS=ON .. && \
  make driver -j
# Validate the kernel module was compiled
RUN test -f /tmp/falco/build/driver/falco.ko

ADD ./load-driver /usr/bin/load-driver
ENTRYPOINT ["load-driver"]

Where load-driver is:

#! /bin/bash -x

insmod /tmp/falco/build/driver/falco.ko
sleep infinity &
trap "echo 'Caught signal'; { kill $!; exit 0; }" HUP INT QUIT PIPE TERM
trap - EXIT
while true; do wait $! || continue; done
exit 0

And k8s deployment configurations as:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: falco
spec:
  serviceName: falco
  replicas: 1
  selector:
    matchLabels:
      app: falco
  template:
    metadata:
      labels:
        app: falco
    spec:
      containers:
        - name: falco
          image: <image>
          securityContext:
            privileged: true

Once the container is running, you can see that the falco module was loaded:

[I] ~/P/b/k/falco> kubectl exec falco-0 -- lsmod | grep falco
falco                 647168  0

Keep in mind that in order to load a kernel module from a container, you need to make sure that kernel.lockdown is set to none in the API.