Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(windows-cilium): windows cilium plugin design #844

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions docs/03-Metrics/plugins/Windows/ciliumeventobserver_windows.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# `ciliumEventObserverWindows`

Collect events from the Windows cilium eBPF event map and configure the event filter.

## DRAFT

This file documents the up-coming retina support for cilium on Windows.
The features described here aren't supported yet.

## Architecture

Windows cilium sends events to an eBPF ring buffer map at various points in the bpf programs.
The events supported include packet drop, trace, and policy verdicts. cilium trace capture events are also supported,
but currently only include the cilium event details (not the raw packet data).

To connect to these events you can use the `ciliumeventobserver_windows` retina plugin.
The Windows cilium plugin will attach to the `cilium_events` ring buffer and read the
the events, then decode and parse the events and wrap them in a flow object.
The flow object will then be sent to the external channel for processing.

### Processing Pipeline

1. Cilium bpf program writes events to ring buffer
2. Windows cilium retina agent plugin will:
1. collect the ring buffer events
2. Decode and parse the events into Hubble-compatible flow objects
3. Write the flow objects to the external channel
3. From the external channel retina will process the events into metrics.

### Cilium generated events

The Windows version of cilium writes observability and debugging events to an eBPF ring buffer map.
The event structures used in the Windows cilium eBPF programs have the same definitions as on linux,
so can use the same decoding logic. The main difference in the Windows eBPF program is the runtime [Event filtering](#event-filter-configuration), where instead of conditionally compiling in the observability reporting
The cilium programs check an observability configuration array map at runtime to decide whether to emit events.

### Collecting cilium events

The cilium events map on windows is a ring buffer map.
[cilium/ebpf-go](https://github.com/cilium/ebpf) does not yet run on Windows,
so we will add a small golang library to connect to the ebpf-for-windows ring buffer.

#### GoLang Shim for Windows eBPF ring buffer

Check failure on line 43 in docs/03-Metrics/plugins/Windows/ciliumeventobserver_windows.md

View workflow job for this annotation

GitHub Actions / markdownlint

Headings should be surrounded by blank lines [Expected: 1; Actual: 0; Below] [Context: "#### GoLang Shim for Windows eBPF ring buffer"]

docs/03-Metrics/plugins/Windows/ciliumeventobserver_windows.md:43 MD022/blanks-around-headings/blanks-around-headers Headings should be surrounded by blank lines [Expected: 1; Actual: 0; Below] [Context: "#### GoLang Shim for Windows eBPF ring buffer"]
- This will be a custom Golang library for consuming the eBPF ring buffer implementation on Windows.

Check failure on line 44 in docs/03-Metrics/plugins/Windows/ciliumeventobserver_windows.md

View workflow job for this annotation

GitHub Actions / markdownlint

Lists should be surrounded by blank lines [Context: "- This will be a custom Golang..."]

docs/03-Metrics/plugins/Windows/ciliumeventobserver_windows.md:44 MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- This will be a custom Golang..."]
- It will provide sufficient functionality to instantiate and consume events from a specified ring buffer in eBPF.
- The ciliumEventObserverWindows module willuse this shim to read the events in the ring buffer and perform further processing.
- This module provides a subset of the functionality of the classes ebpf.Map, ringbuf.Reader classes of cilium/ebpf module.
- This module will call the exported functions from ebpfApi.dll instead of relying on BPF abstraction.

### Event Parsing

The raw event parsing will be done using the existing [cilium/cilium/pkg/monitor](https://github.com/cilium/cilium/tree/main/pkg/monitor), since cilium on windows has the same binary event format.

### Event Processing

The retina daemon listens for the flow objects the plugin emits and sends them to our monitor agent.
Our hubble observer will consume these events and process the flows using our own custom
[parsers](https://github.com/microsoft/retina/tree/main/pkg/hubble/parser).

## Event filter configuration

On linux, cilium observability is configured by recompiling the bpf programs with different precompiler flags,
e.g. `DEBUG`/`TRACE_NOTIFY`.
On Windows, to support HVCI the observability is configured at runtime instead through an event configuration map
stored in an eBPF array `cilium_observability_config`.

To configure the event filter in retina, modify the appropriate values in the `retina-config`
[ConfigMap](../02-Installation/03-Config.md).
When retina reloads the configuration it will write the corresponding values to the event configuration map.

```yaml
cilium_windows:
# NOTE: currently DROP events are always enabled
enabled_events:
- DROP
- TRACE
- POLICY_VERDICT
# supported levels: none, low, medium, maximum
monitor_aggregation_level: maximum
```

### Event types

- DROP - emitted whenever a packet is dropped (these events are currently always enabled)
- TRACE - emitted whenever a packet is forwarded by one of the cilium ebpf programs
- POLICY_VERDICT - reports network policy decisions (allow/drop with reason for drops)
- DEBUG - events emitted by the cilium code for debugging

### Monitor aggregation

The monitor aggregation value has the same meaning as the linux cilium monitor aggregation level:

- **none**: no filtering - report all enabled events
- **low**: only report TX events (filter out RX events)
- **medium**/**maxmimum**: only report each connection once per aggregation interval for a given set of flags

## Code locations

- Plugin and eBPF code: _pkg/plugin/ciliumeventobserver_windows/_

## Metrics

The supported retina metics are TBD.
Loading