ftw.md

FTWTest

Welcome to the FTW YAMLFormat documentation. In this document we will explain all the possible options that can be used within the YAML format. Generally this is the preferred format for writing tests in as they don't require any programming skills in order to understand and change. If you find a bug in this format please open an issue.

FTWTest is the base type used when unmarshaling YAML tests files

---

meta FTWTestMeta

Meta describes the metadata information of this yaml test file

---

rule_id uint

RuleId is the ID of the rule this test targets.

Examples:

# RuleId
rule_id: 123456

---

tests []Test

Tests is a list of FTW tests

Examples:

tests:
    - test_title: 123456-1
      ruleid: 0
      test_id: 0
      desc: Unix RCE using `time`
      stages:
        - description: Get cookie from server
          input:
            dest_addr: 192.168.0.1
            port: 8080
            protocol: http
            uri: /test
            version: HTTP/1.1
            method: REPORT
            headers:
                Accept: '*/*'
                Host: localhost
                User-Agent: CRS Tests
            save_cookie: false
            stop_magic: true
            autocomplete_headers: false
            encoded_request: TXkgRGF0YQo=
          output:
            status: 200
            response_contains: HTTP/1.1
            log_contains: nothing
            no_log_contains: everything
            log:
                expect_ids:
                    - 123456
                no_expect_ids:
                    - 123456
                match_regex: id[:\s"]*123456
                no_match_regex: id[:\s"]*123456
            expect_error: true

---

FTWTestMeta

Appears in:

• FTWTest.meta

---

author string

Author is the list of authors that added content to this file

Examples:

# Author
author: Felipe Zipitria

---

enabled bool

Enabled indicates if the tests are enabled to be run by the engine or not.

Examples:

# Enabled
enabled: false

---

name string

Name is the name of the tests contained in this file.

Examples:

# Name
name: test01

---

description string

Description is a textual description of the tests contained in this file.

Examples:

# Description
description: The tests here target SQL injection.

---

version string

Version is the version of the YAML Schema.

Examples:

# Version
version: v1

---

tags []string

description: | Tags is list of strings that can be used for arbitrary grouping of tests. examples:

• name: Tags value: ["PHP", "bug-123"]

---

Test

Appears in:

• FTWTest.tests

- test_title: 123456-1
  ruleid: 0
  test_id: 0
  desc: Unix RCE using `time`
  stages:
    - description: Get cookie from server
      input:
        dest_addr: 192.168.0.1
        port: 8080
        protocol: http
        uri: /test
        version: HTTP/1.1
        method: REPORT
        headers:
            Accept: '*/*'
            Host: localhost
            User-Agent: CRS Tests
        save_cookie: false
        stop_magic: true
        autocomplete_headers: false
        encoded_request: TXkgRGF0YQo=
      output:
        status: 200
        response_contains: HTTP/1.1
        log_contains: nothing
        no_log_contains: everything
        log:
            expect_ids:
                - 123456
            no_expect_ids:
                - 123456
            match_regex: id[:\s"]*123456
            no_match_regex: id[:\s"]*123456
        expect_error: true

---

test_title string

TestTitle is the title of this particular types. It is used for inclusion/exclusion of each run by the tool.

Examples:

test_title: 123456-1

---

test_id uint

TestId is the ID of the test, in relation to rule_id. When this field is not set, the ID will be inferred from the position.

Examples:

# TestId
test_id: 4

---

desc string

TestDescription is the description for this particular test. Should be used to describe the internals of the specific things this test is targeting.

Examples:

desc: Unix RCE using `time`

---

stages []Stage

Stages is the list of all the stages to perform this test.

Examples:

stages:
    - description: Get cookie from server
      input:
        dest_addr: 192.168.0.1
        port: 8080
        protocol: http
        uri: /test
        version: HTTP/1.1
        method: REPORT
        headers:
            Accept: '*/*'
            Host: localhost
            User-Agent: CRS Tests
        save_cookie: false
        stop_magic: true
        autocomplete_headers: false
        encoded_request: TXkgRGF0YQo=
      output:
        status: 200
        response_contains: HTTP/1.1
        log_contains: nothing
        no_log_contains: everything
        log:
            expect_ids:
                - 123456
            no_expect_ids:
                - 123456
            match_regex: id[:\s"]*123456
            no_match_regex: id[:\s"]*123456
        expect_error: true

---

tags []string

description: | Tags is list of strings that can be used for arbitrary grouping of tests. examples:

• name: Tags value: ["PHP", "bug-123"]

---

Stage

Appears in:

• Test.stages

- description: Get cookie from server
  input:
    dest_addr: 192.168.0.1
    port: 8080
    protocol: http
    uri: /test
    version: HTTP/1.1
    method: REPORT
    headers:
        Accept: '*/*'
        Host: localhost
        User-Agent: CRS Tests
    save_cookie: false
    stop_magic: true
    autocomplete_headers: false
    encoded_request: TXkgRGF0YQo=
  output:
    status: 200
    response_contains: HTTP/1.1
    log_contains: nothing
    no_log_contains: everything
    log:
        expect_ids:
            - 123456
        no_expect_ids:
            - 123456
        match_regex: id[:\s"]*123456
        no_match_regex: id[:\s"]*123456
    expect_error: true

---

description string

Describes the purpose of this stage.

Examples:

description: Get cookie from server

---

input Input

Input is the data that is passed to the test

Examples:

# Input
input:
    dest_addr: 192.168.0.1
    port: 8080
    protocol: http
    uri: /test
    version: HTTP/1.1
    method: REPORT
    headers:
        Accept: '*/*'
        Host: localhost
        User-Agent: CRS Tests
    save_cookie: false
    stop_magic: true
    autocomplete_headers: false
    encoded_request: TXkgRGF0YQo=

---

output Output

Output is the data that is returned from the test

Examples:

# Output
output:
    status: 200
    response_contains: HTTP/1.1
    log_contains: nothing
    no_log_contains: everything
    log:
        expect_ids:
            - 123456
        no_expect_ids:
            - 123456
        match_regex: id[:\s"]*123456
        no_match_regex: id[:\s"]*123456
    expect_error: true

---

Input

Appears in:

• Stage.input

# Input
dest_addr: 192.168.0.1
port: 8080
protocol: http
uri: /test
version: HTTP/1.1
method: REPORT
headers:
    Accept: '*/*'
    Host: localhost
    User-Agent: CRS Tests
save_cookie: false
stop_magic: true
autocomplete_headers: false
encoded_request: TXkgRGF0YQo=

---

dest_addr string

DestAddr is the IP of the destination host that the test will send the message to.

Examples:

# DestAddr
dest_addr: 127.0.0.1

---

port int

Port allows you to declare which port on the destination host the test should connect to.

Examples:

# Port
port: 80

---

protocol string

Protocol allows you to declare which protocol the test should use when sending the request.

Examples:

# Protocol
protocol: http

---

uri string

URI allows you to declare the URI the test should use as part of the request line.

Examples:

# URI
uri: /get?hello=world

---

follow_redirect bool

FollowRedirect will expect the previous stage of the same test to have received a redirect response, it will fail the test otherwise. The redirect location will be used to send the request for the current stage and any settings for port, protocol, address, or URI will be ignored.

Examples:

# follow_redirect
follow_redirect: true

---

version string

Version allows you to declare the HTTP version the test should use as part of the request line.

Examples:

# Version
version: "1.1"

---

method string

Method allows you to declare the HTTP method the test should use as part of the request line.

Examples:

# Method
method: GET

---

headers map[string]string

Headers allows you to declare headers that the test should send.

Examples:

# Headers
headers:
    Accept: '*/*'
    Host: localhost
    User-Agent: CRS Tests

---

data string

Data allows you to declare the payload that the test should in the request body.

Examples:

# Data
data: Bibitti bopi

---

encoded_data string

EncodedData allows you to declare the payload as a base64 encoded string, which will be decoded into bytes and sent verbatimt to the server. This allows for complex payloads that include invisible characters or invalid Unicode byte sequences.

Examples:

# encoded_data
encoded_data: c29tZXRoaW5nIHdpdGgKbmV3bGluZQo=

---

save_cookie bool

SaveCookie allows you to automatically provide cookies if there are multiple stages and save cookie is set

Examples:

# SaveCookie
save_cookie: 80

---

stop_magic bool

StopMagic is deprecated.

Examples:

# StopMagic
stop_magic: false

---

autocomplete_headers bool

AutocompleteHeaders allows the test framework to automatically fill the request with Content-Type and Connection headers. Defaults to true.

Examples:

# StopMagic
autocomplete_headers: false

---

encoded_request string

EncodedRequest will take a base64 encoded string that will be decoded and sent through as the request. It will override all other settings

Examples:

# EncodedRequest
encoded_request: a

---

raw_request string

RAWRequest is deprecated.

Examples:

# RAWRequest
raw_request: TXkgRGF0YQo=

---

response Response

description: | Response describes a response from the web server that a WAF is expected to analyse. Note: This functionality requires a backend that can send the specified request to the reverse proxy. Currently, only Albedo (https://github.com/coreruleset/albedo) is supported. example:

• name Response value: ExampleResponse

---

Response

Appears in:

• Input.response

---

headers map[string]string

Headers defines the headers the response will carry.

Examples:

# Headers
headers:
    Accept: '*/*'
    Host: localhost
    User-Agent: CRS Tests

---

status int

Status describes the HTTP status code of the response. Defaults to 200 if omitted.

Examples:

# Status
status: 302

---

body string

Body defines the body of the response as a plain string.

Examples:

# Body
body: |
    {"aJsonDocument": ["in the response"]}

---

encoded_body string

EncodedBody defines the body of the response as a base64 encoded string. This is useful if the response needs to contain non-printable characters.

Examples:

# EncodedBody
encoded_body: eyJhSnNvbkRvY3VtZW50IjogWyJpbiB0aGUgcmVzcG9uc2UiXX0=

---

log_message string

LogMessage specifies a message to be printed in the log of the backend server that sends the response. This can be helpful when debugging, to match resopnses sent by the backend to test executions.

Examples:

# LogMessage
log_message: Response splitting test 1

---

Output

Appears in:

• Stage.output

# Output
status: 200
response_contains: HTTP/1.1
log_contains: nothing
no_log_contains: everything
log:
    expect_ids:
        - 123456
    no_expect_ids:
        - 123456
    match_regex: id[:\s"]*123456
    no_match_regex: id[:\s"]*123456
expect_error: true

---

status int

Status describes the HTTP status code expected in the response.

Examples:

# Status
status: 200

---

response_contains string

ResponseContains describes the text that should be contained in the HTTP response.

Examples:

# ResponseContains
response_contains: Hello, World

---

log_contains string

LogContains describes the text that should be contained in the WAF logs.

Examples:

# LogContains
log_contains: id 920100

---

no_log_contains string

NoLogContains describes the text that should not be contained in the WAF logs.

Examples:

# NoLogContains
no_log_contains: id 920100

---

log Log

Log is used to configure expectations about the log contents.

Examples:

log:
    expect_ids:
        - 123456
    no_expect_ids:
        - 123456
    match_regex: id[:\s"]*123456
    no_match_regex: id[:\s"]*123456

---

expect_error bool

When ExpectError is true, we don't expect an answer from the WAF, just an error.

Examples:

# ExpectError
expect_error: false

---

retry_once bool

When RetryOnce is true, the test run will be retried once upon failures. This options primary purpose is to work around a race condition in phase 5, where the log entry for a phase 5 rule may appear after the end marker of the previous test.

---

isolated bool

Isolated specifies that the test is expected to trigger a single rule only. If the rule triggers any other rule than the (single) one specified in expect_ids, the test fill be considered a failure. Default: false

Examples:

# Isolated
isolated: true

---

Log

Appears in:

• Output.log

expect_ids:
    - 123456
no_expect_ids:
    - 123456
match_regex: id[:\s"]*123456
no_match_regex: id[:\s"]*123456

---

expect_ids []uint

description: | Expect the given IDs to be contained in the log output. examples: -value: ExampleLog.ExpectIds

---

no_expect_ids []uint

Expect the given IDs not to be contained in the log output.

Examples:

no_expect_ids:
    - 123456

---

match_regex string

Expect the regular expression to match log content for the current types.

Examples:

match_regex: id[:\s"]*123456

---

no_match_regex string

Expect the regular expression to not match log content for the current types.

Examples:

no_match_regex: id[:\s"]*123456

---

FTWOverrides

FTWOverrides describes platform specific overrides for tests

---

version string

The version field designates the version of the schema that validates this file

Examples:

version: v0.1.0

---

meta FTWOverridesMeta

Meta describes the metadata information

Examples:

meta:
    engine: libmodsecurity3
    platform: nginx
    annotations:
        os: Debian Bullseye
        purpose: L7ASR test suite

---

test_overrides []TestOverride

List of test override specifications

Examples:

test_overrides:
    - rule_id: 920100
      test_ids: [4, 6]
      reason: |-
        nginx returns 400 when `Content-Length` header is sent in a
        `Transfer-Encoding: chunked` request.
      output:
        status: 200
        response_contains: HTTP/1.1
        log_contains: nothing
        no_log_contains: everything
        log:
            expect_ids:
                - 123456
            no_expect_ids:
                - 123456
            match_regex: id[:\s"]*123456
            no_match_regex: id[:\s"]*123456
        expect_error: true

---

FTWOverridesMeta

Appears in:

• FTWOverrides.meta

engine: libmodsecurity3
platform: nginx
annotations:
    os: Debian Bullseye
    purpose: L7ASR test suite

---

engine string

The name of the WAF engine the tests are expected to run against

Examples:

engine: coraza

---

platform string

The name of the platform (e.g., web server) the tests are expected to run against

Examples:

platform: nginx

---

annotations map[string]string

Custom annotations; can be used to add additional meta information

Examples:

annotations:
    os: Debian Bullseye
    purpose: L7ASR test suite

---

TestOverride

Appears in:

• FTWOverrides.test_overrides

- rule_id: 920100
  test_ids: [4, 6]
  reason: |-
    nginx returns 400 when `Content-Length` header is sent in a
    `Transfer-Encoding: chunked` request.
  output:
    status: 200
    response_contains: HTTP/1.1
    log_contains: nothing
    no_log_contains: everything
    log:
        expect_ids:
            - 123456
        no_expect_ids:
            - 123456
        match_regex: id[:\s"]*123456
        no_match_regex: id[:\s"]*123456
    expect_error: true

---

rule_id uint

ID of the rule this test targets.

Examples:

rule_id: 920100

---

test_ids []uint

IDs of the tests for rule_id that overrides should be applied to. If this field is not set, the overrides will be applied to all tests of rule_id.

Examples:

test_ids:
    - 4
    - 6

---

stage_ids []uint

IDs of the stages to which overrides should be applied. Stage IDs listed will be overridden for all test IDs listed in TestIds. If this field is not set, the overrides will be applied to all stages.

---

reason string

Describes why this override is necessary.

Examples:

reason: |-
    nginx returns 400 when `Content-Length` header is sent in a
    `Transfer-Encoding: chunked` request.

---

retry_once bool

Whether a stage should be retried once in case of failure. This option is primarily a workaround for a race condition in phase 5, where the log entry of a rule may be flushed after the test end marker.

Examples:

retry_once: true

---

output types.Output

Specifies overrides on the test output. This definition replaces the output definition of the test.

Examples:

output:
    status: 200
    response_contains: HTTP/1.1
    log_contains: nothing
    no_log_contains: everything
    log:
        expect_ids:
            - 123456
        no_expect_ids:
            - 123456
        match_regex: id[:\s"]*123456
        no_match_regex: id[:\s"]*123456
    expect_error: true

---

types.Output

Output defines the expectations of a test

Appears in:

• TestOverride.output

status: 200
response_contains: HTTP/1.1
log_contains: nothing
no_log_contains: everything
log:
    expect_ids:
        - 123456
    no_expect_ids:
        - 123456
    match_regex: id[:\s"]*123456
    no_match_regex: id[:\s"]*123456
expect_error: true

---

status int

Status describes the HTTP status code expected in the response.

Examples:

# Status
status: 200

---

response_contains string

ResponseContains describes the text that should be contained in the HTTP response.

Examples:

# ResponseContains
response_contains: Hello, World

---

log_contains string

LogContains describes the text that should be contained in the WAF logs.

Examples:

# LogContains
log_contains: id 920100

---

no_log_contains string

NoLogContains describes the text that should not be contained in the WAF logs.

Examples:

# NoLogContains
no_log_contains: id 920100

---

log Log

Log is used to configure expectations about the log contents.

Examples:

log:
    expect_ids:
        - 123456
    no_expect_ids:
        - 123456
    match_regex: id[:\s"]*123456
    no_match_regex: id[:\s"]*123456

---

expect_error types.bool

When ExpectError is true, we don't expect an answer from the WAF, just an error.

Examples:

# ExpectError
expect_error: false

---

retry_once types.bool

When RetryOnce is true, the test run will be retried once upon failures. This options primary purpose is to work around a race condition in phase 5, where the log entry for a phase 5 rule may appear after the end marker of the previous test.

---

isolated bool

Isolated specifies that the test is expected to trigger a single rule only. If the rule triggers any other rule than the (single) one specified in expect_ids, the test fill be considered a failure. Default: false

Examples:

# Isolated
isolated: true

---

types.Log

---

expect_ids []uint

description: | Expect the given IDs to be contained in the log output. examples: -value: ExampleLog.ExpectIds

---

no_expect_ids []uint

Expect the given IDs not to be contained in the log output.

Examples:

no_expect_ids:
    - 123456

---

match_regex string

Expect the regular expression to match log content for the current types.

Examples:

match_regex: id[:\s"]*123456

---

no_match_regex string

Expect the regular expression to not match log content for the current types.

Examples:

no_match_regex: id[:\s"]*123456

---