Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: restructure; update naming pattern and weight; fix inner-link #137

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,5 +61,5 @@ jobs:
id: lychee
uses: lycheeverse/lychee-action@f796c8b7d468feb9b8c0a46da3fac0af6874d374 # for v1.9.0
with:
args: "--cookie-jar /tmp/lychee-cookies './content/**/*.md'"
args: "--cookie-jar /tmp/lychee-cookies './content/**/*.md'"
jobSummary: true
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
title: Installing CRS
weight: 20
title: CRS Installation
weight: 11
disableToc: false
chapter: false
---

This guide aims to get a CRS installation up and running. This guide assumes that a compatible ModSecurity engine is already present and working. If unsure then refer to the [extended install]({{% ref "extended_install.md" %}}) page for full details.
This guide aims to get a CRS installation up and running. This guide assumes that a compatible ModSecurity engine is already present and working. If unsure then refer to the [extended install]({{< ref "1-2-extended_install.md" >}}) page for full details.

## Downloading the Rule Set

Expand Down Expand Up @@ -73,7 +73,7 @@ gpg: Good signature from "OWASP CRS <[email protected]>" [ultimate]

Once the rule set has been downloaded and verified, extract the rule set files to a well known location on the server. This will typically be somewhere in the web server directory.

The examples presented below demonstrate using Apache. For information on configuring Nginx or IIS see the [extended install]({{% ref "install.md" %}}) page.
The examples presented below demonstrate using Apache. For information on configuring Nginx or IIS see the [extended install]({{< ref "1-2-extended_install.md" >}}) page.

Note that while it's common practice to make a new `modsecurity.d` folder, as outlined below, this isn't strictly necessary. The path scheme outlined is common on RHEL-based operating systems; the Apache path used may need to be adjusted to match the server's installation.

Expand Down Expand Up @@ -113,7 +113,7 @@ mv crs-setup.conf.example crs-setup.conf

### Include-ing the Rule Files

The last step is to tell the web server where the rules are. This is achieved by `include`-ing the rule configuration files in the `httpd.conf` file. Again, this example demonstrates using Apache, but the process is similar on other systems (see the [extended install]({{% ref "install.md" %}}) page for details).
The last step is to tell the web server where the rules are. This is achieved by `include`-ing the rule configuration files in the `httpd.conf` file. Again, this example demonstrates using Apache, but the process is similar on other systems (see the [extended install]({{< ref "1-2-extended_install.md" >}}) page for details).

```bash
echo 'IncludeOptional {{< param crs_install_dir >}}/crs-setup.conf' >> /etc/httpd/conf/httpd.conf
Expand All @@ -123,15 +123,15 @@ echo 'IncludeOptional {{< param crs_install_dir >}}/rules/*.conf' >> /etc/httpd/
echo 'IncludeOptional {{< param crs_install_dir >}}/plugins/*-after.conf' >> /etc/httpd/conf/httpd.conf
```

Now that everything has been configured, it should be possible to restart and begin using the OWASP CRS. The CRS rules typically require a bit of tuning with rule exclusions, depending on the site and web applications in question. For more information on tuning, see [false positives and tuning]({{% ref "false_positives_tuning.md" %}}).
Now that everything has been configured, it should be possible to restart and begin using the OWASP CRS. The CRS rules typically require a bit of tuning with rule exclusions, depending on the site and web applications in question. For more information on tuning, see [false positives and tuning]({{< ref "2-how-crs-works/2-3-false-positives-and-tuning.md" >}}).

```bash
systemctl restart httpd.service
```

## Alternative: Using Containers

Another quick option is to use the official CRS [pre-packaged containers]({{% ref "../development/useful_tools/#official-crs-maintained-docker-images" %}}). Docker, Podman, or any compatible container engine can be used. The official CRS images are published on [Docker Hub](https://hub.docker.com/r/owasp/modsecurity-crs/) and [GitHub Container Repository](https://github.com/coreruleset/modsecurity-crs-docker/pkgs/container/modsecurity-crs). The image most often deployed is `modsecurity-crs` (`owasp/modsecurity-crs` from Docker Hub or `ghcr.io/coreruleset/modsecurity-crs` from GHCR): it already has everything needed to get up and running quickly.
Another quick option is to use the official CRS [pre-packaged containers]({{% ref "6-development/6-6-useful_tools/#official-crs-maintained-docker-images" %}}). Docker, Podman, or any compatible container engine can be used. The official CRS images are published on [Docker Hub](https://hub.docker.com/r/owasp/modsecurity-crs/) and [GitHub Container Repository](https://github.com/coreruleset/modsecurity-crs-docker/pkgs/container/modsecurity-crs). The image most often deployed is `modsecurity-crs` (`owasp/modsecurity-crs` from Docker Hub or `ghcr.io/coreruleset/modsecurity-crs` from GHCR): it already has everything needed to get up and running quickly.

The CRS project pre-packages both Apache and Nginx web servers along with the appropriate corresponding ModSecurity engine. More engines, like [Coraza](https://coraza.io/), will be added at a later date.

Expand Down Expand Up @@ -183,7 +183,7 @@ Depending on your configurated thresholds, this should be detected as a maliciou

### Upgrading from CRS 3.x to CRS 4

The most impactful change is the removal of application exclusion packages in favor of a plugin system. If you had activated the exclusion packages in CRS 3, you should download the plugins for them and place them in the plugins subdirectory. We maintain the list of plugins in our [Plugin Registry](https://github.com/coreruleset/plugin-registry). You can find detailed information on working with plugins in our [plugins documentation]({{% ref "plugins.md" %}}).
The most impactful change is the removal of application exclusion packages in favor of a plugin system. If you had activated the exclusion packages in CRS 3, you should download the plugins for them and place them in the plugins subdirectory. We maintain the list of plugins in our [Plugin Registry](https://github.com/coreruleset/plugin-registry). You can find detailed information on working with plugins in our [plugins documentation]({{< ref "4-about-plugins/" >}}).

In terms of changes to the detection rules, the amount of changes is smaller than in the CRS 2—3 changeover. Most rules have only evolved slightly, so it is recommended that you keep any existing custom exclusions that you have made under CRS 3.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
title: Extended Install
weight: 30
weight: 12
disableToc: false
chapter: false
---

> All the information needed to properly install CRS is presented on this page. The installation concepts are expanded upon and presented in more detail than the [quick start guide]({{% ref "install.md" %}}).
> All the information needed to properly install CRS is presented on this page. The installation concepts are expanded upon and presented in more detail than the [quick start guide]({{< ref "1-1-crs-installation.md" >}}).

## Contact Us

Expand Down Expand Up @@ -176,7 +176,7 @@ At a minimum, keep in mind the following:

- CRS does not configure features such as the rule engine, audit engine, logging, etc. This task is part of the initial *engine* setup and is not a job for the rule set. For ModSecurity, if not already done, see the [recommended configuration](https://github.com/owasp-modsecurity/ModSecurity/blob/master/modsecurity.conf-recommended).
- Decide what ModSecurity should do when it detects malicious activity, e.g., drop the packet, return a *403 Forbidden* status code, issue a redirect to a custom page, etc.
- Make sure to configure the anomaly scoring thresholds. For more information see [Anomaly]({{% ref "anomaly_scoring.md" %}} "Anomaly").
- Make sure to configure the anomaly scoring thresholds. For more information see [Anomaly]({{< ref "2-how-crs-works/2-1-anomaly_scoring.md" >}} "Anomaly").
- By default, the CRS rules will consider many issues with different databases and languages. If running in a specific environment, e.g., without any SQL database services present, it is probably a good idea to limit this behavior for performance reasons.
- Make sure to add any HTTP methods, static resources, content types, or file extensions that are needed, beyond the default ones listed.

Expand All @@ -191,7 +191,7 @@ In addition to `crs-setup.conf.example`, there are two other ".example" files wi
- `rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf.example`
- `rules/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf.example`

These files are designed to provide the rule maintainer with the ability to modify rules (see [false positives and tuning]({{% ref "#rule-exclusions" %}})) without breaking forward compatibility with rule set updates. These two files should be renamed by removing the `.example` suffix. This will mean that installing updates will *not* overwrite custom rule exclusions. To rename the files in Linux, use a command similar to the following:
These files are designed to provide the rule maintainer with the ability to modify rules (see [false positives and tuning]({{< ref "2-how-crs-works/2-3-false-positives-and-tuning.md" >}}#rule-exclusions)) without breaking forward compatibility with rule set updates. These two files should be renamed by removing the `.example` suffix. This will mean that installing updates will *not* overwrite custom rule exclusions. To rename the files in Linux, use a command similar to the following:

```bash
mv rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf.example rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf
Expand All @@ -202,7 +202,7 @@ mv rules/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf.example rules/RESPONSE-999-

The engine should support the `Include` directive out of the box. This directive tells the engine to parse *additional* files for directives. The question is where to put the CRS rules folder in order for it to be included.

Looking at the CRS files, there are quite a few ".conf" files. While the names attempt to do a good job at describing what each file does, additional information is available in the [rules]({{% ref "rules" %}}) section.
Looking at the CRS files, there are quite a few ".conf" files. While the names attempt to do a good job at describing what each file does, additional information is available in the [rules]({{< ref "3-about-rules/rules.md" >}}) section.

### Includes for Apache

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Using Containers
weight: 20
weight: 13
disableToc: false
chapter: false
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Engine and Integration Options
weight: 10
weight: 14
disableToc: false
chapter: false
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Getting CRS
weight: 10
title: Getting Started
weight: 1
pre: "<b>1. </b>"
chapter: true
---
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Anomaly Scoring
weight: 10
weight: 21
disableToc: false
chapter: false
---
Expand Down Expand Up @@ -63,9 +63,9 @@ SecRule REQUEST_HEADERS:Content-Length "!@rx ^\d+$" \
```

{{% notice info %}}
Notice that the anomaly score variable name has the suffix `pl1`. Internally, CRS keeps track of anomaly scores on a *per* [*paranoia level*]({{< ref "paranoia_levels" >}} "Page describing paranoia levels.") basis. The individual paranoia level anomaly scores are added together before each round of blocking evaluation takes place, allowing the total combined inbound or outbound score to be compared to the relevant anomaly score threshold.
Notice that the anomaly score variable name has the suffix `pl1`. Internally, CRS keeps track of anomaly scores on a *per* [*paranoia level*]({{< ref "2-2-paranoia_levels" >}} "Page describing paranoia levels.") basis. The individual paranoia level anomaly scores are added together before each round of blocking evaluation takes place, allowing the total combined inbound or outbound score to be compared to the relevant anomaly score threshold.

Tracking the anomaly score per paranoia level allows for clever scoring mechanisms to be employed, such as the [executing paranoia level]({{< ref "paranoia_levels#moving-to-a-higher-paranoia-level" >}} "Section describing the executing paranoia level feature.") feature.
Tracking the anomaly score per paranoia level allows for clever scoring mechanisms to be employed, such as the [executing paranoia level]({{< ref "2-2-paranoia_levels#moving-to-a-higher-paranoia-level" >}} "Section describing the executing paranoia level feature.") feature.
{{% /notice %}}

The rules files `REQUEST-949-BLOCKING-EVALUATION.conf` and `RESPONSE-959-BLOCKING-EVALUATION.conf` are responsible for executing the inbound (request) and outbound (response) rounds of blocking evaluation, respectively. The rules in these files calculate the total inbound or outbound transactional anomaly score and then make a blocking decision, by comparing the result to the defined threshold and taking blocking action if required.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Paranoia Levels
weight: 20
weight: 22
disableToc: false
chapter: false
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: False Positives and Tuning
weight: 30
weight: 23
disableToc: false
chapter: false
---
Expand All @@ -9,25 +9,9 @@ chapter: false

## What are False Positives?

CRS provides _generic_ attack detection capabilities. A fresh CRS deployment has no awareness of the web services that may be running behind it, or the quirks of how those services work. The following image describes the possible scenarios for an individual request:

```mermaid
%%{init: {"themeVariables": {"quadrant1Fill": "#00ff00", "quadrant2Fill": "#ff0000", "quadrant4Fill": "#ffff00"} }}%%
quadrantChart
title WAF detection quadrant
x-axis Allowed --> Blocked
y-axis Normal Traffic --> Real attack
quadrant-1 True Positive
quadrant-2 False Negative
quadrant-3 Business as usual
quadrant-4 False Positive
```

In an ideal setting, CRS would _allow all valid requests_ and _block all real attacks_. These two scenarios are depicted in the image as `Business as usual` and `True positive`.

In reality, no system is ideal and CRS will sometimes _block a valid request_ (`False Positive`) or _allow a real attack_ (`False Negative`). Obviously, false negatives are the worse of the two because of the potential repercussions of a successful intrusion.
CRS provides _generic_ attack detection capabilities. A fresh CRS deployment has no awareness of the web services that may be running behind it, or the quirks of how those services work. It is possible that *genuine* transactions may cause some CRS rules to match in error, if the transactions happen to match one of the generic attack behaviors or patterns that are being detected. Such a match is referred to as a *false positive*, or false alarm.

False positives are particularly likely to happen when operating at higher [paranoia levels]({{% ref "paranoia_levels" %}} "Page describing paranoia levels."). While paranoia level 1 is designed to cause few false positives, higher paranoia levels are increasingly likely to cause false positives. Each successive paranoia level introduces additional rules, with *higher* paranoia levels adding *more aggressive* rules. As such, the higher the paranoia level is, the more likely it is that false positives will occur, and the more time must be invested to tune CRS to reduce false positives. There is no value to a system with a high paranoia level that produces many false positives, as it will likely be unusable by benign clients.
False positives are particularly likely to happen when operating at higher [paranoia levels]({{< ref "2-2-paranoia_levels.md" >}} "Page describing paranoia levels."). While paranoia level 1 is designed to cause few, ideally zero, false positives, higher paranoia levels are increasingly likely to cause false positives. Each successive paranoia level introduces additional rules, with *higher* paranoia levels adding *more aggressive* rules. As such, the higher the paranoia level is the more likely it is that false positives will occur. That is the cost of the higher security provided by higher paranoia levels: the additional time it takes to tune away the increasing number of false positives.

### Example False Positive

Expand Down Expand Up @@ -97,7 +81,7 @@ There are alternative ways to deal with false positives, as described below. The

#### Overview

The WAF engine has flexible ways to tune away false positives. It provides several *rule exclusion* (RE) mechanisms which allow rules to be modified *without* directly changing the rules themselves. This makes it possible to work with third-party rule sets, like CRS, by adapting rules as needed while leaving the rule set files intact and unmodified. This allows for easy rule set updates.
The ModSecurity WAF engine has flexible ways to tune away false positives. It provides several *rule exclusion* (RE) mechanisms which allow rules to be modified *without* directly changing the rules themselves. This makes it possible to work with third-party rule sets, like CRS, by adapting rules as needed while leaving the rule set files intact and unmodified. This allows for easy rule set updates.

Two fundamentally different types of rule exclusions are supported:

Expand Down Expand Up @@ -353,11 +337,28 @@ CRS ships with prebuilt *rule exclusion packages* for a selection of popular web

The packages should be viewed as a good *starting point* from which to build upon. Some false positives may still occur, for example if working at a high paranoia level, if using a very new or old version of the application, if using plug-ins, add-ons, or user customizations.

If using a native CRS installation, rule exclusion packages can be downloaded as [plugins]({{% ref "plugins.md" %}}).
If using a native CRS installation, rule exclusion packages can be enabled in the file `crs-setup.conf`. Modify rule 900130 to select the web applications in question, e.g. to enable the DokuWiki rule exclusion package use `setvar:tx.crs_exclusions_dokuwiki=1`, and then uncomment the rule to enable it.

If running CRS where it has been integrated into a commercial product or CDN then support varies. Some vendors expose rule exclusion packages in the GUI while other vendors require custom rules to be written which set the necessary variables. Unfortunately, there are also vendors that don't allow rule exclusion packages to be used at all.

The list of plugins supporting rule exclusions can be found [here](https://github.com/coreruleset/plugin-registry). The names of rule exclusion plugins end with `rule-exclusions`.
{{% notice tip %}}
If running multiple web applications, it is highly recommended to enable a rule exclusion package only for the location where the corresponding web application resides. For example, to enable the WordPress rule exclusion package only for locations under '/wordpress', a rule like the following could be used:

```apache
SecRule REQUEST_URI "@beginsWith /wordpress/" setvar:tx.crs_exclusions_wordpress=1...
```
{{% /notice %}}

Rule exclusion packages are currently available for the following web applications:

- [cPanel](https://cpanel.net)
- [DokuWiki](https://www.dokuwiki.org)
- [Drupal](https://www.drupal.org)
- [Nextcloud](https://nextcloud.com)
- [phpBB](https://www.phpbb.com)
- [phpMyAdmin](https://www.phpmyadmin.net)
- [WordPress](https://wordpress.org)
- [XenForo](https://xenforo.com)

The CRS project is always looking to work with other communities and individuals to add support for additional web applications. Please get in touch via [GitHub](https://github.com/coreruleset/coreruleset) to discuss writing a rule exclusion package for a specific web application.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Sampling Mode
weight: 50
weight: 24
disableToc: false
chapter: false
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: How CRS Works
weight: 20
weight: 2
pre: "<b>2. </b>"
chapter: true
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
---
title: Operation
weight: 30
title: About Rules
weight: 3
pre: "<b>3. </b>"
chapter: true
---

# Rules
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Making
weight: 30
title: Making Rules
weight: 31
disableToc: false
chapter: false
---
Expand Down
File renamed without changes.
Loading
Loading