Skip to content

Commit 1be9f53

Browse files
enhance: allowlist sweep and readability of FAQ (#1030)
1 parent e747827 commit 1be9f53

32 files changed

+307
-294
lines changed

crowdsec-docs/docs/appsec/troubleshooting.md

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,8 @@ sidebar_position: 81
88
## Monitoring with `cscli`
99

1010
`cscli metrics` exposes basic metrics about the AppSec Component:
11-
- Number of requests processed and blocked by the component/data source
12-
- Number of triggers for each rule
11+
- Number of requests processed and blocked by the component/data source
12+
- Number of triggers for each rule
1313

1414
```
1515
Appsec Metrics:
@@ -43,8 +43,8 @@ Setting `log_level` to `debug` or `trace` in the acquisition config section or t
4343

4444

4545
When enabling debug at acquisition or AppSec config level:
46-
- load time debug will be enabled, such as information regarding the translation of the rule to `SecRule` format.
47-
- runtime debug will be enabled for all the rules loaded by the AppSec Component / AppSec config.
46+
- Load-time debug is enabled (for example, rule translation to `SecRule` format).
47+
- Runtime debug is enabled for all rules loaded by the AppSec Component/AppSec config.
4848

4949
When enabling debug directly at the AppSec rule level, only runtime evaluation details for that rule are displayed, such as:
5050

@@ -107,10 +107,10 @@ INFO[2023-12-06 14:58:19] loading acquisition file : ...
107107

108108
## Testing a given rule
109109

110-
We can create a skeleton environment with:
111-
- acquisition config that is going to load your test AppSec config
112-
- appsec config to load the test rule
113-
- the test rule itself
110+
Create a minimal test environment with:
111+
- An acquisition config that loads your test AppSec config
112+
- An AppSec config that loads the test rule
113+
- The test rule itself
114114

115115
> /etc/crowdsec/acquis.d/test_appsec.yaml
116116
```bash
@@ -170,13 +170,13 @@ time="2023-12-20 13:39:29" level=info msg="Appsec Runner ready to process event"
170170

171171
## Interacting with the AppSec Component
172172

173-
To test that the AppSec Component is working correctly, you can send requests directly to it. A few things to know:
174-
- To query the AppSec Component, you need to have a valid remediation component API Key
175-
- The AppSec Component expects to receive some of the elements in specific headers
173+
To test that the AppSec Component is working, send requests directly to it. Keep in mind:
174+
- You need a valid remediation component API key
175+
- The AppSec Component expects specific values in headers
176176

177177

178-
We are going to test that the AppSec Component detects correctly CVE-2023-42793, which is part of the [virtual patching collection](https://app.crowdsec.net/hub/author/crowdsecurity/collections/appsec-virtual-patching), that should be installed for this to work (see `cscli appsec-rules list`).
179-
[This rule](https://app.crowdsec.net/hub/author/crowdsecurity/appsec-rules/vpatch-CVE-2023-42793) is pretty straightforward and detects requests to an URI ending with `/rpc2`:
178+
This example tests detection for CVE-2023-42793, part of the [virtual patching collection](https://app.crowdsec.net/hub/author/crowdsecurity/collections/appsec-virtual-patching). Make sure the collection is installed (`cscli appsec-rules list`).
179+
[This rule](https://app.crowdsec.net/hub/author/crowdsecurity/appsec-rules/vpatch-CVE-2023-42793) detects requests to a URI ending with `/rpc2`:
180180

181181
> cat /etc/crowdsec/appsec-rules/vpatch-CVE-2023-42793.yaml
182182
```yaml
@@ -204,13 +204,13 @@ labels:
204204
- cwe.CWE-288
205205
```
206206
207-
To be able to communicate with the AppSec Component, let's create a bouncer API Key:
207+
To communicate with the AppSec Component, create a bouncer API key:
208208
209209
```bash
210210
cscli bouncers add appsec_test -k this_is_a_bad_password
211211
```
212212

213-
We can now query our AppSec Component (we're assuming here that it runs on the default `127.0.0.1:7422`, see the `listen_addr` parameter of the acquisition config):
213+
You can now query the AppSec Component (assuming the default `127.0.0.1:7422`; see the `listen_addr` setting in your acquisition config):
214214

215215
```bash
216216
▶ curl -X POST localhost:7422/ -i -H 'x-crowdsec-appsec-ip: 192.168.1.1' -H 'x-crowdsec-appsec-uri: /rpc2' -H 'x-crowdsec-appsec-host: google.com' -H 'x-crowdsec-appsec-verb: POST' -H 'x-crowdsec-appsec-api-key: this_is_a_bad_password'
@@ -222,15 +222,15 @@ Content-Type: text/plain; charset=utf-8
222222
{"action":"ban"}
223223
```
224224

225-
And we see the alert appearing in `crowdsec.log` :
225+
The alert should appear in `crowdsec.log`:
226226

227227
```
228228
...
229229
INFO[2023-12-05 12:17:52] (test) alert : crowdsecurity/vpatch-CVE-2023-42793 by ip 192.168.1.1
230230
...
231231
```
232232

233-
And in `cscli alerts list` :
233+
And in `cscli alerts list`:
234234

235235
```
236236
╭────┬────────────────┬─────────────────────────────────────┬─────────┬────┬───────────┬───────────────────────────────╮

crowdsec-docs/docs/local_api/allowlists.md

Lines changed: 24 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -6,10 +6,16 @@ sidebar_position: 7
66

77
# AllowLists
88

9-
The AllowLists feature in CrowdSec lets users manage IP-based allowlists at the LAPI level, affecting both local decisions and blocklist pulls. [Paying customers can also control AllowLists directly from the console for added convenience](/u/console/allowlists). This ensures greater flexibility in managing trusted IPs while maintaining CrowdSec’s robust security measures.
9+
The AllowLists feature in CrowdSec lets you manage IP-based allowlists at the LAPI level. It affects both local decisions and blocklist pulls, giving you more flexibility to trust specific IPs while keeping CrowdSec security controls in place.
1010

11+
:::tip CrowdSec Premium: Centralized Console Management
1112

12-
The AllowLists affect local decision and blocklist pulls in different ways:
13+
Premium users can manage AllowLists directly from the [CrowdSec Console](/u/console/allowlists), enabling centralized management across multiple Security Engines and Integrations. Console-managed allowlists require subscribing entities (Security Engines, Integrations, or Organizations) after creation.
14+
15+
:::
16+
17+
18+
AllowLists affect local decisions and blocklist pulls in different ways:
1319

1420
| Area | Action | Real Time |
1521
|-------|------|------|
@@ -19,35 +25,35 @@ The AllowLists affect local decision and blocklist pulls in different ways:
1925
| cscli | Decision is blocked unless special flag is provided ||
2026

2127

22-
AllowLists are limited to IP/Range based rules. If you need rules that rely on log elements such as URL and so on, [Parser Whitelists](/log_processor/whitelist/introduction.md) or [Profile Rules](/local_api/profiles/format.md) might more relevant.
28+
AllowLists are limited to IP/range-based rules. If you need rules based on log elements (such as URLs), [Parser Whitelists](/log_processor/whitelist/introduction.md) or [Profile Rules](/local_api/profiles/format.md) might be more relevant.
2329

2430

25-
### Creating an allowlist
31+
## Creating an allowlist
2632

27-
Allowlists creation is done with `cscli allowlists create`, for example: `cscli allowlists create my_allowlistd -d safe_ips`.
33+
Create an allowlist with `cscli allowlists create`, for example: `cscli allowlists create my_allowlist -d safe_ips`.
2834

29-
The `-d` parameter is mandatory, it's a description for the allowlist for future reference:
35+
The `-d` flag is mandatory. It sets a description for future reference:
3036
```bash
3137
$ cscli allowlists create my_allowlist --description "test allowlist"
3238
allowlist 'my_allowlist' created successfully
3339
```
3440

3541
This command must be run on the LAPI.
3642

37-
### Adding entries to an allowlist
43+
## Adding entries to an allowlist
3844

39-
Adding new entries to an allowlist is done with `cscli allowlists add <allowlist_name> value_1 value_2 ...`.
45+
Add entries with `cscli allowlists add <allowlist_name> value_1 value_2 ...`.
4046

4147
The allowlist must exist.
4248

4349
By default, allowlist entries have no expiration, but you can specify one with the `-e` flag:
4450

4551
```bash
46-
$ cscli allowlist add my_allowlist 1.2.3.4 -e 7d
52+
$ cscli allowlists add my_allowlist 1.2.3.4 -e 7d
4753
added 1 values to allowlist my_allowlist
4854
```
4955

50-
Values can be either IPs or ranges.
56+
Values can be IPs or ranges.
5157

5258
You can add an optional description for each entry with the `-d` flag:
5359

@@ -56,14 +62,14 @@ $ cscli allowlists add my_allowlist 1.2.3.4 -e 7d -d "pentest IPs"
5662
added 1 values to allowlist my_allowlist
5763
```
5864

59-
You cannot add the same values twice to an allowlist: if you want to edit an entry, you'll need to remove it first then add it again.
65+
You cannot add the same value twice to an allowlist. To edit an entry, remove it first, then add it again.
6066

6167
This command must be run on the LAPI.
6268

6369

64-
### Removing entries from an allowlist
70+
## Removing entries from an allowlist
6571

66-
Removing entries from an allowlist is done with `cscli allowlists remove <allowlist_name> value_1 value_2 ...`:
72+
Remove entries with `cscli allowlists remove <allowlist_name> value_1 value_2 ...`:
6773
```bash
6874
$ cscli allowlists remove my_allowlist 1.2.3.4
6975
removed 1 values from allowlist my_allowlist
@@ -72,12 +78,12 @@ removed 1 values from allowlist my_allowlist
7278
This command must be run on the LAPI.
7379

7480

75-
### Viewing the content of an allowlist
81+
## Viewing the contents of an allowlist
7682

77-
Allowlists can be inspected with `cscli allowlists inspect <allowlist_name>`:
83+
Inspect an allowlist with `cscli allowlists inspect <allowlist_name>`:
7884

7985
```bash
80-
$ cscli allowlist inspect my_allowlist
86+
$ cscli allowlists inspect my_allowlist
8187

8288
──────────────────────────────────────────────
8389
Allowlist: my_allowlist
@@ -99,9 +105,9 @@ $ cscli allowlist inspect my_allowlist
99105

100106
This command can be run on the LAPI or any log processor machine.
101107

102-
### Deleting an allowlist
108+
## Deleting an allowlist
103109

104-
Allowlists can be deleted with `cscli allowlists delete <allowlist_name>`:
110+
Delete an allowlist with `cscli allowlists delete <allowlist_name>`:
105111

106112
```bash
107113
$ cscli allowlists delete my_allowlist

crowdsec-docs/docs/log_processor/whitelist/capi_based_whitelist.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -6,29 +6,29 @@ title: CAPI
66
:::warning
77

88
This option is deprecated.
9-
You should use [centralized allowlists](local_api/allowlists.md) instead.
9+
You should use [centralized allowlists](/local_api/allowlists.md) instead.
1010

1111
:::
1212

1313
## Whitelists from CAPI (Central API) community blocklist or third party blocklist
1414

15-
From version 1.5.0 a user can specify a list of IP's or IP ranges to be whitelisted from a community blocklist or third party blocklist. You will have to specify a path to the file within `config.yaml` as by default there is no file specified.
15+
From version 1.5.0, you can define IPs or IP ranges to whitelist from the community blocklist or third-party blocklists. Set the whitelist file path in `config.yaml` (no default path is set).
1616

1717
```yaml
1818
api:
1919
server:
2020
capi_whitelists_path: <path_to_capi_whitelists_file>
2121
```
2222
23-
We recommend to use the following files for each OS:
23+
Recommended file paths:
2424
2525
- Linux `/etc/crowdsec/capi-whitelists.yaml`
2626
- Freebsd `/usr/local/etc/crowdsec/capi-whitelists.yaml`
2727
- Windows `c:/programdata/crowdsec/config/capi-whitelists.yaml`
2828

29-
*These files **DO NOT** exist and you **MUST** create them manually and configure the above settings*
29+
*These files **DO NOT** exist by default. You **MUST** create them manually and set the path above.*
3030

31-
The following snippet should be used as a guide
31+
Example file content:
3232

3333
```yaml
3434
ips:
@@ -44,8 +44,8 @@ sudo systemctl reload crowdsec
4444

4545
:::warning
4646

47-
The whitelist only applies when crowdsec pulls the blocklist from CAPI. This means that any IPs already in your local database will not get whitelisted.
47+
The whitelist applies only when CrowdSec pulls blocklists from CAPI. IPs already in your local database are not retroactively whitelisted.
4848

49-
You can either manually delete the decisions for the IPs you want to whitelist with `cscli decisions delete`, or delete all alerts and active decisions from the database with `cscli alerts delete --all` and restart crowdsec.
49+
You can either delete decisions for specific IPs with `cscli decisions delete`, or delete all alerts and active decisions with `cscli alerts delete --all` and then restart CrowdSec.
5050

5151
:::

crowdsec-docs/docs/log_processor/whitelist/expr_based_whitelist.md

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -26,23 +26,23 @@ There are two main ways to create an expression-based whitelist:
2626

2727
### Path 1: Starting from an alert
2828

29-
When you have a false positive alert, inspect it to extract event details and build a whitelist.
29+
When you have a false-positive alert, inspect it to extract event details and build a whitelist.
3030

31-
#### Step 1: Identify the alert and extract its events
31+
#### Step 1: Identify the alert and extract events
3232

3333
1. List recent alerts:
3434

3535
```bash
3636
sudo cscli alerts list
3737
```
3838

39-
2. Inspect the alert with event details:
39+
2. Inspect an alert with event details:
4040

4141
```bash
4242
sudo cscli alerts inspect <ALERT_ID> -d
4343
```
4444

45-
The `-d/--details` flag shows the events associated with the alert. From the output, note:
45+
The `-d/--details` flag shows events associated with the alert. From the output, note:
4646

4747
- The **log type** (e.g., `nginx`, `apache2`, `sshd`, etc.)
4848
- Any helpful **meta** fields (http path, status, verb, user-agent, etc.)
@@ -152,13 +152,13 @@ In this example, the events section lists keys such as `http_path`, `http_status
152152

153153
</details>
154154

155-
#### Step 2: Extract a representative log line
155+
#### Step 2: Pick a representative log line
156156

157-
From the alert details, pick one triggering log line. You will need the raw line for `cscli explain` in the next step.
157+
From the alert details, pick one triggering log line. You will use the raw line with `cscli explain` in the next step.
158158

159159
#### Step 3: Use `cscli explain` to reveal parsed fields
160160

161-
To write a safe whitelist, you need the exact field names and values CrowdSec has at parse/enrich time.
161+
To write a safe whitelist, use the exact field names and values available at parse/enrich time.
162162

163163
Run `cscli explain` against the log line:
164164

@@ -169,7 +169,7 @@ sudo cscli explain \
169169
-v
170170
```
171171

172-
`cscli explain -v` shows which parsers ran and what they populated in `evt.Parsed.*`, `evt.Meta.*`, and so on.
172+
`cscli explain -v` shows which parsers ran and what they populated in `evt.Parsed.*`, `evt.Meta.*`, and related fields.
173173

174174
**What to look for** in the explain output:
175175

@@ -186,7 +186,7 @@ When you already know the log line pattern you want to whitelist (e.g., health c
186186

187187
#### Step 1: Use `cscli explain` to reveal parsed fields
188188

189-
You can use [cscli explain](/cscli/cscli_explain.md) to generate output from a log line or a log file.
189+
Use [cscli explain](/cscli/cscli_explain.md) on a log line or a log file.
190190

191191
For example, with a single log line:
192192

@@ -284,7 +284,7 @@ line: 5.5.8.5 - - [04/Jan/2020:07:25:02 +0000] "GET /.well-known/acme-challenge/
284284
└ 🟢 crowdsecurity/http-probing
285285
```
286286

287-
You can see what data is available from the `s01-parse` stage. Look for fields in `evt.Parsed.*` and `evt.Meta.*` that you can use in your whitelist expression.
287+
This output shows what data is available from the `s01-parse` stage. Look for fields in `evt.Parsed.*` and `evt.Meta.*` that you can use in your whitelist expression.
288288

289289
</details>
290290

@@ -324,7 +324,7 @@ whitelist:
324324

325325
:::tip
326326

327-
Keep whitelist expressions as narrow as possible (path + verb + maybe user-agent) to avoid hiding real attacks.
327+
Keep whitelist expressions as narrow as possible (path + verb + optional user-agent) to avoid hiding real attacks.
328328

329329
:::
330330

0 commit comments

Comments
 (0)