Skip to content

Commit 0826924

Browse files
ColtonPayneColtonPayne
authored
Merge pull request #150 from lab-v2/daniel/counterfactual-tutorial
Counterfactual reasoning tutorial
2 parents 43c51cf + 46565d1 commit 0826924

3 files changed

Lines changed: 733 additions & 0 deletions

File tree

‎docs/source/tutorials/counterfactual_tutorial.rst‎

Lines changed: 317 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,317 @@
1+
Counterfactual Reasoning
2+
=========================
3+
4+
.. note::
5+
6+
Find the full, executable code `here <https://github.com/lab-v2/pyreason/blob/main/examples/counterfactual_tutorial_ex.py>`_
7+
8+
This tutorial extends the cybersecurity inconsistency tutorial. The
9+
inconsistency tutorial asked: *"do these facts conflict?"* This
10+
tutorial asks the opposite question: *"if I changed something, would
11+
the conclusions still hold?"* That is what a counterfactual is -- a
12+
"what if" question answered by re-running reasoning on a modified
13+
input.
14+
15+
PyReason has no built-in counterfactual operator. We implement them
16+
manually: run reasoning on the original graph, run again on a modified
17+
copy, and compare the two final states. The differences tell us which
18+
conclusions depended on the change.
19+
20+
.. note::
21+
22+
This tutorial reuses the cybersecurity knowledge graph from the
23+
*Cybersecurity Inconsistency* tutorial. Familiarity with that tutorial
24+
is assumed.
25+
26+
What a Grounding Is
27+
-------------------
28+
29+
Before the demos: a brief note on terminology, since the rest of the
30+
tutorial leans on this concept.
31+
32+
A rule like ``at_risk(X) <- runs(X, Y), has_cve(Y, Z)`` does not "fire"
33+
once. It fires once for each combination of nodes that satisfies the
34+
body. Each such combination is called a **grounding**. In the
35+
cybersecurity graph there are three groundings of this rule, one for
36+
each asset:
37+
38+
============= =========================== ============================
39+
Grounding for X = ... Y = ..., Z = ...
40+
============= =========================== ============================
41+
1 ``web_server`` ``sudo_1_9_5p1``, ``cve_2021_3156``
42+
2 ``workstation_1`` ``linux_kernel_5_1``, ``cve_2022_0185``
43+
3 ``dev_server`` ``openssl_3_0_1``, ``cve_2022_26923``
44+
============= =========================== ============================
45+
46+
Each grounding is independent. A counterfactual perturbation can add,
47+
modify, or remove specific groundings -- adding an edge or fact can
48+
introduce new groundings, modifying a bound can change whether a
49+
grounding's threshold is met, and removing an edge or fact eliminates
50+
any grounding that depended on it. The others fire normally. This is
51+
why counterfactual perturbations have such localized effects -- they
52+
target specific groundings without touching the rest.
53+
54+
The Three Demos
55+
---------------
56+
57+
The script ``counterfactual_tutorial_ex.py`` walks through three
58+
counterfactuals:
59+
60+
- **Demo 1** -- remove a graph edge. Show that one grounding of
61+
``exposure_rule`` disappears, and the cascade collapses for the
62+
affected asset.
63+
- **Demo 2** -- inject a contradicting fact. Show that PyReason
64+
detects the conflict and reports an inconsistency.
65+
- **Demo 3** -- start from a graph that *already has* an inconsistency.
66+
Counterfactually remove a candidate cause and check whether the
67+
inconsistency disappears.
68+
69+
Each run writes a CSV trace alongside the script. Trace rows
70+
are referenced inline below.
71+
72+
Demo 1: Remove a Graph Edge
73+
---------------------------
74+
75+
**Question:** if ``web_server`` did not run ``sudo_1_9_5p1``, would it
76+
still be classified as at risk?
77+
78+
The ``exposure_rule`` says: a host is at risk if it runs some software
79+
that has a CVE. In PyReason, a graph edge with an attribute is treated
80+
as a fact -- the ``runs=1`` edge between ``web_server`` and
81+
``sudo_1_9_5p1`` becomes the fact ``runs(web_server, sudo_1_9_5p1)``
82+
during reasoning. Removing the edge removes that fact, so the
83+
grounding of ``exposure_rule`` for ``web_server`` no longer has
84+
anything to bind ``Y`` to and cannot fire.
85+
86+
In the baseline trace, the rule fires three times -- once per asset.
87+
The relevant rows are::
88+
89+
Time Op Node Label Old Bound New Bound Caused By Consistent Clause-1
90+
0 1 web_server at_risk [0.0,1.0] [1.0,1.0] exposure_rule True [('web_server', 'sudo_1_9_5p1')]
91+
0 1 workstation_1 at_risk [0.0,1.0] [1.0,1.0] exposure_rule True [('workstation_1', 'linux_kernel_5_1')]
92+
0 1 dev_server at_risk [0.0,1.0] [1.0,1.0] exposure_rule True [('dev_server', 'openssl_3_0_1')]
93+
94+
The ``Clause-1`` column shows which graph elements satisfied each
95+
grounding's body. After we remove the edge, the counterfactual trace
96+
contains only the second and third rows. The ``web_server`` row is
97+
gone -- because the edge it depended on no longer exists::
98+
99+
Time Op Node Label Old Bound New Bound Caused By Consistent Clause-1
100+
0 1 workstation_1 at_risk [0.0,1.0] [1.0,1.0] exposure_rule True [('workstation_1', 'linux_kernel_5_1')]
101+
0 1 dev_server at_risk [0.0,1.0] [1.0,1.0] exposure_rule True [('dev_server', 'openssl_3_0_1')]
102+
103+
The three downstream rules each require ``at_risk`` to have fired for
104+
the same node before they can produce a grounding:
105+
106+
- ``vulnerability_rule``: ``vulnerable(X):[0.8, 1.0] <- at_risk(X)``
107+
- ``compromise_rule``: ``compromised(X):[0.8, 1.0] <- vulnerable(X):[0.5, 1.0]``
108+
- ``unpatched_rule``: ``patch_confidence(X):[0.0, 0.2] <- compromised(X):[0.5, 1.0]``
109+
110+
With ``at_risk(web_server)`` absent from the counterfactual trace,
111+
none of them have a valid grounding for ``web_server``.
112+
113+
Diff vs. baseline (final state of each run, side by side):
114+
115+
================ ================= ================= ==================
116+
node predicate baseline counterfactual
117+
================ ================= ================= ==================
118+
web_server at_risk [1.0, 1.0] (none)
119+
web_server vulnerable [0.8, 1.0] (none)
120+
web_server compromised [0.8, 1.0] (none)
121+
web_server patch_confidence [0.0, 0.2] (none)
122+
================ ================= ================= ==================
123+
124+
``workstation_1`` and ``dev_server`` are unaffected -- their groundings
125+
remain intact. One edge removed, four conclusions lost, all on a single
126+
asset. That is the basic shape of a counterfactual: a small input
127+
change has a localized but compounded effect downstream.
128+
129+
Demo 2: Inject a Contradicting Fact
130+
-----------------------------------
131+
132+
**Question:** what happens if we assert that ``workstation_1`` is
133+
*definitely not* at risk, even though the graph would normally infer
134+
that it is?
135+
136+
We add a fact ``at_risk(workstation_1):[0.0, 0.0]`` -- meaning "I am
137+
100% certain this is false." The graph still says it should be
138+
``[1.0, 1.0]`` (true), so two sources will disagree.
139+
140+
In the trace, the injected fact lands first::
141+
142+
Time Op Node Label Old Bound New Bound Caused By Consistent
143+
0 0 workstation_1 at_risk [0.0,1.0] [0.0,0.0] cf_not_at_risk True
144+
145+
Then the ``exposure_rule`` fires and tries to write ``[1.0, 1.0]``.
146+
The two bounds do not overlap, so PyReason flags an inconsistency::
147+
148+
Time Op Node Label Old Bound New Bound Caused By Consistent
149+
0 1 workstation_1 at_risk [0.0,0.0] [0.0,1.0] exposure_rule False
150+
151+
Inconsistency Message:
152+
"Inconsistency occurred. Conflicting bounds for at_risk(workstation_1).
153+
Update from [0.000, 0.000] to [1.000, 1.000] is not allowed.
154+
Setting bounds to [0,1] and static=True for this timestep."
155+
156+
For that one timestep, the bound is set to ``[0.0, 1.0]`` (fully
157+
unknown) and marked static. At the next timestep, the injected fact
158+
re-asserts itself (its time window covers all timesteps), pulling the
159+
bound back to ``[0.0, 0.0]``. The final state shows ``[0.0, 0.0]`` --
160+
the injection wins by being re-asserted.
161+
162+
**Why the cascade breaks downstream:**
163+
164+
The next rule in the chain is ``vulnerability_rule``::
165+
166+
vulnerable(X):[0.8, 1.0] <- at_risk(X)
167+
168+
In the baseline trace, this rule fires three times (one grounding per
169+
asset)::
170+
171+
Time Op Node Label Caused By Consistent Clause-1
172+
0 2 web_server vulnerable vulnerability_rule True ['web_server']
173+
0 2 workstation_1 vulnerable vulnerability_rule True ['workstation_1']
174+
0 2 dev_server vulnerable vulnerability_rule True ['dev_server']
175+
176+
In the counterfactual trace, only two rows appear -- the
177+
``workstation_1`` grounding is missing::
178+
179+
0 2 web_server vulnerable vulnerability_rule True ['web_server']
180+
0 2 dev_server vulnerable vulnerability_rule True ['dev_server']
181+
182+
That grounding does not fire because its body
183+
(``at_risk(workstation_1)``) is held at ``[0.0, 0.0]``, which fails
184+
the rule's threshold check. With no ``vulnerable(workstation_1)``,
185+
``compromise_rule`` cannot fire for ``workstation_1`` either, and the
186+
chain breaks one grounding at a time.
187+
188+
The same three downstream rules from Demo 1 apply here:
189+
190+
- ``vulnerability_rule``: ``vulnerable(X):[0.8, 1.0] <- at_risk(X)``
191+
- ``compromise_rule``: ``compromised(X):[0.8, 1.0] <- vulnerable(X):[0.5, 1.0]``
192+
- ``unpatched_rule``: ``patch_confidence(X):[0.0, 0.2] <- compromised(X):[0.5, 1.0]``
193+
194+
Diff vs. baseline:
195+
196+
================ ================= ================= ==================
197+
node predicate baseline counterfactual
198+
================ ================= ================= ==================
199+
workstation_1 at_risk [1.0, 1.0] [0.0, 0.0]
200+
workstation_1 vulnerable [0.8, 1.0] (none)
201+
workstation_1 compromised [0.8, 1.0] (none)
202+
workstation_1 patch_confidence [0.0, 0.2] (none)
203+
================ ================= ================= ==================
204+
205+
A single injected fact eliminated three downstream groundings, exactly
206+
as in Demo 1 -- but via a different mechanism. In Demo 1 a graph edge
207+
was removed; in Demo 2 a fact was added that produced an inconsistency,
208+
and the resolved bound was incompatible with downstream rules.
209+
210+
Demo 3: Diagnose an Existing Inconsistency
211+
------------------------------------------
212+
213+
Demos 1 and 2 perturbed an otherwise-consistent baseline. Demo 3 is
214+
different: **the baseline already contains an inconsistency** before
215+
any perturbation. The counterfactual question is: which fact is
216+
causing it?
217+
218+
**The baseline inconsistency:**
219+
220+
The four-rule chain ends with ``unpatched_rule``, which says: a
221+
compromised host has low patch confidence::
222+
223+
patch_confidence(X):[0.0, 0.2] <- compromised(X):[0.5, 1.0]
224+
225+
This rule fires for all three assets. Separately, the fact
226+
``dev_patch_db_fact`` asserts ``patch_confidence(dev_server):[0.9, 1.0]``
227+
-- "the patch database says dev_server is well patched."
228+
229+
For ``dev_server``, both writes target the same atom with
230+
non-overlapping bounds. Looking at the baseline trace::
231+
232+
Time Op Node Label Old Bound New Bound Caused By Consistent
233+
0 0 dev_server patch_confidence [0.0,1.0] [0.9,1.0] dev_patch_db_fact True
234+
...
235+
0 4 dev_server patch_confidence [0.9,1.0] [0.0,1.0] unpatched_rule False
236+
237+
Inconsistency Message:
238+
"Inconsistency occurred. Conflicting bounds for patch_confidence(dev_server).
239+
Update from [0.900, 1.000] to [0.000, 0.200] is not allowed.
240+
Setting bounds to [0,1] and static=True for this timestep."
241+
242+
For ``web_server`` and ``workstation_1`` the same rule operation is
243+
consistent::
244+
245+
0 4 web_server patch_confidence [0.0,1.0] [0.0,0.2] unpatched_rule True
246+
0 4 workstation_1 patch_confidence [0.0,1.0] [0.0,0.2] unpatched_rule True
247+
248+
Why is only ``dev_server`` inconsistent? Because only ``dev_server``
249+
had a prior asserted bound for ``patch_confidence`` (the
250+
``dev_patch_db_fact``). The other two assets had the default
251+
``[0.0, 1.0]`` bound, which the rule's update fits inside.
252+
253+
After the inconsistency, the asserted fact re-asserts at later
254+
timesteps, so the final state ends up at ``[0.9, 1.0]``. But the
255+
trace preserves the record of the conflict that occurred along the way.
256+
257+
**The counterfactual:**
258+
259+
We re-run reasoning with ``dev_patch_db_fact`` removed.
260+
261+
In the counterfactual trace, the same ``unpatched_rule`` grounding
262+
fires for ``dev_server`` -- the rule body still holds. But now there
263+
is no prior bound to conflict with::
264+
265+
0 4 dev_server patch_confidence [0.0,1.0] [0.0,0.2] unpatched_rule True
266+
267+
Consistent. No inconsistency message.
268+
269+
Diff vs. baseline:
270+
271+
================ ================= ================= ==================
272+
node predicate baseline counterfactual
273+
================ ================= ================= ==================
274+
dev_server patch_confidence [0.9, 1.0] [0.0, 0.2]
275+
================ ================= ================= ==================
276+
277+
Removing one fact eliminated the baseline inconsistency. This is the
278+
diagnostic pattern -- when an inconsistency exists and you want to
279+
know which fact caused it, counterfactually remove each candidate and
280+
check whether the conflict goes away.
281+
282+
In a real system with many facts and many rules, this becomes a
283+
systematic technique: remove each fact one at a time, observe which
284+
removals eliminate the inconsistency, and you have identified the
285+
load-bearing inputs.
286+
287+
Notice that the *grounding itself* is identical in both runs. The
288+
``unpatched_rule`` grounding for ``dev_server`` fires in both. What
289+
changes is the outcome of that grounding's update -- inconsistent in
290+
the baseline, consistent in the counterfactual. That is a more subtle
291+
effect than Demos 1 and 2, where the fact change eliminated
292+
groundings outright. Here the grounding stays; only its consistency
293+
changes.
294+
295+
Running the Code
296+
----------------
297+
298+
::
299+
300+
python examples/counterfactual_tutorial_ex.py
301+
302+
CSV traces are written to the working directory. The script runs
303+
all three demos in sequence and prints the diffs above.
304+
305+
Summary
306+
-------
307+
308+
Two things to take away:
309+
310+
1. **Counterfactuals are re-runs and diffs.** PyReason does not
311+
provide them as a built-in operator. The pattern is: run, perturb,
312+
run again, compare.
313+
314+
2. **Perturbations affect groundings.** The unit of rule firing is the
315+
grounding -- a specific instantiation of a rule's variables.
316+
Counterfactual changes either eliminate groundings (Demos 1 and 2)
317+
or change whether their resulting updates are consistent (Demo 3).

‎docs/source/tutorials/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ Contents
1919
./image_classifier_reasoning.rst
2020
./temporal_classifier_tutorial.rst
2121
./cybersecurity_inconsistency.rst
22+
./counterfactual_tutorial.rst
2223
./load_rules_facts_from_file.rst
2324
./llm_generated_rules.rst
2425
./natural_language_to_pyreason.rst

0 commit comments

Comments
 (0)