boolean queries can give wrong results

[simonmichael]

I found an example that's easy to reproduce:

$ hledger -f examples/sample2.journal reg expr:'(checking and amt:>0) or credit'
2025-01-01 starting balances    assets:bank:checking   1000.00 USD   1000.00 USD
                                li:credit card         -400.00 USD    600.00 USD
2025-01-01 pay rent             assets:bank:checking   -800.00 USD   -200.00 USD
2025-01-02 salary               assets:bank:checking   1000.00 USD    800.00 USD
2025-01-03 pay half of credi..  assets:bank:checking   -200.00 USD    600.00 USD
                                li:credit card          200.00 USD    800.00 USD
2025-01-04 shopping             assets:bank:checking   -250.00 USD    550.00 USD

This should not be showing the negative checking postings. I expected to see:

2025-01-01 starting balances    assets:bank:checking   1000.00 USD   1000.00 USD
                                li:credit card         -400.00 USD    600.00 USD
2025-01-02 salary               assets:bank:checking   1000.00 USD   1600.00 USD
2025-01-03 pay half of credi..  li:credit card          200.00 USD   1800.00 USD

The problem is with the filterQuery calls in journalValueAndFilterPostingsWith. It doesn't know how to handle a non-trivial boolean query, and ends up altering it:

ghci> filterQuery queryIsAmtOrSym $ Or [ And [ Acct ( toRegex' "checking" ) , Amt Gt 0 ] , Acct ( toRegex' "credit" ) ]
Any
ghci> filterQuery (not.queryIsAmtOrSym) $ Or [ And [ Acct ( toRegex' "checking" ) , Amt Gt 0 ] , Acct ( toRegex' "credit" ) ]
Or
    [ Acct
        ( Regexp "checking" )
    , Acct
        ( Regexp "credit" )
    ]

[simonmichael]

And for the record, removing those filterQuery calls (just using q for both) fixes the above case, but breaks others, like this one:

hledger -f examples/sample2.journal reg not:desc:pay

[simonmichael]

The query is split into two "equivalent" queries which can be applied in stages, before and after valuation:

We make sure to first filter by amt: and cur: terms, then value the 'Journal', then filter by the remaining terms.

which I believe was done to accommodate #1625 (2021), making cur: and amt: match the pre-valuation amounts. But this is incompatible with boolean queries (#1989, 2023), which can't be split this way without changing their meaning.

[simonmichael]

Posted a draft fix, #2373, with some ideas.

[simonmichael]

Fixed by #2387.