Stop using double quoted string literals in SQL

[tuffnatty]

This PR addresses the problem in #2001 and #2473, which is also touched by #2004 and #2474.

The failing tests are failing in the current main as well.

---

📚 Documentation preview 📚: https://datasette--2616.org.readthedocs.build/en/2616/

[tuffnatty]

One of these changes probably needs some discussion. It's c6a47e8.

Before disabling double-quoted string literals, passing extra query string parameters not matching any known column name would add always-false WHERE clauses to SQL, e.g.: &z=baz would generate WHERE "z" = 'baz', which, when double-quoted string literals are enabled, is an always-false string literal comparison, and the view returns an empty result set. But when double-quoted strings are disabled, this WHERE clause generates an error.

Here I have chosen to ignore query string parameters not matching any known column. It changes the existing behaviour. Another option is to return an error. I am not sure returning an empty result set is a behaviour worth keeping.

[tuffnatty]

I have switched from compiling SQLite3 with -DSQLITE_DQS=0 to disabling double-quoted strings programmatically with Connection.setconfig() in tests on supported platforms (SQLite 3.29.0+, Python 3.12+). This is enough for testing, allows offline testing without rebuilding SQLite, and can also be easily backported to 0.65.x.

[tuffnatty]

Hi @simonw ,

should I rebase this PR on the master branch fixing the new conflicts and keep supporting it, or should I bury hopes that one day datasette will switch to standard string quoting? More and more distributions switch to compiling SQLite with -DSQLITE_DQS=0 (which is sane, standard-compliant and recommended by the upstream). My personal point of concern is FreeBSD binary packages. But if you have some reasons to keep using double quotes for string quoting throughout the code, I'll just keep using my fork and forget about this PR and #2617.

Looking forward to hearing from you.