1039 Improve LazyTableReference (#1040)

* add test

* improve lazy table refs

* improve docs for `LazyTableReference`

* add docs for circular imports

Author: dantownsend

docs/src/piccolo/projects_and_apps/included_apps.rst

@@ -38,6 +38,8 @@ Lets you scaffold an ASGI web app. See :ref:`ASGICommand`.
 
 -------------------------------------------------------------------------------
 
+.. _Fixtures:
+
 fixtures
 ~~~~~~~~
 

docs/src/piccolo/schema/m2m.rst

@@ -54,8 +54,9 @@ We create it in Piccolo like this:
 
 
 .. note::
-    We use ``LazyTableReference`` because when Python evaluates ``Band`` and
-    ``Genre``, the ``GenreToBand`` class doesn't exist yet.
+    We use :class:`LazyTableReference <piccolo.columns.reference.LazyTableReference>`
+    because when Python evaluates ``Band`` and ``Genre``, the ``GenreToBand``
+    class doesn't exist yet.
 
 By using ``M2M`` it unlocks some powerful and convenient features.
 

docs/src/piccolo/tutorials/avoiding_circular_imports.rst

@@ -0,0 +1,82 @@
+Avoiding circular imports
+=========================
+
+How Python imports work
+-----------------------
+
+When Python imports a file, it evaluates it from top to bottom.
+
+With :class:`ForeignKey  <piccolo.columns.column_types.ForeignKey>` columns we
+sometimes have to reference tables lower down in the file (which haven't been
+evaluated yet).
+
+The solutions are:
+
+* Try and move the referenced table to a different Python file.
+* Use :class:`LazyTableReference <piccolo.columns.reference.LazyTableReference>`
+
+Import ``Table`` definitions as early as possible
+-------------------------------------------------
+
+In the entrypoint to your app, at the top of the file, it's recommended to
+import your tables.
+
+.. code-block:: python
+
+    # main.py
+    from my_app.tables import Manager, Band
+
+This ensures that the tables are imported, and setup correctly.
+
+Keep table files focused
+------------------------
+
+You should try and keep your ``tables.py`` files pretty focused (i.e.
+just contain your ``Table`` definitions).
+
+If you have lots of logic alongside your ``Table`` definitions, it might cause
+your ``LazyTableReference`` references to evaluate too soon (causing circular
+import errors). An example of this is with
+:func:`create_pydantic_model <piccolo.utils.pydantic.create_pydantic_model>`:
+
+.. literalinclude:: avoiding_circular_imports_src/tables.py
+
+Simplify your schema if possible
+--------------------------------
+
+Even with :class:`LazyTableReference <piccolo.columns.reference.LazyTableReference>`,
+you may run into some problems if your schema is really complicated.
+
+An example is when you have two tables, and they have foreign keys to each other.
+
+.. code-block:: python
+
+    class Band(Table):
+        name = Varchar()
+        manager = ForeignKey("Manager")
+
+
+    class Manager(Table):
+        name = Varchar()
+        favourite_band = ForeignKey(Band)
+
+
+Piccolo should be able to create these tables, and query them. However, some
+Piccolo tooling may struggle - for example when loading :ref:`fixtures <Fixtures>`.
+
+A joining table can help in these situations:
+
+.. code-block:: python
+
+    class Band(Table):
+        name = Varchar()
+        manager = ForeignKey("Manager")
+
+
+    class Manager(Table):
+        name = Varchar()
+
+
+    class ManagerFavouriteBand(Table):
+        manager = ForeignKey(Manager, unique=True)
+        band = ForeignKey(Band)

docs/src/piccolo/tutorials/avoiding_circular_imports_src/tables.py

@@ -0,0 +1,22 @@
+# tables.py
+
+from piccolo.columns import ForeignKey, Varchar
+from piccolo.table import Table
+from piccolo.utils.pydantic import create_pydantic_model
+
+
+class Band(Table):
+    name = Varchar()
+    # This automatically gets converted into a LazyTableReference, because a
+    # string is passed in:
+    manager = ForeignKey("Manager")
+
+
+# This is not recommended, as it will cause the LazyTableReference to be
+# evaluated before Manager has imported.
+# Instead, move this to a separate file, or below Manager.
+BandModel = create_pydantic_model(Band)
+
+
+class Manager(Table):
+    name = Varchar()

docs/src/piccolo/tutorials/deployment.rst

@@ -35,7 +35,7 @@ This is a very simple Dockerfile, and illustrates the basics:
 .. code-block:: dockerfile
 
     # Specify the base image:
-    FROM python:3.10-slim-bullseye
+    FROM python:3.12-bookworm
 
     # Install the pip requirements:
     RUN pip install --upgrade pip
@@ -77,3 +77,11 @@ When we run the container (usually via `Kubernetes <https://kubernetes.io/>`_,
 `Docker Compose <https://docs.docker.com/compose/>`_, or similar),
 we can specify the database credentials using environment variables, which will
 be used by our application.
+
+Accessing a local Postgres database
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Bear in mind that if you have Postgres running locally on the server (i.e. on
+``localhost``), your Docker container won't automatically be able to access it.
+You can try Docker's host based networking, or just run Postgres within a
+Docker container.

docs/src/piccolo/tutorials/index.rst

@@ -11,3 +11,4 @@ help you solve common problems:
     ./using_sqlite_and_asyncio_effectively
     ./deployment
     ./fastapi
+    ./avoiding_circular_imports

piccolo/columns/column_types.py

@@ -2183,7 +2183,7 @@ def __getattribute__(self, name: str) -> t.Union[Column, t.Any]:
         # If the ForeignKey is using a lazy reference, we need to set the
         # attributes here. Attributes starting with an underscore are
         # unlikely to be column names.
-        if not name.startswith("__"):
+        if not name.startswith("_") and name not in dir(self):
             try:
                 _foreign_key_meta = object.__getattribute__(
                     self, "_foreign_key_meta"
@@ -2196,12 +2196,9 @@ def __getattribute__(self, name: str) -> t.Union[Column, t.Any]:
                 ):
                     object.__getattribute__(self, "set_proxy_columns")()
 
-        try:
-            value = object.__getattribute__(self, name)
-        except AttributeError:
-            raise AttributeError
+        value = object.__getattribute__(self, name)
 
-        if name == "_":
+        if name.startswith("_"):
             return value
 
         foreignkey_class: t.Type[ForeignKey] = object.__getattribute__(

piccolo/columns/reference.py

@@ -30,6 +30,9 @@ class LazyTableReference:
         If specified, the ``Table`` subclass is imported from this path.
         For example, ``'my_app.tables'``.
 
+        .. hint::
+            If the table is in the same file, you can pass in ``__name__``.
+
     """
 
     table_class_name: str

tests/columns/test_reference.py

@@ -5,10 +5,41 @@
 
 from unittest import TestCase
 
+from piccolo.columns import ForeignKey, Varchar
 from piccolo.columns.reference import LazyTableReference
+from piccolo.table import Table
+from tests.base import TableTest
 
 
-class TestLazyTableReference(TestCase):
+class Band(Table):
+    manager: ForeignKey["Manager"] = ForeignKey(
+        LazyTableReference("Manager", module_path=__name__)
+    )
+    name = Varchar()
+
+
+class Manager(Table):
+    name = Varchar()
+
+
+class TestQueries(TableTest):
+    tables = [Band, Manager]
+
+    def setUp(self):
+        super().setUp()
+        manager = Manager({Manager.name: "Guido"})
+        manager.save().run_sync()
+        band = Band({Band.name: "Pythonistas", Band.manager: manager})
+        band.save().run_sync()
+
+    def test_select(self):
+        self.assertListEqual(
+            Band.select(Band.name, Band.manager._.name).run_sync(),
+            [{"name": "Pythonistas", "manager.name": "Guido"}],
+        )
+
+
+class TestInit(TestCase):
     def test_init(self):
         """
         A ``LazyTableReference`` must be passed either an ``app_name`` or
@@ -34,6 +65,8 @@ def test_init(self):
             module_path="tests.example_apps.music.tables",
         )
 
+
+class TestStr(TestCase):
     def test_str(self):
         self.assertEqual(
             LazyTableReference(