|
1 | 1 | # FastAPI AsyncPG
|
2 | 2 |
|
3 |
| -## Authors |
| 3 | +FastAPI integration for AsyncPG |
| 4 | + |
| 5 | +## Narrative |
| 6 | + |
| 7 | +First of all, so sorry for my poor english. I will be so happy, |
| 8 | +if someone pushes a PR correcting all my english mistakes. Anyway |
| 9 | +I will try to do my best. |
| 10 | + |
| 11 | +Looking at fastapi ecosystem seems like everyone is trying to integrate |
| 12 | +fastapi with orms, but from my experience working with raw |
| 13 | +sql I'm so productive. |
| 14 | + |
| 15 | +If you think a bit around, your real model layer, is the schema on your |
| 16 | +db (you can add abastractions on top of it), but what ends |
| 17 | +is your data, and these are tables, columns and rows. |
| 18 | + |
| 19 | +Also, sql, it's one of the best things I learned |
| 20 | +because it's something that always is there. |
| 21 | + |
| 22 | +On another side, postgresql it's robust and rock solid, |
| 23 | +thousands of projects depend on it, and use it as their storage layer. |
| 24 | +AsyncPG it's a crazy fast postgresql driver |
| 25 | +written from scratch. |
| 26 | + |
| 27 | +FastAPI seems like a clean, and developer productive approach to web |
| 28 | +frameworks. It's crazy how well it integrates with OpenAPI, |
| 29 | +and how easy makes things to a developer to move on. |
| 30 | + |
| 31 | +## Integration |
| 32 | + |
| 33 | +fastapi_asyncpg trys to integrate fastapi and asyncpg in an idiomatic way. |
| 34 | +fastapi_asyncpg when configured exposes two injectable providers to |
| 35 | +fastapi path functions, can use: |
| 36 | + |
| 37 | +- `db.connection` : it's just a raw connection picked from the pool, |
| 38 | + that it's auto released when pathfunction ends, this is mostly |
| 39 | + merit of the DI system around fastapi. |
| 40 | + |
| 41 | +- `db.transaction`: the same, but wraps the pathfuncion on a transaction |
| 42 | + this is more or less the same than the `atomic` decorator from Django. |
| 43 | + also `db.atomic` it's aliased |
| 44 | + |
| 45 | +```python |
| 46 | +from fastapi import FastAPI |
| 47 | +from fastapi import Depends |
| 48 | +from fastapi_asyncpg import configure_asyncpg |
| 49 | + |
| 50 | +app = FastAPI() |
| 51 | +# we need to pass the fastapi app to make use of lifespan asgi events |
| 52 | +db = configure_asyncpg(app, "postgresql://postgres:postgres@localhost/db") |
| 53 | + |
| 54 | +@db.on_init |
| 55 | +async def initialization(conn): |
| 56 | + # you can run your db initialization code here |
| 57 | + await conn.execute("SELECT 1") |
| 58 | + |
| 59 | + |
| 60 | +@app.get("/") |
| 61 | +async def get_content(db=Depends(db.connection)): |
| 62 | + rows = await db.fetch("SELECT wathever FROM tablexxx") |
| 63 | + return [dict(r) for r in rows] |
| 64 | + |
| 65 | +@app.post("/") |
| 66 | +async def mutate_something_compled(db=Depends(db.atomic)) |
| 67 | + await db.execute() |
| 68 | + await db.execute() |
| 69 | + # if something fails, everyting is rolleback, you know all or nothing |
| 70 | +``` |
| 71 | + |
| 72 | +And there's also an `initialization` callable on the main factory function. |
| 73 | +That can be used like in flask to initialize whatever you need on the db. |
| 74 | +The `initialization` is called right after asyncpg stablishes a connection, |
| 75 | +and before the app fully boots. (Some projects use this as a poor migration |
| 76 | +runner, not the best practice if you are deploying multiple |
| 77 | +instances of the app). |
| 78 | + |
| 79 | +## Testing |
| 80 | + |
| 81 | +For testing we use [pytest-docker-fixtures](https://pypi.org/project/pytest-docker-fixtures/), it requires docker on the host machine or on whatever CI you use |
| 82 | +(seems like works as expected with github actions) |
| 83 | + |
| 84 | +It works, creating a container for the session and exposing it as pytest fixture. |
| 85 | +It's a good practice to run tests with a real database, and |
| 86 | +pytest-docker-fixtures make it's so easy. As a bonus, all fixtures run on a CI. |
| 87 | +We use Jenkins witht docker and docker, but also seems like travis and github actions |
| 88 | +also work. |
| 89 | + |
| 90 | +The fixture needs to be added to the pytest plugins `conftest.py` file. |
| 91 | + |
| 92 | +on conftest.py |
| 93 | + |
| 94 | +```python |
| 95 | +pytest_plugins = [ |
| 96 | + "pytest_docker_fixtures", |
| 97 | +] |
| 98 | +``` |
| 99 | + |
| 100 | +With this in place, we can just yield a pg fixture |
| 101 | + |
| 102 | +```python |
| 103 | +from pytest_docker_fixtures import images |
| 104 | + |
| 105 | +# image params can be configured from here |
| 106 | +images.configure( |
| 107 | + "postgresql", "postgres", "11.1", env={"POSTGRES_DB": "test_db"} |
| 108 | +) |
| 109 | + |
| 110 | +# and then on our test we have a pg container running |
| 111 | +# ready to recreate our db |
| 112 | +async def test_pg(pg): |
| 113 | + host, port = pg |
| 114 | + dsn = f"postgresql://postgres@{host}:{port}/test_db" |
| 115 | + await asyncpg.Connect(dsn=dsn) |
| 116 | + # let's go |
| 117 | + |
| 118 | +``` |
| 119 | + |
| 120 | +With this in place, we can just create our own pytest.fixture that |
| 121 | +_patches_ the app dsn to make it work with our custom created |
| 122 | +container. |
| 123 | + |
| 124 | +````python |
| 125 | + |
| 126 | +from .app import app, db |
| 127 | +from async_asgi_testclient import TestClient |
| 128 | + |
| 129 | +import pytest |
| 130 | + |
| 131 | +pytestmark = pytest.mark.asyncio |
| 132 | + |
| 133 | +@pytest.fixture |
| 134 | +async def asgi_app(pg) |
| 135 | + host, port = pg |
| 136 | + dsn = f"postgresql://postgres@{host}:{port}/test_db" |
| 137 | + # here we patch the dsn for the db |
| 138 | + # con_opts: are also accessible |
| 139 | + db.dsn = dsn |
| 140 | + yield app, db |
| 141 | + |
| 142 | +async def test_something(asgi_app): |
| 143 | + app, db = asgi_app |
| 144 | + async with db.pool.acquire() as db: |
| 145 | + # setup your test state |
| 146 | + |
| 147 | + # this context manager handlers lifespan events |
| 148 | + async with TestClient(app) as client: |
| 149 | + res = await client.request("/") |
| 150 | +``` |
| 151 | + |
| 152 | +Anyway if the application will grow, to multiples subpackages, |
| 153 | +and apps, we trend to build the main app as a factory, that |
| 154 | +creates it, something like: |
| 155 | + |
| 156 | +```python |
| 157 | +from fastapi_asyncpg import configure_asyncpg |
| 158 | +from apppackage import settings |
| 159 | + |
| 160 | +import venusian |
| 161 | + |
| 162 | +def make_asgi_app(settings): |
| 163 | + app = FastAPI() |
| 164 | + db = configure_asyncpg(settings.DSN) |
| 165 | + |
| 166 | + scanner = venusian.Scanner(app=app) |
| 167 | + venusian.scan(theapp) |
| 168 | + return app |
| 169 | +```` |
| 170 | + |
| 171 | +Then on the fixture, we just need, to factorze and app from our function |
| 172 | + |
| 173 | +```python |
| 174 | + |
| 175 | +from .factory import make_asgi_app |
| 176 | +from async_asgi_testclient import TestClient |
| 177 | + |
| 178 | +import pytest |
| 179 | + |
| 180 | +pytestmark = pytest.mark.asyncio |
| 181 | + |
| 182 | +@pytest.fixture |
| 183 | +async def asgi_app(pg) |
| 184 | + host, port = pg |
| 185 | + dsn = f"postgresql://postgres@{host}:{port}/test_db" |
| 186 | + app = make_asgi_app({"dsn": dsn}) |
| 187 | + # ther's a pointer on the pool into app.state |
| 188 | + yield app |
| 189 | + |
| 190 | +async def test_something(asgi_app): |
| 191 | + app = asgi_app |
| 192 | + pool = app.state.pool |
| 193 | + async with db.pool.acquire() as db: |
| 194 | + # setup your test state |
| 195 | + |
| 196 | + # this context manager handlers lifespan events |
| 197 | + async with TestClient(app) as client: |
| 198 | + res = await client.request("/") |
| 199 | + |
| 200 | +``` |
| 201 | + |
| 202 | +There's also another approach exposed and used on [tests](tests/test_db.py), |
| 203 | +that exposes a single connection to the test and rolls back changes on end. |
| 204 | +We use this approach on a large project (500 tables per schema and |
| 205 | +multiples schemas), and seems like it speeds up a bit test creation. |
| 206 | +This approach is what [Databases](https://www.encode.io/databases/) it's using. |
| 207 | +Feel free to follow the tests to see if it feets better. |
| 208 | + |
| 209 | +## Extras |
| 210 | + |
| 211 | +There are some utility functions I daily use with asyncpg that helps me |
| 212 | +speed up some sql operations like, they are all on sql.py, and mostly are |
| 213 | +self documented. They are in use on tests. |
| 214 | + |
| 215 | +### Authors |
4 | 216 |
|
5 | 217 | `fastapi_asyncpg` was written by `Jordi collell <jordic@gmail.com>`\_.
|
0 commit comments