Skip to content

Commit 7af40cb

Browse files
authored
Add some docs and fix github actions
1 parent f634d09 commit 7af40cb

File tree

9 files changed

+274
-76
lines changed

9 files changed

+274
-76
lines changed

.github/ci.yml

Lines changed: 0 additions & 59 deletions
This file was deleted.

.github/workflows/python-package.yml

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -27,19 +27,20 @@ jobs:
2727

2828
- name: Install package
2929
run: |
30-
pip install flake8==3.7.7
31-
pip install mypy==0.720
32-
pip install black==19.10b0
33-
pip install isort==4.3.21
30+
pip install mypy
31+
pip install black
32+
pip install isort
3433
- name: Run pre-checks
3534
run: |
36-
flake8 fastapi_asyncpg --config=setup.cfg
37-
mypy fastapi_asyncpg/ --ignore-missing-imports
35+
mypy fastapi_asyncpg/
3836
isort -c -rc fastapi_asyncpg/
39-
black -l 80 --check --verbose fastapi_asyncpg
37+
black -l 80 --check --verbose fastapi_asyncpg/
4038
# Job to run tests
4139
tests:
4240
runs-on: ubuntu-latest
41+
strategy:
42+
matrix:
43+
python-version: [3.7, 3.8, 3.9]
4344
# Set environment variables
4445
steps:
4546
- name: Checkout the repository
@@ -56,7 +57,7 @@ jobs:
5657
5758
- name: Run tests
5859
run: |
59-
pytest -rfE --cov=fastapi_asyncpg -s --tb=native -v --cov-report xml --cov-append tests
60+
pytest -vs tests/
6061
6162
- name: Upload coverage to Codecov
6263
uses: codecov/codecov-action@v1

CHANGELOG.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
## 1.0.0
2+
3+
- Initial release

README.md

Lines changed: 213 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,217 @@
11
# FastAPI AsyncPG
22

3-
## Authors
3+
FastAPI integration for AsyncPG
4+
5+
## Narrative
6+
7+
First of all, so sorry for my poor english. I will be so happy,
8+
if someone pushes a PR correcting all my english mistakes. Anyway
9+
I will try to do my best.
10+
11+
Looking at fastapi ecosystem seems like everyone is trying to integrate
12+
fastapi with orms, but from my experience working with raw
13+
sql I'm so productive.
14+
15+
If you think a bit around, your real model layer, is the schema on your
16+
db (you can add abastractions on top of it), but what ends
17+
is your data, and these are tables, columns and rows.
18+
19+
Also, sql, it's one of the best things I learned
20+
because it's something that always is there.
21+
22+
On another side, postgresql it's robust and rock solid,
23+
thousands of projects depend on it, and use it as their storage layer.
24+
AsyncPG it's a crazy fast postgresql driver
25+
written from scratch.
26+
27+
FastAPI seems like a clean, and developer productive approach to web
28+
frameworks. It's crazy how well it integrates with OpenAPI,
29+
and how easy makes things to a developer to move on.
30+
31+
## Integration
32+
33+
fastapi_asyncpg trys to integrate fastapi and asyncpg in an idiomatic way.
34+
fastapi_asyncpg when configured exposes two injectable providers to
35+
fastapi path functions, can use:
36+
37+
- `db.connection` : it's just a raw connection picked from the pool,
38+
that it's auto released when pathfunction ends, this is mostly
39+
merit of the DI system around fastapi.
40+
41+
- `db.transaction`: the same, but wraps the pathfuncion on a transaction
42+
this is more or less the same than the `atomic` decorator from Django.
43+
also `db.atomic` it's aliased
44+
45+
```python
46+
from fastapi import FastAPI
47+
from fastapi import Depends
48+
from fastapi_asyncpg import configure_asyncpg
49+
50+
app = FastAPI()
51+
# we need to pass the fastapi app to make use of lifespan asgi events
52+
db = configure_asyncpg(app, "postgresql://postgres:postgres@localhost/db")
53+
54+
@db.on_init
55+
async def initialization(conn):
56+
# you can run your db initialization code here
57+
await conn.execute("SELECT 1")
58+
59+
60+
@app.get("/")
61+
async def get_content(db=Depends(db.connection)):
62+
rows = await db.fetch("SELECT wathever FROM tablexxx")
63+
return [dict(r) for r in rows]
64+
65+
@app.post("/")
66+
async def mutate_something_compled(db=Depends(db.atomic))
67+
await db.execute()
68+
await db.execute()
69+
# if something fails, everyting is rolleback, you know all or nothing
70+
```
71+
72+
And there's also an `initialization` callable on the main factory function.
73+
That can be used like in flask to initialize whatever you need on the db.
74+
The `initialization` is called right after asyncpg stablishes a connection,
75+
and before the app fully boots. (Some projects use this as a poor migration
76+
runner, not the best practice if you are deploying multiple
77+
instances of the app).
78+
79+
## Testing
80+
81+
For testing we use [pytest-docker-fixtures](https://pypi.org/project/pytest-docker-fixtures/), it requires docker on the host machine or on whatever CI you use
82+
(seems like works as expected with github actions)
83+
84+
It works, creating a container for the session and exposing it as pytest fixture.
85+
It's a good practice to run tests with a real database, and
86+
pytest-docker-fixtures make it's so easy. As a bonus, all fixtures run on a CI.
87+
We use Jenkins witht docker and docker, but also seems like travis and github actions
88+
also work.
89+
90+
The fixture needs to be added to the pytest plugins `conftest.py` file.
91+
92+
on conftest.py
93+
94+
```python
95+
pytest_plugins = [
96+
"pytest_docker_fixtures",
97+
]
98+
```
99+
100+
With this in place, we can just yield a pg fixture
101+
102+
```python
103+
from pytest_docker_fixtures import images
104+
105+
# image params can be configured from here
106+
images.configure(
107+
"postgresql", "postgres", "11.1", env={"POSTGRES_DB": "test_db"}
108+
)
109+
110+
# and then on our test we have a pg container running
111+
# ready to recreate our db
112+
async def test_pg(pg):
113+
host, port = pg
114+
dsn = f"postgresql://postgres@{host}:{port}/test_db"
115+
await asyncpg.Connect(dsn=dsn)
116+
# let's go
117+
118+
```
119+
120+
With this in place, we can just create our own pytest.fixture that
121+
_patches_ the app dsn to make it work with our custom created
122+
container.
123+
124+
````python
125+
126+
from .app import app, db
127+
from async_asgi_testclient import TestClient
128+
129+
import pytest
130+
131+
pytestmark = pytest.mark.asyncio
132+
133+
@pytest.fixture
134+
async def asgi_app(pg)
135+
host, port = pg
136+
dsn = f"postgresql://postgres@{host}:{port}/test_db"
137+
# here we patch the dsn for the db
138+
# con_opts: are also accessible
139+
db.dsn = dsn
140+
yield app, db
141+
142+
async def test_something(asgi_app):
143+
app, db = asgi_app
144+
async with db.pool.acquire() as db:
145+
# setup your test state
146+
147+
# this context manager handlers lifespan events
148+
async with TestClient(app) as client:
149+
res = await client.request("/")
150+
```
151+
152+
Anyway if the application will grow, to multiples subpackages,
153+
and apps, we trend to build the main app as a factory, that
154+
creates it, something like:
155+
156+
```python
157+
from fastapi_asyncpg import configure_asyncpg
158+
from apppackage import settings
159+
160+
import venusian
161+
162+
def make_asgi_app(settings):
163+
app = FastAPI()
164+
db = configure_asyncpg(settings.DSN)
165+
166+
scanner = venusian.Scanner(app=app)
167+
venusian.scan(theapp)
168+
return app
169+
````
170+
171+
Then on the fixture, we just need, to factorze and app from our function
172+
173+
```python
174+
175+
from .factory import make_asgi_app
176+
from async_asgi_testclient import TestClient
177+
178+
import pytest
179+
180+
pytestmark = pytest.mark.asyncio
181+
182+
@pytest.fixture
183+
async def asgi_app(pg)
184+
host, port = pg
185+
dsn = f"postgresql://postgres@{host}:{port}/test_db"
186+
app = make_asgi_app({"dsn": dsn})
187+
# ther's a pointer on the pool into app.state
188+
yield app
189+
190+
async def test_something(asgi_app):
191+
app = asgi_app
192+
pool = app.state.pool
193+
async with db.pool.acquire() as db:
194+
# setup your test state
195+
196+
# this context manager handlers lifespan events
197+
async with TestClient(app) as client:
198+
res = await client.request("/")
199+
200+
```
201+
202+
There's also another approach exposed and used on [tests](tests/test_db.py),
203+
that exposes a single connection to the test and rolls back changes on end.
204+
We use this approach on a large project (500 tables per schema and
205+
multiples schemas), and seems like it speeds up a bit test creation.
206+
This approach is what [Databases](https://www.encode.io/databases/) it's using.
207+
Feel free to follow the tests to see if it feets better.
208+
209+
## Extras
210+
211+
There are some utility functions I daily use with asyncpg that helps me
212+
speed up some sql operations like, they are all on sql.py, and mostly are
213+
self documented. They are in use on tests.
214+
215+
### Authors
4216

5217
`fastapi_asyncpg` was written by `Jordi collell <jordic@gmail.com>`\_.

fastapi_asyncpg/__init__.py

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,19 @@ def __init__(
2020
pool=None, # usable on testing
2121
**options,
2222
):
23+
"""This is the entry point to configure an asyncpg pool with fastapi.
24+
25+
Arguments
26+
app: The fastapp application that we use to store the pool
27+
and bind to it's initialitzation events
28+
dsn: A postgresql desn like postgresql://user:password@postgresql:5432/db
29+
init_db: Optional callable that receives a db connection,
30+
for doing an initialitzation of it
31+
pool: This is used for testing to skip the pool initialitzation
32+
an just use the SingleConnectionTestingPool
33+
**options: connection options to directly pass to asyncpg driver
34+
see: https://magicstack.github.io/asyncpg/current/api/index.html#connection-pools
35+
"""
2336
self.app = app
2437
self.dsn = dsn
2538
self.init_db = init_db
@@ -29,6 +42,8 @@ def __init__(
2942
self.app.router.add_event_handler("shutdown", self.on_disconnect)
3043

3144
async def on_connect(self):
45+
"""handler called during initialitzation of asgi app, that connects to
46+
the db"""
3247
if self._pool:
3348
self.app.state.pool = self._pool
3449
return
@@ -49,16 +64,35 @@ def pool(self):
4964
return self.app.state.pool
5065

5166
async def connection(self):
67+
"""
68+
A ready to use connection Dependency just usable
69+
on your path functions that gets a connection from the pool
70+
Example:
71+
db = configure_asyncpg(app, "dsn://")
72+
@app.get("/")
73+
async def get_content(db = Depens(db.connection)):
74+
await db.fetch("SELECT * from pg_schemas")
75+
"""
5276
async with self.pool.acquire() as db:
5377
yield db
5478

5579
async def transaction(self):
80+
"""
81+
A ready to use transaction Dependecy just usable on a path function
82+
Example:
83+
db = configure_asyncpg(app, "dsn://")
84+
@app.get("/")
85+
async def get_content(db = Depens(db.transaction)):
86+
await db.execute("insert into keys values (1, 2)")
87+
await db.execute("insert into keys values (1, 2)")
88+
All view function executed, are wrapped inside a postgresql transaction
89+
"""
5690
async with self.pool.acquire() as db:
5791
txn = db.transaction()
5892
await txn.start()
5993
try:
6094
yield db
61-
except:
95+
except: # noqa
6296
await txn.rollback()
6397
raise
6498
else:

0 commit comments

Comments
 (0)