Skip to content

feat: Rdo 2 all errors to file sd ks #251

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 29 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
671d5f8
"added to the readme to have a quickstart section"
John-Memphis Dec 21, 2023
54b893d
Update README.md, spelling mistake
John-Memphis Dec 21, 2023
324660d
Update README.md, added import for type hinting
John-Memphis Dec 21, 2023
1405f7b
"Removed extra indenting"
John-Memphis Dec 21, 2023
02049f3
"Added comment on asyncio"
John-Memphis Dec 21, 2023
179fd4a
"Forgot ()"
John-Memphis Dec 21, 2023
6ca0378
"Changed to use memphis.produce and memphis.fetch_messages in order t…
John-Memphis Dec 21, 2023
127971a
Merge branch 'master' into RDO-6-1-liner-sdk-examples
John-Memphis Dec 27, 2023
739778b
Added close reminder
John-Memphis Dec 28, 2023
44f4991
"Updated examples for python"
John-Memphis Dec 29, 2023
8d067ca
Merge branch 'RDO-6-1-liner-sdk-examples' of github.com-memphis:memph…
John-Memphis Dec 29, 2023
cb00c9f
"First iteration of errors to a file"
John-Memphis Dec 29, 2023
9b35f1d
"touched up the formatting a little bit"
John-Memphis Dec 30, 2023
db5dbad
"fixed all linting/formatting issues"
John-Memphis Dec 30, 2023
1e10c8b
"Changed params to generic params"
John-Memphis Jan 2, 2024
0013137
"Made examples use generic names"
John-Memphis Jan 4, 2024
ff857ef
"Added while true loop"
John-Memphis Jan 9, 2024
7099b3a
Merge branch 'master' into RDO-6-1-liner-sdk-examples
John-Memphis Jan 9, 2024
a611076
"Added comment on accountID so build completes"
John-Memphis Jan 13, 2024
4b9f5b2
"removed unsused imports"
John-Memphis Jan 13, 2024
9197b9c
Merge branch 'master' into RDO-2-all-errors-to-file-sd-ks
John-Memphis Jan 15, 2024
6ab029b
removed circular import issues I think
John-Memphis Jan 16, 2024
cd62c3d
"Messed with formatting"
John-Memphis Jan 16, 2024
2efa6a8
Merge branch 'RDO-6-1-liner-sdk-examples' into RDO-2-all-errors-to-fi…
John-Memphis Jan 16, 2024
4b3baf6
"just working on formatting..."
John-Memphis Jan 16, 2024
9c103ed
"Added staticclass decorators"
John-Memphis Jan 16, 2024
0919ad0
"more linting fixes"
John-Memphis Jan 16, 2024
9f66e04
"Changed memphis-accountID to memphis-account-id
John-Memphis Jan 16, 2024
b38cc4d
"changed some error text"
John-Memphis Jan 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,68 @@ from memphis.types import Retention, Storage
import asyncio
```

### Quickstart - Producing and Consuming

The most basic functionaly of memphis is the ability to produce messages to a station and to consume those messages.

> The Memphis.py SDK uses asyncio for many functions. Make sure to call the following code in an async function:

```python
async def main():
...

if __name__ == '__main__':
asyncio.run(main())
```

First, a connection to Memphis must be made:

```python
from memphis import Memphis

# Connecting to the broker
memphis = Memphis()

await memphis.connect(
host = "<memphis-host>",
username = "<memphis-username>",
password = "<memphis-password>",
account_id = <mmephis-account-id> # For cloud users, at the top of the overview page
)
```

Then, to produce a message, call the `memphis.produce` function or create a producer and call its `producer.produce` function:

```python
await memphis.produce(
station_name="<station-name>",
producer_name="<producer-name>",
message={
"id": i,
"chocolates_to_eat": 3
}
)
```

Lastly, to consume this message, call the `memphis.fetch_messages` function or create a consumer and call its `consumer.fetch` function:

```python
from memphis.message import Message

messages: list[Message] = await memphis.fetch_messages(
station_name="<station-name>",
consumer_name="<consumer-name>",
) # Type-hint the return here for LSP integration

for consumed_message in messages:
msg_data = json.loads(consumed_message.get_data())
print(f"Ate {msg_data['chocolates_to_eat']} chocolates... Yum")

await consumed_message.ack()
```

> Remember to call `memphis.close()` to close the connection.

### Connecting to Memphis

First, we need to create Memphis `object` and then connect with Memphis by using `memphis.connect`.
Expand Down
55 changes: 31 additions & 24 deletions examples/consumer.py
Original file line number Diff line number Diff line change
@@ -1,46 +1,53 @@
from __future__ import annotations
"""
An example consumer for the Memphis.dev python SDK.
"""

import asyncio

from memphis import Memphis, MemphisConnectError, MemphisError, MemphisHeaderError
import json
from memphis import Memphis
from memphis.message import Message


async def main():
async def msg_handler(msgs, error, _):
try:
for msg in msgs:
print("message: ", msg.get_data())
await msg.ack()
if error:
print(error)
except (MemphisError, MemphisConnectError, MemphisHeaderError) as e:
print(e)
return

"""
Async main function used for the asyncio runtime.
"""
try:
# Connecting to the broker
memphis = Memphis()

await memphis.connect(
host="<memphis-host>",
username="<application type username>",
connection_token="<broker-token>",
username="<memphis-username>",
password="<memphis-password>",
# account_id=<memphis-account-id>, # For cloud users on, at the top of the overview page
)

consumer = await memphis.consumer(
station_name="<station-name>",
consumer_name="<consumer-name>",
consumer_group="",
)

consumer.set_context({"key": "value"})
consumer.consume(msg_handler)
# Keep your main thread alive so the consumer will keep receiving data
await asyncio.Event().wait()
while True:
messages: list[
Message
] = await consumer.fetch() # Type-hint the return here for LSP integration

except (MemphisError, MemphisConnectError) as e:
print(e)
if len(messages) == 0:
continue

for consumed_message in messages:
msg_data = json.loads(consumed_message.get_data())

# Do something with the message data
print(msg_data)
await consumed_message.ack()

except Exception as e:
print(e)
finally:
await memphis.close()
if memphis != None:
await memphis.close()


if __name__ == "__main__":
Expand Down
49 changes: 21 additions & 28 deletions examples/producer.py
Original file line number Diff line number Diff line change
@@ -1,45 +1,38 @@
from __future__ import annotations
"""
An example producer for the Memphis.dev python SDK.
"""

import asyncio

from memphis import (
Headers,
Memphis,
MemphisConnectError,
MemphisError,
MemphisHeaderError,
MemphisSchemaError,
)
from memphis import Memphis, MemphisConnectError, MemphisError


async def main():
"""
Async main function used for the asyncio runtime.
"""
try:
# Connecting to the broker
memphis = Memphis()

await memphis.connect(
host="<memphis-host>",
username="<application type username>",
connection_token="<broker-token>",
username="<memphis-username>",
password="<memphis-password>",
# account_id=<memphis-account-id>, # For cloud users on, at the top of the overview page
)

# Creating a producer and producing a message.
# You can also use the memphis.producer function
producer = await memphis.producer(
station_name="<station-name>", producer_name="<producer-name>"
station_name="<station-name>", # Matches the station name in memphis cloud
producer_name="<producer-name>",
)
headers = Headers()
headers.add("key", "value")
for i in range(5):
await producer.produce(
bytearray("Message #" + str(i) + ": Hello world", "utf-8"),
headers=headers,
) # you can send the message parameter as dict as well

except (
MemphisError,
MemphisConnectError,
MemphisHeaderError,
MemphisSchemaError,
) as e:
print(e)

for i in range(10):
await producer.produce(message={"id": i, "chocolates_to_eat": 3})

except (MemphisError, MemphisConnectError) as e:
print(e)
finally:
await memphis.close()

Expand Down
Loading