Skip to content

Commit eb3f856

Browse files
xSAVIKxhoeslerpre-commit-ci[bot]h4l
authored
V2 sync histories (#273)
* fix kafka unmarshaller args typing and defaults (#240) * fix kafka unmarshaller args typing and defaults Signed-off-by: Christoph Hösler <christoph.hoesler@inovex.de> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Christoph Hösler <christoph.hoesler@inovex.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * chore: release 1.11.1 (#241) Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> * chpre: disable attestations while we're not using trusted publishing (#243) Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> * Improve public API type annotations & fix unit test type errors (#248) * chore: improve typing of functions returning AnyCloudEvent kafka.conversion.from_binary() and from_structured() return AnyCloudEvent type var according to their event_type argument, but when event_type is None, type checkers cannot infer the return type. We now use an overload to declare that the return type is http.CloudEvent when event_type is None. Previously users had to explicitly annotate this type when calling without event_type. This happens quite a lot in this repo's test_kafka_conversions.py — this fixes quite a few type errors like: > error: Need type annotation for "result" [var-annotated] Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk> * chore: type v1.Event chainable Set*() methods The v1.Event self-returning Set*() methods like SetData() were returning BaseEvent, which doesn't declare the same Set* methods. As a result, chaining more than one Set* method would make the return type unknown. This was causing type errors in test_event_pipeline.py. The Set*() methods now return the Self type. Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk> * chore: fix type errors in tests mypy was failing with lots of type errors in test modules. I've not annotated all fixtures, mostly fixed existing type errors. Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk> * chore: allow non-dict headers types in from_http() from_http() conversion function was requiring its headers argument to be a typing.Dict, which makes it incompatible with headers types of http libraries, which support features like multiple values per key. typing.Mapping and even _typeshed.SupportsItems do not cover these types. For example, samples/http-image-cloudevents/image_sample_server.py was failing to type check where it calls `from_http(request.headers, ...)`. To support these kind of headers types in from_http(), we now define our own SupportsDuplicateItems protocol, which is broader than _typeshed.SupportsItems. I've only applied this to from_http(), as typing.Mapping is OK for most other methods that accept dict-like objects, and using this more lenient interface everywhere would impose restrictions on our implementation, even though it might be more flexible for users. Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk> * build: run mypy via tox Tox now runs mypy on cloudevents itself, and the samples. Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk> * build(ci): run mypy in CI alongside linting Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk> * chore: fix minor mypy type complaint in samples Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk> * feat: use Mapping, not Dict for input arguments Mapping imposes less restrictions on callers, because it's read-only and allows non-dict types to be passed without copying them as dict(), or passing dict-like values and ignoring the resulting type error. Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk> * chore: fix tests on py3.8 Tests were failing because the sanic dependency dropped support for py3.8 in its current release. sanic is now pinned to the last compatible version for py3.8 only. Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk> * feat: support new model_validate_json() kwargs Pydantic added by_alias and by_name keyword arguments to BaseModel.model_validate_json in 2.11.1: pydantic/pydantic@acb0f10 This caused mypy to report that that the Pydantic v2 CloudEvent did not override model_validate_json() correctly. Our override now accepts these newly-added arguments. They have no effect, as the implementation does not use Pydantic to validate the JSON, but we also don't use field aliases, so the only effect they could have in the superclass would be to raise an error if they're both False. Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk> * chore: accept Mapping as well as SupportsDuplicateItems Although our types.SupportsDuplicateItems type is wider than Dict and Mapping, it's not a familar type to users, so explicitly accepting Mapping in the from_http() functions should make it more clear to users that a dict-like object is required for the headers argument. Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk> * chore: constrain deps to maintain py 3.8 support Python 3.8 is unsupported and dependencies (such as pydantic) are now shipping releases that fail to type check with mypy running in 3.8 compatibility mode. We run mypy in py 3.8 compatibility mode, so the mypy tox environments must only use deps that support 3.8. And unit tests run by py 3.8 must only use deps that support 3.8. To constrain the deps for 3.8 support, we use two constraint files, one for general environments that only constrains the dependencies that python 3.8 interpreters use, and another for mypy that constraints the dependencies that all interpreters use. Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk> --------- Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk> * Drop EOL Python 3.8 support (#249) * chore: add missing changelog items Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> * chore: drop Python 3.8 support Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> * chore: add a changelog item on Python 3.8 removal Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> * chore: remove mypy-constrains reference as we don't need it anymore Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> * chore: Update pre-commit check versions. Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> * chore: fix isort pre-commit Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * chore: Use Python 3.12 as base version Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> --------- Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * chore: release 1.12.1 CloudEvents v1 goes into maintenance mode (#271) * chore: release 1.12.1 with CloudEvents v2 update and v1 support adjustment Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> * chore: update mypy config to use Python 3.9 Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> * chore: update pytest version range in test requirements Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> * chore: update pytest version range in sample requirements Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> --------- Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> * chore(CHANGELOG): add missing PR link Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> * chore(tox): remove tox configuration file Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> * chore(cloudevents): bump version to 2.0.0-alpha4 Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> * chore(CHANGELOG): update for CloudEvents v2 readiness (PR #273) Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> --------- Signed-off-by: Christoph Hösler <christoph.hoesler@inovex.de> Signed-off-by: Yurii Serhiichuk <savik.ne@gmail.com> Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk> Co-authored-by: Christoph Hösler <christoph.hoesler@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Hal Blackburn <hal.blackburn@gmail.com>
1 parent a35def5 commit eb3f856

File tree

21 files changed

+167
-83
lines changed

21 files changed

+167
-83
lines changed

‎CHANGELOG.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
66

77
## [Unreleased]
88

9+
## [2.0.0.alpha4]
10+
11+
### Changed
12+
13+
- CloudEvents v2 is ready to become the main version. ([#273])
14+
15+
## [1.12.1]
16+
17+
### Changed
18+
19+
- CloudEvents v1 moved to security fixes support stage.
20+
CloudEvents v2 is a rewrite with ongoing development ([#271])
21+
22+
## [1.12.0]
23+
24+
### Changed
25+
26+
- Dropped Python3.8 support while it has reached EOL. ([#249])
27+
28+
## [1.11.1]
29+
30+
### Fixed
31+
- Kafka `conversion` marshaller and unmarshaller typings ([#240])
32+
- Improved public API type annotations and fixed unit test type errors ([#248])
33+
934
## [1.11.0]
1035

1136
### Fixed
@@ -287,3 +312,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
287312
[#232]: https://github.com/cloudevents/sdk-python/pull/232
288313
[#235]: https://github.com/cloudevents/sdk-python/pull/235
289314
[#236]: https://github.com/cloudevents/sdk-python/pull/236
315+
[#240]: https://github.com/cloudevents/sdk-python/pull/240
316+
[#248]: https://github.com/cloudevents/sdk-python/pull/248
317+
[#249]: https://github.com/cloudevents/sdk-python/pull/249
318+
[#271]: https://github.com/cloudevents/sdk-python/pull/271
319+
[#273]: https://github.com/cloudevents/sdk-python/pull/273

‎samples/http-image-cloudevents/client.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@
2828
image_bytes = resp.content
2929

3030

31-
def send_binary_cloud_event(url: str):
31+
def send_binary_cloud_event(url: str) -> None:
3232
# Create cloudevent
3333
attributes = {
3434
"id": "123",
@@ -47,7 +47,7 @@ def send_binary_cloud_event(url: str):
4747
print(f"Sent {event.get_id()} of type {event.get_type()}")
4848

4949

50-
def send_structured_cloud_event(url: str):
50+
def send_structured_cloud_event(url: str) -> None:
5151
# Create cloudevent
5252
attributes = {
5353
"id": "123",

‎src/cloudevents/__init__.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,4 +12,4 @@
1212
# License for the specific language governing permissions and limitations
1313
# under the License.
1414

15-
__version__ = "2.0.0-alpha3"
15+
__version__ = "2.0.0-alpha4"

‎src/cloudevents/v1/__init__.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,4 +12,4 @@
1212
# License for the specific language governing permissions and limitations
1313
# under the License.
1414

15-
__version__ = "1.11.0"
15+
__version__ = "1.12.1"

‎src/cloudevents/v1/abstract/event.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ class CloudEvent:
3232
@classmethod
3333
def create(
3434
cls: typing.Type[AnyCloudEvent],
35-
attributes: typing.Dict[str, typing.Any],
35+
attributes: typing.Mapping[str, typing.Any],
3636
data: typing.Optional[typing.Any],
3737
) -> AnyCloudEvent:
3838
"""

‎src/cloudevents/v1/conversion.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,9 @@ def from_json(
9191

9292
def from_http(
9393
event_type: typing.Type[AnyCloudEvent],
94-
headers: typing.Mapping[str, str],
94+
headers: typing.Union[
95+
typing.Mapping[str, str], types.SupportsDuplicateItems[str, str]
96+
],
9597
data: typing.Optional[typing.Union[str, bytes]],
9698
data_unmarshaller: typing.Optional[types.UnmarshallerType] = None,
9799
) -> AnyCloudEvent:
@@ -260,7 +262,7 @@ def best_effort_encode_attribute_value(value: typing.Any) -> typing.Any:
260262

261263
def from_dict(
262264
event_type: typing.Type[AnyCloudEvent],
263-
event: typing.Dict[str, typing.Any],
265+
event: typing.Mapping[str, typing.Any],
264266
) -> AnyCloudEvent:
265267
"""
266268
Constructs an Event object of a given `event_type` from

‎src/cloudevents/v1/http/conversion.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,9 @@ def from_json(
3737

3838

3939
def from_http(
40-
headers: typing.Dict[str, str],
40+
headers: typing.Union[
41+
typing.Mapping[str, str], types.SupportsDuplicateItems[str, str]
42+
],
4143
data: typing.Optional[typing.Union[str, bytes]],
4244
data_unmarshaller: typing.Optional[types.UnmarshallerType] = None,
4345
) -> CloudEvent:
@@ -58,7 +60,7 @@ def from_http(
5860

5961

6062
def from_dict(
61-
event: typing.Dict[str, typing.Any],
63+
event: typing.Mapping[str, typing.Any],
6264
) -> CloudEvent:
6365
"""
6466
Constructs a CloudEvent from a dict `event` representation.

‎src/cloudevents/v1/http/event.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,11 +34,13 @@ class CloudEvent(abstract.CloudEvent):
3434

3535
@classmethod
3636
def create(
37-
cls, attributes: typing.Dict[str, typing.Any], data: typing.Optional[typing.Any]
37+
cls,
38+
attributes: typing.Mapping[str, typing.Any],
39+
data: typing.Optional[typing.Any],
3840
) -> "CloudEvent":
3941
return cls(attributes, data)
4042

41-
def __init__(self, attributes: typing.Dict[str, str], data: typing.Any = None):
43+
def __init__(self, attributes: typing.Mapping[str, str], data: typing.Any = None):
4244
"""
4345
Event Constructor
4446
:param attributes: a dict with cloudevent attributes. Minimally

‎src/cloudevents/v1/kafka/conversion.py‎

Lines changed: 55 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,14 @@
2121
from cloudevents_v1.kafka.exceptions import KeyMapperError
2222
from cloudevents_v1.sdk import types
2323

24-
DEFAULT_MARSHALLER: types.MarshallerType = json.dumps
25-
DEFAULT_UNMARSHALLER: types.MarshallerType = json.loads
26-
DEFAULT_EMBEDDED_DATA_MARSHALLER: types.MarshallerType = lambda x: x
24+
JSON_MARSHALLER: types.MarshallerType = json.dumps
25+
JSON_UNMARSHALLER: types.UnmarshallerType = json.loads
26+
IDENTITY_MARSHALLER = IDENTITY_UNMARSHALLER = lambda x: x
27+
28+
DEFAULT_MARSHALLER: types.MarshallerType = JSON_MARSHALLER
29+
DEFAULT_UNMARSHALLER: types.UnmarshallerType = JSON_UNMARSHALLER
30+
DEFAULT_EMBEDDED_DATA_MARSHALLER: types.MarshallerType = IDENTITY_MARSHALLER
31+
DEFAULT_EMBEDDED_DATA_UNMARSHALLER: types.UnmarshallerType = IDENTITY_UNMARSHALLER
2732

2833

2934
class KafkaMessage(typing.NamedTuple):
@@ -106,11 +111,29 @@ def to_binary(
106111
return KafkaMessage(headers, message_key, data)
107112

108113

114+
@typing.overload
109115
def from_binary(
110116
message: KafkaMessage,
111-
event_type: typing.Optional[typing.Type[AnyCloudEvent]] = None,
112-
data_unmarshaller: typing.Optional[types.MarshallerType] = None,
117+
event_type: None = None,
118+
data_unmarshaller: typing.Optional[types.UnmarshallerType] = None,
119+
) -> http.CloudEvent:
120+
pass
121+
122+
123+
@typing.overload
124+
def from_binary(
125+
message: KafkaMessage,
126+
event_type: typing.Type[AnyCloudEvent],
127+
data_unmarshaller: typing.Optional[types.UnmarshallerType] = None,
113128
) -> AnyCloudEvent:
129+
pass
130+
131+
132+
def from_binary(
133+
message: KafkaMessage,
134+
event_type: typing.Optional[typing.Type[AnyCloudEvent]] = None,
135+
data_unmarshaller: typing.Optional[types.UnmarshallerType] = None,
136+
) -> typing.Union[http.CloudEvent, AnyCloudEvent]:
114137
"""
115138
Returns a CloudEvent from a KafkaMessage in binary format.
116139
@@ -139,10 +162,11 @@ def from_binary(
139162
raise cloud_exceptions.DataUnmarshallerError(
140163
f"Failed to unmarshall data with error: {type(e).__name__}('{e}')"
141164
)
165+
result: typing.Union[http.CloudEvent, AnyCloudEvent]
142166
if event_type:
143167
result = event_type.create(attributes, data)
144168
else:
145-
result = http.CloudEvent.create(attributes, data) # type: ignore
169+
result = http.CloudEvent.create(attributes, data)
146170
return result
147171

148172

@@ -205,12 +229,32 @@ def to_structured(
205229
return KafkaMessage(headers, message_key, value)
206230

207231

232+
@typing.overload
208233
def from_structured(
209234
message: KafkaMessage,
210-
event_type: typing.Optional[typing.Type[AnyCloudEvent]] = None,
211-
data_unmarshaller: typing.Optional[types.MarshallerType] = None,
235+
event_type: None = None,
236+
data_unmarshaller: typing.Optional[types.UnmarshallerType] = None,
237+
envelope_unmarshaller: typing.Optional[types.UnmarshallerType] = None,
238+
) -> http.CloudEvent:
239+
pass
240+
241+
242+
@typing.overload
243+
def from_structured(
244+
message: KafkaMessage,
245+
event_type: typing.Type[AnyCloudEvent],
246+
data_unmarshaller: typing.Optional[types.UnmarshallerType] = None,
212247
envelope_unmarshaller: typing.Optional[types.UnmarshallerType] = None,
213248
) -> AnyCloudEvent:
249+
pass
250+
251+
252+
def from_structured(
253+
message: KafkaMessage,
254+
event_type: typing.Optional[typing.Type[AnyCloudEvent]] = None,
255+
data_unmarshaller: typing.Optional[types.UnmarshallerType] = None,
256+
envelope_unmarshaller: typing.Optional[types.UnmarshallerType] = None,
257+
) -> typing.Union[http.CloudEvent, AnyCloudEvent]:
214258
"""
215259
Returns a CloudEvent from a KafkaMessage in structured format.
216260
@@ -222,7 +266,7 @@ def from_structured(
222266
:returns: CloudEvent
223267
"""
224268

225-
data_unmarshaller = data_unmarshaller or DEFAULT_EMBEDDED_DATA_MARSHALLER
269+
data_unmarshaller = data_unmarshaller or DEFAULT_EMBEDDED_DATA_UNMARSHALLER
226270
envelope_unmarshaller = envelope_unmarshaller or DEFAULT_UNMARSHALLER
227271
try:
228272
structure = envelope_unmarshaller(message.value)
@@ -259,8 +303,9 @@ def from_structured(
259303
attributes["datacontenttype"] = val.decode()
260304
else:
261305
attributes[header.lower()] = val.decode()
306+
result: typing.Union[AnyCloudEvent, http.CloudEvent]
262307
if event_type:
263308
result = event_type.create(attributes, data)
264309
else:
265-
result = http.CloudEvent.create(attributes, data) # type: ignore
310+
result = http.CloudEvent.create(attributes, data)
266311
return result

‎src/cloudevents/v1/pydantic/v1/conversion.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,9 @@
2121

2222

2323
def from_http(
24-
headers: typing.Dict[str, str],
24+
headers: typing.Union[
25+
typing.Mapping[str, str], types.SupportsDuplicateItems[str, str]
26+
],
2527
data: typing.Optional[typing.AnyStr],
2628
data_unmarshaller: typing.Optional[types.UnmarshallerType] = None,
2729
) -> CloudEvent:
@@ -63,7 +65,7 @@ def from_json(
6365

6466

6567
def from_dict(
66-
event: typing.Dict[str, typing.Any],
68+
event: typing.Mapping[str, typing.Any],
6769
) -> CloudEvent:
6870
"""
6971
Construct an CloudEvent from a dict `event` representation.

0 commit comments

Comments
 (0)