Bug? Folder.sync_items() doesn't return some fields of CalendarItem

[suhanoves]

Hi, Erik!
I discovered an interesting behavior: I get unfilled some fields of CalendarItem when I am using Folder.sync_items method.

Steps to Reproduce:

1. Create an Exchange recurring event (e.g. daily)
2. Get a message using a method Folder.sync_items()
3. Get a message using a method Folder.get(id=)

account = Account(...)
calendar = account.calendar
for _, item in calendar.sync_items():
    sync_item = item
get_item = calendar.get(id=sync_item.id)
sync_item == get_item  # True

Expected Result:

• sync_item and get_item are absolutely identical

Actual Result:

• sync_item and get_item have identical id and changekey, but some params are different:

param	sync_item	get_item
adjacent_meeting_count	None	{int}
body	None	{HtmlBody}
conflicting_meeting_count	None	{int}
deleted_occurrences	None	{list} of {DeletedOccurrence}
first_occurrence	None	{FirstOccurrence}
last_occurrence	None	{LastOccurrence}
mime_content	None	{bytes}
modified_occurrences	None	{Occurrence}
recurrence	None	{Recurrence}
response_objects	None	{ResponseObjects}
text_body	None	{str}

I tried to set only specific fields by using only_fields param or changed sync_scope param.
The first step didn't do anything, the attribute values were exactly the same (i.e. are None). Changing sync_scope to "NormalAndAssociatedItems" for some reason even reduced the number of CalendarItem attributes by removing the recurrence attributes :)

I don't understand why the same Item has empty fields.
The body fields and all fields related to recurrence are extremely important to me, but using the sync_itemmethod I can't get them.
Any ideas?

P.S.: I will never tire of thanking you for a really cool library, it taught me a lot!

[ecederstrand]

Thanks for the kind words! I'm glad you ind the library useful and that you're learning from it :-)

A common feature of most of these fields is that they are what EWS calls complex fields. You can see this with e.g. CalendarItem.FIELDS["body"].is_complex. These are fields that either contain large amounts of data, are calculated from other fields, or are more complex data types than booleans, integers, strings etc.

Complex fields are not returned by the FindItem service, and can only be fetched using the GetItem service. It's probable that the SyncFolderItems service only returns non-complex fields. You can check this yourself by enabling debug logging and checking which fields are requested, and which are returned, in the XML going back and forth. If that's the case, calling calendar.get(id=...) is exactly what you need to do to get the remaining fields. item.refresh() would probably achieve the same.

exchangelib could probably hide this fact from the user and do a GetItem call for all items transparently if complex fields are requested explicitly or implicitly, but .sync_items() is usually called in a performance-sensitive context, so I'm a bit hesitant do do that here.

[suhanoves]

Thanks for such a detailed answer!
I think a refresh for each CalendarItem will be monstrous in performance.

It seems to me that at the moment I have found the optimal solution for how to get complex fields with just one additional request:

1. Call .sync_items(only_fields=['id', 'changekey'])
2. Call account.fetch(ids=[(sync_item.id, sync_item.changekey) for sync_item in sync_items])

Am I thinking in the right direction?

[ecederstrand]

You definitely are - bulk operations and only_fields are the way to go to improve performance. If you need additional performance, you can use the only_fields attribute to .fetch() to avoid fetching heavy fields you don't need, for example attachments and mime_content.