State deletion from storage

[raffaeler]

I am not sure whether this use-case will fit with ClearStateAsync.
I have a sort of workflow (represented by a grain) whose activities are also represented as grains. Since the workflow is long-running, each grain storage seems to fit the logic as it can persist its state and continue later on.
When the duty of a workflow activity is finished its state must be absorbed by the previous of following activities. This implies:

• passing (back or forward) the data to another grain
• deleting the grain state entirely

In other words, after the grain has finished its duties, its state is totally useless (and unusable) and should go away.
Once the workflow has finished, the final state will be written elsewhere, so I will need to delete its storage as well.

Is ClearStateAsync sufficient to avoid the storage growing over time?

[r-khanmirzaee]

@raffaeler I also would like to know how we can do it , I found this sample and I guess that is the way to delete state but I haven't tested it myself https://github.com/dotnet/samples/blob/main/orleans/Blazor/BlazorServer/Grains/TodoGrain.cs ( ClearAsync method )

[raffaeler]

@r-khanmirzaee Thanks for entering the conversation.
The general principle "forget about the implementation details" should not apply here as the storaget will never scale.
It certainly works in system where you want to always keep the state forever, but it does not for small lifecycle where you want to ensure the state is saved only during the processing and destroyed at the end.

FYI the MongoDB provider completely clears all the documents when you call ClearStateAsync, but I cannot assume this for all the providers and this opens a not-trivial problem.

[dbinky]

In Orleans 9 AzureBlobStorage provider, the blobs are deleted from storage when you call ClearStateAsync. See here:

orleans/src/Azure/Orleans.Persistence.AzureStorage/Providers/Storage/AzureBlobStorage.cs

Lines 206 to 207 in f2ad524

	await DoOptimisticUpdate(() => blob.DeleteIfExistsAsync(DeleteSnapshotsOption.None, conditions: conditions),
	blob, grainState.ETag).ConfigureAwait(false);

[raffaeler]

Thank you @dbinky. My point is that this behavior should be standardized over all the storage providers.

In certain scenarios you want to delete everything about a Grain. And if the storage provider does not effectively delete all, you will suffer from a growing storage over time, which can lead to very bad situations.

[dbinky]

I agree that there should be some more consistency in the way this is expressed, but your desired behavior is already the default in Orleans 9. I just put in a PR to allow users to explicitly specify the delete on clear behavior in azure blob storage. It defaults to true (so current implementations aren't impacted), but there could be use cases where people may want to preserve the old blob states (to shuffle into Synapse, for example).

[raffaeler]

@dbinky I understand your point, but in this case it would be better exposing two different APIs (methods) to choose the behavior.

[dbinky]

Like... separate implementations for "ClearStateAsync()" vs "ClearStateAndDeleteAsync()"?

[raffaeler]

@dbinky yes, exactly

[dbinky]

That seems like a little on the nose. I'd argue that the implementation in my PR - where you get to specify the delete behavior on a per-provider basis during configuration - is a better way to handle it so that it doesn't impact current implementations. If you updated ClearStateAsync to not delete, current implementations would be impacted and projects would have to be updated to use the new method that deletes. I guess you could have ClearStateAsync() - which still deletes - and then ClearStateButPreserveAsync() that doesn't, but that sounds awkward. I like the options approach better.

[raffaeler]

Probably, the ClearStateAsync should be obsoleted and a new method with a boolean to delete or not the state on the storage:

• [Obsolete] ClearStateAsync: leave the current behavior
• ClearStateAsync(bool deleteFromStorage): new API

This allows all the current API consumers to take a decision on the desired behavior.