Add CHASM Visibility registration and task processing

[awln-temporal]

What changed?

Adds CHASM search attribute provider and mapper implementation. To use CHASM search attributes, embed your component with ComponentSearchAttributesProvider and register the search attributes to the CHASM Registry.

To define a component search attribute, define a SearchAttribute as a package/global scoped variable (CHASM search attributes should reference exported variables like SearchAttributeFieldInt01, SearchAttributeFieldDateTime01:

var (
	TestComponentStartTimeSearchAttribute = NewSearchAttributeTime(testComponentStartTimeSAKey, SearchAttributeFieldDateTime01)

	_ VisibilitySearchAttributesProvider = (*TestComponent)(nil)
	_ VisibilityMemoProvider             = (*TestComponent)(nil)
	_ VisibilitySearchAttributesMapper   = (*TestComponent)(nil)
)

To update SearchAttributes during an execution of a CHASM archetype, implement the provider SearchAttributes in the root component, and call the NewValue method on a SearchAttribute within the Provider method to generate a new SearchAttributeValue. On SearchAttribute updates during CHASM transactions, the framework will detect changes and submit a Visibility task for persistence.

type TestComponent struct {
	UnimplementedComponent

	...
	Visibility Field[*Visibility]
}


func (tc *TestComponent) SearchAttributes(ctx Context) []SearchAttributeValue {
	return []SearchAttributeValue{
             TestComponentStartTimeSearchAttribute.NewValue(time)
        }
}

To register the search attribute with the CHASM Registry, WithSearchAttributes must be passed as a RegistrableComponentOption to the root component of the library. This is required to support Visibility queries.

return []*chasm.RegistrableComponent{
		chasm.NewRegistrableComponent[*Scheduler]("scheduler", WIthSearchAttributes([]SearchAttributeDefinition{TestComponentStartTimeSearchAttribute}),
		chasm.NewRegistrableComponent[*Generator]("generator"),
		chasm.NewRegistrableComponent[*Invoker]("invoker"),
		chasm.NewRegistrableComponent[*Backfiller]("backfiller"),
	}

Why?

Required to support CHASM Search Attributes.

How did you test it?

• built
• run locally and tested manually
• covered by existing tests
• added new unit test(s)
• added new functional test(s)

Potential risks

TBD

[bergundy]

Started reviewing but I think we still don't agree on the API, so let's discuss that before and agree before implementing anything.

[bergundy]

This is already very close.
Please document all exported fields from the chasm package.

[bergundy]

Approved with comments. Overall this looks great. Please add the missing docstrings and test coverage before merging.

[bergundy]

Can you use one attribute that is an alias to an existing pre-defined field for completeness? I feel like you're not understanding what I'm asking.

[awln-temporal]

apologies, there was a reference to use a "predefined" field (not TemporalInt01 etc) in test_component_test.go. I also added a reference to the predefined field SearchAttributeTemporalScheduledByID.

As for whether to include it in Payload.go, there was a previous TODO to track Count and Size as search attributes, and I wasn't sure that adding a reference to TemporalScheduledById made sense within the scope of the Payload component. This component only tracks the current count and size of payloads, and a map from the PayloadKey to its Payload blob. What would be the values provided to the search attribute?

I agree that functional tests should cover this case, I just wasn't sure if it was acceptable style to add a search attribute that doesn't seem related to the component. Added a reference in payload.go to this.

[rodrigozhou]

I suppose you can't just skip the CHASM search attributes based on this comment.

[awln-temporal]

This is temporary, we won't skip CHASM search attributes once we consolidate the type maps on the query path.

[rodrigozhou]

Ditto, I think it's better that you check if it's a CHASM search attribute before checking the type. Also, in this code, if it's a CHASM search attribute, saType will be UNSPECIFIED, so you'd be setting the wrong type for CHASM search attributes.

[awln-temporal]

If UNSPECIFIED, SetMetadataType will return early. To be clear, this is just temp code that will need to change when the query path is implemented.

func SetMetadataType(p *commonpb.Payload, t enumspb.IndexedValueType) {
	if t == enumspb.INDEXED_VALUE_TYPE_UNSPECIFIED {
		return
	}

	_, isValidT := enumspb.IndexedValueType_name[int32(t)]
	if !isValidT {
		panic(fmt.Sprintf("unknown index value type %v", t))
	}
	p.Metadata[MetadataType] = []byte(enumspb.IndexedValueType(t).String())
}

[rodrigozhou]

This is in the write path, not query path. What I mean is this code will always set the type for CHASM search attribute as UNSPECIFIED, and as you pointed out, the function will just ignore, ie., the CHASM search attributes will never have its type set in the metadata.
Or did I miss something?

[awln-temporal]

Encode is on the read path. I suppose either way, the MetadataType is somewhat ignored on the write path, and we basically reconstruct the MetadataType solely from the saTypeMap right now? Either way, without merging the saTypeMap with the CHASM component's saTypeMap, current functionality won't work. On the next PR to support querying, we can re-evaluate how we want to support setting the MetadataType. This portion of code is mainly to get functional tests to succeed.

[rodrigozhou]

ditto

Suggested change

	// If the search attribute name is not in typeMap but the payload has type metadata,
	// we can still decode it (e.g., for CHASM search attributes).
	if _, hasTypeMetadata := saPayload.Metadata[MetadataType]; !hasTypeMetadata {
	lastErr = err
	}
	// If the search attribute name is not in typeMap but the payload has type metadata,
	// we can still decode it (e.g., for CHASM search attributes).
	saType = saPayload.Metadata[MetadataType]
	if saType == enumspb.INDEXED_VALUE_TYPE_UNSPECIFIED {
	lastErr = err
	}

[awln-temporal]

this isn't necessary, DecodeValue handles this case

func DecodeValue(
	value *commonpb.Payload,
	t enumspb.IndexedValueType,
	allowList bool,
) (any, error) {
	if t == enumspb.INDEXED_VALUE_TYPE_UNSPECIFIED {
		var err error
		t, err = enumspb.IndexedValueTypeFromString(string(value.Metadata[MetadataType]))
		if err != nil {
			return nil, fmt.Errorf("%w: %v", ErrInvalidType, t)
		}
	}

	switch t {
	case enumspb.INDEXED_VALUE_TYPE_BOOL:
		return decodeValueTyped[bool](value, allowList)
	case enumspb.INDEXED_VALUE_TYPE_DATETIME:
		return decodeValueTyped[time.Time](value, allowList)
	case enumspb.INDEXED_VALUE_TYPE_DOUBLE:
		return decodeValueTyped[float64](value, allowList)
	case enumspb.INDEXED_VALUE_TYPE_INT:
		return decodeValueTyped[int64](value, allowList)
	case enumspb.INDEXED_VALUE_TYPE_KEYWORD:
		return validateStrings(decodeValueTyped[string](value, allowList))
	case enumspb.INDEXED_VALUE_TYPE_TEXT:
		return validateStrings(decodeValueTyped[string](value, allowList))
	case enumspb.INDEXED_VALUE_TYPE_KEYWORD_LIST:
		return validateStrings(decodeValueTyped[[]string](value, false))
	default:
		return nil, fmt.Errorf("%w: %v", ErrInvalidType, t)
	}
}

[rodrigozhou]

Please add unit tests for changes you did.

[awln-temporal]

there are unit test validations for the changes. Previously, key2 would have failed decoding due it missing in the saTypeMap