-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Consolidated and refactored activity request validations. #8508
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: standalone-activity
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice improvement -- the existing validation code was messy and complex and it's good to encapsulate it.
I have some suggestions for improving this further. Here's the API this PR introduces:
Workflow Activity
validator := activity.NewRequestAttributesValidator(...)
err := validator.ValidateAndAdjustTimeouts(runTimeout)
activityOptions := validator.GetActivityOptions()
// use activityOptions to mutate ScheduleActivityTaskCommandAttributes
Standalone Activity
validator := NewRequestAttributesValidator(...)
err = validator.ValidateAndAdjustTimeouts(durationpb.New(0))
modifiedAttributes, err := validator.ValidateStandaloneActivity()
// use modifiedAttributes to mutate StartActivityExecutionRequest
Two issues with this that I'd like to improve are
- Someone could forget to call
ValidateAndAdjustTimeouts
before callingGetActivityOptions
orValidateStandaloneActivity
- We're mutating the timeouts on the passed-in
req.Options
in the Standalone case; I am not sure this mutation is correct/desirable given that the request is re-used by retries.
My first idea is how about we change the API to something like
type ValidatedActivityRequest struct {
Options *activitypb.ActivityOptions
RequestID string
SearchAttributesUnaliased *commonpb.SearchAttributes
}
func ValidateActivityRequest(
activityID string,
activityType string,
// ...
runTimeout *durationpb.Duration,
standaloneAttrs *StandaloneActivityAttributes,
) (*ValidatedActivityRequest, error)
Then ValidateActivityRequest
will take care of cloning input structs when necessary and adjusting the timeouts, and both code paths can use the returned struct of validated data as needed (e.g. update their own request / task attribute structs).
searchAttributesUnaliased *commonpb.SearchAttributes // Unaliased search attributes after validation | ||
} | ||
|
||
func NewRequestAttributesValidator( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe all public functions should have a docstring.
// ValidateStandaloneActivity validates the standalone activity specific attributes. It will return the modified | ||
// attributes that are not idempotent, letting the caller decide how to handle them. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"idempotent" usually refers to a function. What does it mean for an attribute to be idempotent?
dynamicconfig.DefaultActivityRetryPolicy.Get(h.dc), | ||
dynamicconfig.MaxIDLengthLimit.Get(h.dc)(), | ||
namespaceID, | ||
req.Options, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ValidateAndAdjustTimeouts
is going to mutate the timeouts on the passed-in req.Options
. Is this really what we want to do, given that the request struct is reused during retries?
// Since searchAttributesUnaliased is not idempotent, we need to clone the request so that in case of retries, | ||
// the field is set to the original value. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same q about meaning of idempotent.
and nit... :)
// Since searchAttributesUnaliased is not idempotent, we need to clone the request so that in case of retries, | |
// the field is set to the original value. | |
// Since searchAttributesUnaliased is not idempotent, we need to clone the request so that in the case of retries, | |
// the field is set to the original value. |
}, | ||
) | ||
|
||
err = validator.ValidateAndAdjustTimeouts(durationpb.New(0)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function must always be called before making use of the validator, but a user of this API might forget to call it. I made a suggestion in my review comment.
@dandavison - good questions, and the proto mutation is a thorn that needs improvement. We mutably sanitize the incoming request as that's what the current codebase does. On your concerns:
I'm ok with that, separating out the mutated fields to a new return struct and letting the caller decide what to do. I'd rename
Yea def a concern with the retries. If you look at StartActivityExecution in frontend.go, the request is essentially forwarded directly over to the history handler.go rpc. If we truly want to not affect retries, we should reallly clone the request with mutated fields before forwarding. The input can potentially be big. Based on your experience, do you think cloning the request will cause performance issues? UPDATE: per discussion, we'll clone the whole request if anything needs to be mutated. The safe thing to do for predictable retries. |
// 3. If neither ScheduleToClose nor StartToClose is set, return error | ||
// 4. Ensure all timeouts do not exceed runTimeout if runTimeout is set (>0) | ||
// 5. Ensure HeartbeatTimeout does not exceed StartToClose | ||
func (v *RequestAttributesValidator) ValidateAndAdjustTimeouts(runTimeout *durationpb.Duration) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I need to add input payload size validator
What changed?
Consolidated and refactored activity request validations to the chasm package.
Added additional tests for the validator.
Why?
Existing workflow activities and standalone activities should share the same validation code. Standalone activities also directly process the frontend request and therefore has additional fields to validate compared to embedded activities.
How did you test it?
Potential risks
This refactors the existing stack, but the validation is basically exactly copied over and all relevant tests are passing.