-
Notifications
You must be signed in to change notification settings - Fork 744
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GOBBLIN-2153]added search attributes to Temporal workflows #4052
[GOBBLIN-2153]added search attributes to Temporal workflows #4052
Conversation
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/ddm/util/TemporalWorkFlowUtils.java
Outdated
Show resolved
Hide resolved
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/ddm/util/TemporalWorkFlowUtils.java
Outdated
Show resolved
Hide resolved
...n-temporal/src/test/java/org/apache/gobblin/temporal/ddm/util/TemporalWorkFlowUtilsTest.java
Outdated
Show resolved
Hide resolved
Map<String, Object> convertedAttributes = new HashMap<>(); | ||
|
||
convertedAttributes.putAll(searchAttributes); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems like the intention of this method is to convert the input map Map<String, List<?>>
into a Map<String, Object>
without transforming the list values into single objects. This method seems to be written for setSearchAttributes on ChildWorkflowOptions in AbstractNestingExecWorkflowImpl(which expects Map<String, Object>), we can simplify the implementation by directly using new HashMap<>(searchAttributes)
. This would achieve the same result more efficiently.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
simplified the conversion, however have kept this function as we need validations as new HashMap<>(searchAttributes) throws NPE in case passed parameter is null
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/ddm/util/TemporalWorkFlowUtils.java
Outdated
Show resolved
Hide resolved
} catch (Exception ignored) { | ||
log.error("Exception occured during performing Work", ignored); | ||
return CommitStats.createEmpty(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the rationale for this try catch? Seems dangerous to ignore exceptions during processworkunits step, shouldn't we want it to fail loudly?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Help.loadJobState(workSpec, Help.loadFileSystem(workSpec)) throws a checked exception which needs to be catched, however have reduced the scope of the try catch block to this line
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it throws an exception when loading the jobState (which I presume would lead to an empty jobState) we should just throw an ApplicationException in this scenario, so that we can push the Temporal framework to retry the workflow rather than progressing silently.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
makes sense, addressed
d21c88b
to
3b9ff48
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA
Description
Added flowGroup.flowName and user.to.proxy as SearchAttributes to filter Temporal Flows in the UI
Tests
Commits