Martin Bode on 16 Sep 2024 06:46:44
ADF already has a feature for retrying pipeline activities on error.
The problem that we face here is that a retry only makes sense if there are transient (mostly network) problems.
But if there is an error with the data, we can't simply retry the activity since we need to fix the data first.
That is why I think it would be very helpful if we could define a condition for a retry.
Practically I am thinking about defining an expression that returns a Boolean (https://learn.microsoft.com/en-us/azure/data-factory/control-flow-expression-language-functions).
This way we could analyze the error message and look for error codes or specific keywords "timeout", for deciding if a retry makes sense.
Administrator on 30 Sep 2024 22:17:32
Work in progress