Prevent none value in gradients when some of the inputs have not impact to the target

[badcount]

This is to fix the gradients which could have none values. There are scenarios that some of the inputs have not impact to the target value, IG generates None type grads, something like [ [gradient 1], [gradient 2], None, None], and then when calculating the sum in the _calculate_sum_int method, "grads = tf.concat(batches[j], 0)" line throws errors like "None type can't convert to tensor".

Here is the full stack of errors:

An error was encountered:
Attempt to convert a value (None) with an unsupported type (<class 'NoneType'>) to a Tensor.
Traceback (most recent call last):
File "/tmp/5948054080360197755", line 229, in execute
exec(code, global_dict)
File "", line 8, in
baselines=None)
File "/usr/local/lib/python3.7/dist-packages/alibi/explainers/integrated_gradients.py", line 828, in explain
attribute_to_layer_inputs)
File "/usr/local/lib/python3.7/dist-packages/alibi/explainers/integrated_gradients.py", line 1069, in _compute_attributions_list_input
step_sizes, j)
File "/usr/local/lib/python3.7/dist-packages/alibi/explainers/integrated_gradients.py", line 614, in _calculate_sum_int
grads = tf.concat(batches[j], 0)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py", line 206, in wrapper
return target(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/array_ops.py", line 1769, in concat
return gen_array_ops.concat_v2(values=values, axis=axis, name=name)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 1218, in concat_v2
values, axis, name=name, ctx=_ctx)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 1248, in concat_v2_eager_fallback
_attr_T, values = _execute.args_to_matching_eager(list(values), ctx, [])
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py", line 274, in args_to_matching_eager
t, dtype, preferred_dtype=default_dtype, ctx=ctx)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/profiler/trace.py", line 163, in wrapped
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 1566, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/constant_op.py", line 346, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/constant_op.py", line 272, in constant
allow_broadcast=True)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/constant_op.py", line 283, in _constant_impl
return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/constant_op.py", line 308, in _constant_eager_impl
t = convert_to_eager_tensor(value, ctx, dtype)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/constant_op.py", line 106, in convert_to_eager_tensor
return ops.EagerTensor(value, ctx.device_name, dtype)
ValueError: Attempt to convert a value (None) with an unsupported type (<class 'NoneType'>) to a Tensor.

[HughChen]

Checking on the rationale for setting None gradients to zero and checking if it is needed for a non-list input x.

[HughChen]

If our input x is not a list, I think tape.gradient may directly output the gradient for x, in which case we may not want to have this enumerate step which seems to assume that the grads is a list of gradient tensors (one for each input).

[HughChen]

And actually, if our input x isn't a list, would we encounter the None gradients?

It seems like this it primarily comes up for us because we have outputs y1, y2, y3 which depend on different subsets of inputs x1, x2, x3. If y1 only depends on x1, then if we try to explain the model, we can run into issues because the gradients for x2 and x3 will be none.

But if the input isn't a list and is just x, then it seems like every output would need to depend on the whole input tensor?

[HughChen]

If that is the case, maybe we can only do this gradient zero-ing for when x is a list? Something like:

if isinstance(x, list):
    for idx, grad in enumerate(grads):
        if grad is None:
            grads[idx] = tf.convert_to_tensor(np.zeros(shape), dtype=x[idx].dtype)

[HughChen]

I think in an earlier commit you had x[idx].shape, which seems to make more sense in case each input has a different shape.

[HughChen]

Slight nit: Maybe "If certain inputs don't impact the target, the gradient is None, but we need to return a tensor"

[badcount]

Thanks so much for the input @HughChen , updated the PR

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}