Aborted (core dumped) with tf.raw_ops.CombinedNonMaxSuppression

[LongZE666]

Issue type

Bug

Have you reproduced the bug with TensorFlow Nightly?

Yes

Source

source

TensorFlow version

tf 2.16.1

Custom code

Yes

OS platform and distribution

Ubuntu 20.04

Mobile device

No response

Python version

3.11

Bazel version

No response

GCC/compiler version

No response

CUDA/cuDNN version

No response

GPU model and memory

No response

Current behavior?

If max_total_size is too large , tf.raw_ops.CombinedNonMaxSuppression encounters "Aborted (core dumped)".

Standalone code to reproduce the issue

import tensorflow as tf

pad_per_class = True
clip_boxes = False
boxes = tf.constant(3.5e+35, shape=[], dtype=tf.float32)
scores = tf.constant(0, shape=[], dtype=tf.float32)
max_output_size_per_class = tf.constant(8, shape=[], dtype=tf.int32)
max_total_size = tf.constant(1879048192, shape=[], dtype=tf.int32)
iou_threshold = tf.constant(0.5, shape=[], dtype=tf.float32)
score_threshold = tf.constant(0.5, shape=[], dtype=tf.float32)
tf.raw_ops.CombinedNonMaxSuppression(boxes=boxes, scores=scores, max_output_size_per_class=max_output_size_per_class, max_total_size=max_total_size, iou_threshold=iou_threshold, score_threshold=score_threshold, pad_per_class=pad_per_class, clip_boxes=clip_boxes)

Relevant log output

2024-05-01 02:52:31.372751: W tensorflow/core/kernels/image/non_max_suppression_op.cc:998] Detected a large value for `max_total_size`. This may cause OOM error. (max_total_size: 1879048192)
2024-05-01 02:52:31.372808: F tensorflow/core/framework/tensor_shape.cc:357] Check failed: d < dims() (2 vs. 0)
Aborted (core dumped)

[Venkat6871]

Hi @LongZE666 ,

I tried to run your code on colab using TF v2.16.1, nightly and faced the same issue. Please find the gist here for reference.

Thank you!