Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graph disconected: custom model with two inputs #48

Open
SergioG-M opened this issue Jan 26, 2021 · 2 comments
Open

Graph disconected: custom model with two inputs #48

SergioG-M opened this issue Jan 26, 2021 · 2 comments
Assignees

Comments

@SergioG-M
Copy link

Hi, I'm getting an error when I try to get the grad-cam visualizations for a custom model with two inputs. A minimal working example follows

img_width=224
img_height=224

model = EfficientNetB0(include_top=False, input_shape=(img_width, img_height, 3),
                   weights='imagenet', drop_connect_rate=0.2)

input1 = layers.Input(shape=(img_width, img_height, 3))
input2 = layers.Input(shape=(img_width, img_height, 3))

output1 = model(input1)
output2 = model(input2)

conc = layers.Concatenate(axis=-1, name='Concat_features')([output1, output2])
# Rebuild last layer
x = layers.GlobalAveragePooling2D(name="avg_pool")(conc)
x = layers.BatchNormalization()(x)

top_dropout_rate = 0.2
x = layers.Dropout(top_dropout_rate, name="top_dropout")(x)
outputs = layers.Dense(self.num_classes, activation="softmax", name="predictions")(x)

dual_model = Model([input1, input2], outputs)
optimizer = optimizers.Adam(learning_rate=1e-4)

dual_model.compile(optimizer=optimizer, loss="categorical_crossentropy",
              metrics=["accuracy"])    

Then when I try to call to GradCam I get:

ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_1:0", shape=(None, 224, 224, 3), dtype=float32) at layer "rescaling". The following previous layers were accessed without issue: []

Any idea how can i solve this?

@keisen keisen self-assigned this Jan 27, 2021
@keisen
Copy link
Owner

keisen commented Jan 28, 2021

Hi, @SergioG-M , Sorry for late reply.

The cause of the problem is that your model is cascaded, that is, the model includes EfficientNet models.
When tf.keras.Model is used as Layer, Tensorflow Graph is disconnected between the model of top level and included model.
But GradCAM (and other viualization methods) need the graph that continuously connect from the input layer to the output layer of the model.

So you need to slightly devise to build such model.
Please modify the code below ...

model = EfficientNetB0(include_top=False, input_shape=(img_width, img_height, 3),
                   weights='imagenet', drop_connect_rate=0.2)

input1 = layers.Input(shape=(img_width, img_height, 3))
input2 = layers.Input(shape=(img_width, img_height, 3))

output1 = model(input1)
output2 = model(input2)

... to like below:

input1 = layers.Input(shape=(img_width, img_height, 3))
input2 = layers.Input(shape=(img_width, img_height, 3))

model1 = EfficientNetB0(include_top=False, input_tensor=input1,
                        weights='imagenet', drop_connect_rate=0.2)
model2 = EfficientNetB0(include_top=False, input_tensor=input2,
                        weights='imagenet', drop_connect_rate=0.2)

output1 = model2.output
output2 = model2.output

Then, please try to GradCAM!

Thanks!

@nadezhda95
Copy link

nadezhda95 commented Sep 11, 2023

Hi! I also used transfer learning and your suggestion in the previous comment helped me to solve the issue.
But I noticed, that I get the same error of disconnection when I use data augmentation layers

def build_model():

    input_shape = (300, 150, 3)
    inputs = tf.keras.layers.Input(shape=input_shape)
    inputs = tf.keras.layers.RandomRotation(0.15)(inputs)
    inputs = tf.keras.layers.RandomFlip()(inputs)
    inputs = tf.keras.layers.RandomZoom(0.15)(inputs)

    global_average_layer = tf.keras.layers.GlobalAveragePooling2D()
    prediction_layer = tf.keras.layers.Dense(1, activation='sigmoid')
    base_model = tf.keras.applications.EfficientNetV2S(input_shape=input_shape,
                                                        include_top=False,
                                                        weights='imagenet',
                                                        input_tensor=inputs)
    base_model.trainable = False

    bm_output = base_model.output
        
    x = tf.keras.layers.Dense(units=512, activation='relu', use_bias=False)(bm_output)
    x = tf.keras.layers.BatchNormalization()(x)
    x = tf.keras.layers.Dropout(0.5)(x)
        
    x = global_average_layer(x)
    
    outputs = prediction_layer(x)
    
    model = keras.Model(inputs, outputs)
    model.compile(
        optimizer=tf.keras.optimizers.Adam(),
        loss=tf.keras.losses.BinaryCrossentropy(),
        metrics='accuracy'
    )

    return model



augmentation_model = Sequential(
    [
        tf.keras.layers.RandomRotation(factor=0.15),
        tf.keras.layers.RandomFlip(),
        tf.keras.layers.RandomZoom(0.15)
    ]
)

The function builds model, but layerCam doesn't work if I apply data processing layers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants