Crash during training

[jurukode]

Hi @ganeshjawahar,

i got experienced that the script always got killed after seven iterations automatically. Not sure what's happening. do you have any idea why?

[joeybose]

So in my experience, there is a resource exhaustion error and thats why the process is killed. There seems to be a bug in this code that uses a lot more GPU RAM than the original MemN2N, but I'm not sure where this bug is.

[jurukode]

Hi @0220joey,

yeah, by the way i'm only using CPU right now and the training process eat so many RAM until my laptop got hang. Thanks for the info by the way!

[joeybose]

I've managed to fix the issue, its in all the tf.assign which adds more nodes to the graph. So comment out lines 156-170, 211-225 in model.py. Although, the accuracy is about 10% less than the paper.

[jurukode]

Thanks @0220joey,

it works well and run faster. Wondering if the commented code are vital to increase accuracy or not

[joeybose]

I dont think so, but the paper does a few more tricks that this code doesnt do but I cant be too sure if thats the cause of the accuracy increase.