Recursive, Graph-based functions definitions

[DirkToewe]

As @doofin pointed out in PR#143, the real power of TF comes into play when it's building blocks are combined using control flow. One of the most powerful means to that end are graph-based (recursive) function definitions.

Thanks to lazy values in Scala, recursive function could be quite elegantly defined in Tensorflow4Scala. It could look something along the lines of:

lazy val Fib = tf.func[Long,Long]{
  (n: Output[Long]) => tf.cond(
    n <= 1,
    1,
    Fib(n-1) + Fib(n-2)
  )
}

// ...

sess.run( FeedMap(), Seq(Fib(13)) )

[eaplatanios]

I don't this this is possible due to limitations in TensorFlow functions. In TF Scala, TF functions are effectively traced the first time they are invoked and given that they do not support recursion, something like the above example would unfortunately not work. I believe there are future plans to add support for recursion in TF functions, but not sure when that might happen.

[DirkToewe]

Are You sure? I believe it was at least possible at some point in the past, see for example this Stackoverflow Post. I believe I had recursion running at one point as well but it must have been many TF version's ago.

[eaplatanios]

Oh that's interesting. I hadn't actually looked into that before. I may take a look if I can find some time, but it is definitely not as simply as just defining a lazy Scala value, because we'd have to modify the function tracing code and add support for forward declarations of TF functions.

[DirkToewe]

Do You mean the implementation or the API? The implementation is not going to be trivial, I realize that. I will try to find out more myself as well.

The TF4S API in the end however could be as simple as the example above, I believe. And it would allow to write really clean and beautiful TF code in Scala once it does work.

[eaplatanios]

Yeah sorry...that's what I meant. I agree with respect to the API, but we would need to think about how to deal with forward declaration of recursive functions. I think there are some simple solutions, but the implementation will indeed not going to be trivial.

[doofin]

It seems that this is trival with the new tf eager ? I see there are eager ops in this project but haven't tested yet