SymPy integration

[Marlin-Na]

We can use SymPy to provide missing functionalities in SymEngine.

s4basic_as_py <- function(x, convert=FALSE) {
    symengine_py_module <- reticulate::import("symengine", convert=FALSE)
    ## Convert external pointer to pycapsule
    cap <- reticulate::r_to_py(x@ptr, convert=convert)
    ## Convert to symengine py object
    ans <- symengine_py_module$lib$symengine_wrapper$sympify_pycapsule(cap)
    ans
}

s4basic_from_py <- function(x) {
    symengine_py_module <- reticulate::import("symengine", convert=FALSE)
    ans <- symengine:::s4basic()
    symengine_py_module$lib$symengine_wrapper$assign_to_pycapsule(
        reticulate::r_to_py(ans@ptr), x)
    ans
}

r_to_py.Basic <- function(x, convert=FALSE) {
    s4basic_as_py(x, convert)
}

py_to_r.symengine.lib.symengine_wrapper.Basic <- function(x) {
    s4basic_from_py(x)
}

py_to_r.sympy.core.basic.Basic <- function(x) {
    s4basic_from_py(x)
}

Then this provides nice seamless integration through reticulate:

x <- S("x")
sympy <- reticulate::import("sympy")
sympy$integrate(x^2L/2L)
# (Mul)	(1/6)*x^3
sympy$integrate(x^2L/2L, tuple(x, 1L, 2L))
# (Rational)	7/6

The symengine object in R is first passed to python through pycapsule, then converted to symengine.py object. Thanks to symengine.py's compatibility layer, sympy functions can work on them and the result is seamlessly converted back to R's symengine object.

Related issues: symengine/symengine.py#319, rstudio/reticulate#660

[mattfidler]

The problem with this is that adding reticulate to allow SymPy usage means that the R package can't be called from python.

This is discussed on the RxODE issue here:

nlmixrdevelopment/RxODE#170 (comment)

Is there any way to make this a suggests instead of a dependency so that my python users can call the R package with rpy2, or if you integrate it to allow it to be turned off.

[Marlin-Na]

I think yes - reticulate and sympy stuff should be optional and there should be option to disable them.

[mattfidler]

I think that would be great. Thank you for your accommodation.

[mattfidler]

@rikardn do you know if the calling of rpy2 only fails if the package calls python->R->python? If it is simply loaded it doesn't affect the results?

[Marlin-Na]

@mattfidler Also I think loading reticulate package itself doesn't automatically initialize the python session. If we avoid initializing the python session ourselves I think it should be okay.

[mattfidler]

Thanks @Marlin-Na, good to know.

[rikardn]

@mattfidler If I remember correctly my python script that called nlmixr froze/crashed the python console only when nlmixr was calling python. But I didn't test much more than one estimation.

Another way of calling nlmixr from python would of course be to not use rpy2 and to do command line calls instead. That should always work. I would of course be grateful if you could isolate the parts needing python so that rpy2 could be used.

Just out of curiosity: how general are the functions that you want to integrate?

[Marlin-Na]

@rikardn In my mind, this is to implement functionalities that are not available yet in the SymEngine C++ library, e.g. integration, simplification. In future, as more functions are implemented in C++, we may replace them with C++ ones.

[rikardn]

@Marlin-Na Ok. Expect it to stay this way for a while since the development of symengine is very slow currently and that no Google summer of code project was announced for symengine this year.

[mattfidler]

Thank you for the clarification @rikardn. rpy2 is the preferred option, of course. @Marlin-Na thank you for the clarification too; I guess functions requiring SymPy will alert the user of course.

[bgoodri]

Thanks for doing all this stuff with symengine and SymPy. But I think I must be missing a step to integrate with SymPy. I installed symengine.py from GitHub (although its latest commit did not build so I had to checkout 62a0d89 "Merge pull request #319 from Marlin-Na/pycapsule"). When I run R code like in the OP, it does not seem to have the pycapsule functionality

symengine_py_module <- reticulate::import_from_path("symengine", convert = FALSE,
                                                    path = "/usr/local/lib/python3.7/dist-packages/")
symengine_py_module$lib$symengine_wrapper$sympify_pycapsule

Error in py_get_attr_impl(x, name, silent) : AttributeError: module 'symengine.lib.symengine_wrapper' has no attribute 'sympify_pycapsule'

even though the shared object seems to have the relevant symbols:

nm /usr/local/lib/python3.7/dist-packages/symengine/lib/symengine_wrapper.cpython-37m-x86_64-linux-gnu.so | \
c++filt | grep -F -i "capsule"
                 U PyCapsule_GetPointer
                 U PyCapsule_New
000000000049e030 r __pyx_k_capsule
0000000000563950 b __pyx_n_s_capsule
00000000002b59c0 t __pyx_pw_9symengine_3lib_17symengine_wrapper_1capsule_to_basic(_object*, _object*)
000000000015931f t __pyx_pw_9symengine_3lib_17symengine_wrapper_1capsule_to_basic(_object*, _object*) [clone .cold.2414]
0000000000266420 t __pyx_pw_9symengine_3lib_17symengine_wrapper_3assign_to_capsule(_object*, _object*, _object*)
0000000000155d23 t __pyx_pw_9symengine_3lib_17symengine_wrapper_3assign_to_capsule(_object*, _object*, _object*) [clone .cold.2267]
0000000000561a58 b __pyx_f_9symengine_3lib_17symengine_wrapper_assign_to_capsule(_object*, _object*, int)::__pyx_dict_version
0000000000561a50 b __pyx_f_9symengine_3lib_17symengine_wrapper_assign_to_capsule(_object*, _object*, int)::__pyx_dict_cached_value
000000000055dac0 d __pyx_pw_9symengine_3lib_17symengine_wrapper_3assign_to_capsule(_object*, _object*, _object*)::__pyx_pyargnames
00000000004908a0 r __pyx_pw_9symengine_3lib_17symengine_wrapper_3assign_to_capsule(_object*, _object*, _object*)::__PRETTY_FUNCTION__

Does anyone know what I overlooked?