Symbolic regression with "conditional constants"

[mkellerressel]

I would like to do symbolic regression on a dataset with two variables, say $x$ (real scalar) and $i$ (categorical) with response $y$.
The goal is to find a symbolic function $f(x) \approx y$ that may involve constants conditional on the value of $i$, that is the constants in $f$ should be the same for any given value $i = c$, but may be different for, say $i = d \neq c$
Is it possible to do this with PySR?
Note that this is not equivalent to splitting the dataset given the level of $i$ and fitting a separate symbolic regression model on each part, as this will give different functions $f_i(x) \approx y$ on each part. I would like to have the same symbolic form only with different constants for each level of $i$.

Any advice is appreciated!

[MilesCranmer]

Hey @mkellerressel,

Great question. So the first thing I would suggest is trying the approach used here: https://arxiv.org/abs/2202.02306. This learns a mass parameter for each planet while simultaneously learning a model that takes those mass parameters as input. Then, finally, the neural network is approximated with symbolic regression.

Now, to fit separate constants during symbolic regression itself (so you could solve this just with PySR/SR.jl), its not set up yet but I am planning to add this via MilesCranmer/SymbolicRegression.jl#271 as it would let you define custom types of trees — including trees that have a variable set of constants!

Cheers,
Miles

[mkellerressel]

Thanks a lot for the quick reply @MilesCranmer! I will have a look at the paper.

best,
Martin

[gm89uk]

Hi @mkellerressel,

I had the same problem and solved it with a custom loss function:
#701 (comment).

The categorial group is passed as a nominal group as weights from pysr to the custom loss function.