Model.fit stopped performing correctly after updates

[acouturi7]

I was using Model.fit with an install of lmfit from April 2022. It worked great for my equation which varies 5 parameters. With the current version of lmfit, it does not fit well at all and is extremely sensitive to initial guesses. I have another code file for a different application where I use the minimize function instead, and it behaves the same as before. Interestingly, I converted the Model.fit code to use minimize (trying to fix the problem) and it gives me the same exact results as Model.fit does. So I'm guessing Model.fit used to minimize in a different way than the minimize function, but now it minimizes in the same way. Could someone please shed some light on what was changed in Model.fit that could be causing this?

[reneeotten]

Please provide the lmfit version that "worked" for you before and I'm assuming you're now using the latest release? I assume you did read through the release notes to see what has changed between these versions to see whether that could explain the differences you're seeing?

Also, please provide the code you're using that would help pinpointing any potential cause.

[acouturi7]

It looks like the version from April 2022 was 1.0.3. Yes, I am now using the latest release which is 1.2.2. Yes, I read the release notes here: https://github.com/lmfit/lmfit-py/releases. The only mention of Model.fit is:

"Model.fit now takes a coerce_farray option, defaulting to True to
control whether to input data and independent variables that are "array-like"
are coerced to ndarrays of datatype Float64 or Complex128. If set to
False then independent data that "array-like" (pandas.Series, int32
arrays, etc) will be sent to the model function unaltered. The user may then
use other features of these objects, but may also need to explicitly coerce
the datatype of the result the change described above about coercing the
result causes problems. (Discussion #873; PR #899)"

I don't see how this would explain the issue. My use of Model.fit is pretty straightforward. The parameters just take on float values. And I want to stress that I changed nothing whatsoever about the code between data always fitting well on 1.0.3 and the same data not fitting well on 1.2.2. I am attaching a matplotlib graph of the data and the fit line to show how badly it is fitting some of my data (some data is still fit well), which I could probably fit better by manually brute forcing parameter value variation by hand. What I am wondering is if some default tolerance value or something was changed? Anything at all in Model.fit that wasn't reported in the notes? Here are some relevant lines of my code:

def circuitEqtn(f, fn, gamman, V_off, An, Bn):
    return V_off + 10*np.log10(C**2 + (Bn**2 * f**2) + ((An**2 + (2*C*An) - ((Bn*An*(f**2 - fn**2))/gamman))/
                                              (1 + ((f**2 - fn**2)**2)/(4*gamman**2 * f**2))))
circuitModel = Model(circuitEqtn)
paramsBareQCM = circuitModel.make_params(fn = fnGuess, gamman=50, V_off=0, An=17.61/10, Bn=1e-9)
resultBareQCM = circuitModel.fit(ampToFitBareQCM[i], paramsBareQCM, f = freqToFitBareQCM[i])

fnGuess is a float, array values at index i are themselves arrays of floats.

Thank you for taking the time to help.

[newville]

@acouturi7 if you're comparing 1.2.2 to 1.0.3, you may need to look further back in the Changelog.

A couple of possibilities come to mind:
a) the default value for the parameter controlling "step size for numerical derivative" got a little bit bigger between 1.0.3 and 1.2.2. If that is important, your fit was almost certainly never actually stable.
b) the handling of data (and independent variables) that is not strictly numpy ndarrays with datatype of Float64 is a little different. We are trying to be a bit more accommodating of data that is not "strict dtype=Float64" data, but this transition may not be complete.

The code you provided is not complete enough to know the type of data you are working with.
Your snippet of code is not complete, it is not self-contained. Are you expecting us to know what "ampToFitBareQCM[i]" is?
You did not include any fit reports or error messages.

For us to give anything beyond broad general statements you will have to fix these things.

[acouturi7]

Yes, that is the only mention of Model.fit between 1.0.3 and 1.2.2. Perhaps there is a more comprehensive changelog somewhere?

I see, the step size change could definitely be the culprit. It wouldn't necessarily surprise me if I had just gotten lucky before with a fairly unstable fit. If it's just a default value, is there a way I can change it? Also could you provide me with the 1.0.3 version?

I don't know about possibility b. I tried running it with the data types both coerced and not coerced, but maybe still could be it.

I provided all of the relevant parts (function calls, data types). I can't provide my full research code on the open internet.

I was mostly just asking for broad general statements like this. I don't expect you to fix anything for me.

[newville]

Yes, that is the only mention of Model.fit between 1.0.3 and 1.2.2. Perhaps there is a more comprehensive changelog somewhere?

There are Changelogs for most versions in the Releases section of GitHub.

I see, the step size change could definitely be the culprit. It wouldn't necessarily surprise me if I had just gotten lucky before with a fairly unstable fit.

Weird. Why would that not surprise you? It seems like you are saying that you know the fit is unstable. If you know that your fits are so unstable that they are sensitive to the initial steps taken to calculate the derivative, then you sort of know that the fits are not reliable.

If it's just a default value, is there a way I can change it?

Yes. You can read the docs and or the code.

Also could you provide me with the 1.0.3 version?

Um, well you can fork the repo and find it yourself. It should go without saying that we will not help you with that.

I don't know about possibility b. I tried running it with the data types both coerced and not coerced, but maybe still could be it.

Again, we do not know what you did.

I provided all of the relevant parts (function calls, data types).

Nope, you did not provide all the relevant parts. You did not even show a fit report.

I can't provide my full research code on the open internet.

Um, who said you did? Correct me if I am wrong here, but I believe that you literally asked us for help for code that we wrote (and have copyrighted), documented, released under a liberal open-source license, and put on the open internet. We responded to your request for help, free of charge. And you cannot put a complete enough example of code to show what you are doing? OK, then.

I was mostly just asking for broad general statements like this. I don't expect you to fix anything for me.

Good, we agree. This discussion is done.