-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unreliable auto-casting of pandas data in model fitters #9205
Comments
maciejskorski
changed the title
Non-reliable auto-casting of pandas data in model fitters
Unreliable auto-casting of pandas data in model fitters
Apr 13, 2024
This was referenced Apr 13, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
As of now
np.asarray
with no flags is used to convert model input into a proper numerical format, when fitting models such as OLS, WLS etc.statsmodels/statsmodels/base/data.py
Lines 504 to 511 in a0eca86
This is non-reliable, particularly for mixed data types, as shown below:
Code Sample, a copy-pastable example if possible
Note: As you can see, there are many issues on our GitHub tracker, so it is very possible that your issue has been posted before. Please check first before submitting so that we do not have to handle and close duplicates.
Note: Please be sure you are using the latest released version of
statsmodels
, or a recent build ofmain
. If your problem has been fixed in an unreleased version, you might be able to usemain
until a new release occurs.Note: If you are using a released version, have you verified that the bug exists in the main branch of this repository? It helps the limited resources if we know problems exist in the current main branch so that they do not need to check whether the code sample produces a bug in the next release.
Related Issues
Here #8794 opens a narrower discussion about non-reliability of handling nulls.
Suggested solution
I suggest to inform numpy about numerical types when casting
Expected Output
Fitting does not produce errors.
Output of
import statsmodels.api as sm; sm.show_versions()
[paste the output of
import statsmodels.api as sm; sm.show_versions()
here below this line]INSTALLED VERSIONS
Python: 3.10.12.final.0
OS: Linux 6.1.58+ #1 SMP PREEMPT_DYNAMIC Sat Nov 18 15:31:17 UTC 2023 x86_64
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
statsmodels
Installed: 0.14.1 (/usr/local/lib/python3.10/dist-packages/statsmodels)
Required Dependencies
cython: 3.0.10 (/usr/local/lib/python3.10/dist-packages/Cython)
numpy: 1.25.2 (/usr/local/lib/python3.10/dist-packages/numpy)
scipy: 1.11.4 (/usr/local/lib/python3.10/dist-packages/scipy)
pandas: 2.0.3 (/usr/local/lib/python3.10/dist-packages/pandas)
dateutil: 2.8.2 (/usr/local/lib/python3.10/dist-packages/dateutil)
patsy: 0.5.6 (/usr/local/lib/python3.10/dist-packages/patsy)
Optional Dependencies
matplotlib: 3.7.1 (/usr/local/lib/python3.10/dist-packages/matplotlib)
backend: module://matplotlib_inline.backend_inline
cvxopt: 1.3.2 (/usr/local/lib/python3.10/dist-packages/cvxopt)
joblib: 1.4.0 (/usr/local/lib/python3.10/dist-packages/joblib)
Developer Tools
IPython: 7.34.0 (/usr/local/lib/python3.10/dist-packages/IPython)
jinja2: 3.1.3 (/usr/local/lib/python3.10/dist-packages/jinja2)
sphinx: 5.0.2 (/usr/local/lib/python3.10/dist-packages/sphinx)
pygments: 2.16.1 (/usr/local/lib/python3.10/dist-packages/pygments)
pytest: 7.4.4 (/usr/local/lib/python3.10/dist-packages/pytest)
virtualenv: Not installed
The text was updated successfully, but these errors were encountered: