Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deserialize by using __new__ whenever possible #73

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

oldium
Copy link

@oldium oldium commented Nov 3, 2023

Using of the class constructor and passing it the args values is not always correct, the custom parameters could be different to the args actually recorded by the built-in Exception class. Try to use the __new__ to create class instance and initialize the args attribute afterwards to prevent calling constructor with wrong arguments. Other instance attributes are initialized in a standard way by pickle module, so this way the reconstruction of the exception is complete.

Subclasses of OSError are known exceptions to this, because they initialize read-only attributes from the constructor argument values, so usage of the new factory method leads to lost values of those read-only attributes.

Fixes: #65

@oldium oldium force-pushed the fix-type-error branch 3 times, most recently from 211dcff to b2764c9 Compare November 3, 2023 13:09
@oldium
Copy link
Author

oldium commented Nov 3, 2023

Applied changes suggested by the check job.

@oldium oldium force-pushed the fix-type-error branch 2 times, most recently from 0fc3000 to 62b857c Compare November 7, 2023 21:53
@oldium
Copy link
Author

oldium commented Nov 7, 2023

Tests added

@oldium
Copy link
Author

oldium commented Nov 7, 2023

Looks like (first guess) that PyPy does not like test for built-in reducer. I will check it.

@oldium
Copy link
Author

oldium commented Nov 8, 2023

I created a bug report for PyPy pypy/pypy#4026 regarding the builtin method check. For now, we could switch to inspect.isbuiltin() (it looks it is not only a BuiltinFunctionType check in PyPy) and skip the test on PyPy until it is fixed?

@oldium
Copy link
Author

oldium commented Nov 8, 2023

I did what I proposed - there are two more commits, one switching to inspect.isbuiltin() (i.e. no functional change) and second one disabling issue #64 test. This means that under PyPy the behaviour of tblib is unchanged, the bug is still there. I hope PyPy team fixes/addresses the issue soon.

@oldium
Copy link
Author

oldium commented Nov 8, 2023

In the PyPy bug report one suggestion was to use obj.__class__.__reduce__ is BaseException.__reduce__ and it works both in CPython and PyPy. See the latest commit, which fixes the problem of #64 also for PyPy. Now the tests should be all green 😁

@oldium
Copy link
Author

oldium commented Nov 18, 2023

I am done with this PR.

@oldium oldium changed the title Deserialize by using __new__ instead of constructor if possible Deserialize by using __new__ whenever possible Feb 19, 2024
@oldium
Copy link
Author

oldium commented Feb 19, 2024

Squashed related commits, updated reasoning in description and updated naming in code (no functional change, just added one more test). Also added two fixes for the tox build job, feel free to cherry-pick it separately 😊

@oldium oldium force-pushed the fix-type-error branch 2 times, most recently from 78ea010 to d5713ba Compare February 19, 2024 11:55
@oldium
Copy link
Author

oldium commented Feb 19, 2024

Rebased to latest master

@oldium
Copy link
Author

oldium commented Jul 8, 2024

Rebased to the latest master and as a bonus fixed build with Python 3.8, 3.9 and 3.10 on macos.

Enjoy 😁

cjwatson added 3 commits July 8, 2024 23:11
The doctests weren't being run and had bitrotted.  Make them work again.
macos-latest doesn't seem to have working binaries for these.
@oldium oldium mentioned this pull request Jul 26, 2024
@stuaxo
Copy link

stuaxo commented Jul 31, 2024

@oldium not sure if you noticed, but PyPy fixed the isbultin bug you reported.

@oldium
Copy link
Author

oldium commented Jul 31, 2024

@stuaxo Are you sure it is fixed, or you found that the bug is closed?

This fixes `CoverageWarning: Couldn't parse Python file
'.../tests/badsyntax.py' (couldnt-parse)`.

Based on a change by Oldřich Jedlička.
@stuaxo
Copy link

stuaxo commented Aug 5, 2024

@oldium ooh, good point, looking at the ticket again I am not sure - apologies for the noise.

oldium added 2 commits August 22, 2024 17:48
Using of the class constructor and passing it the args values is not always
correct, the custom parameters could be different to the args actually recorded
by the built-in Exception class. Try to use the __new__ to create class instance
and initialize the args attribute afterwards to prevent calling constructor with
wrong arguments. Other instance attributes are initialized in a standard way by
pickle module, so this way the reconstruction of the exception is complete.

Subclasses of OSError are known exceptions to this, because they initialize
read-only attributes from the constructor argument values, so usage of the
__new__ factory method leads to lost values of those read-only attributes.

Fixes: ionelmc#65

Signed-off-by: Oldřich Jedlička <[email protected]>
@oldium
Copy link
Author

oldium commented Aug 22, 2024

Rebased on top of #79

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unpickling exceptions can fail with TypeError about the number of arguments given in specific scenarios
4 participants