GH-117881: fix athrow().throw()/asend().throw() concurrent access

[graingert]

• Issue: async generator allows concurrent access via async_gen_athrow_throw and async_gen_asend_throw #117881

[graingert]

this was set in async_gen_athrow_send but forgotten in async_gen_asend_send, and also results in an unawaited coroutine warning

[graingert]

cpython/Objects/genobject.c

Lines 2098 to 2113 in 8418dcc

	if (o->agt_state == AWAITABLE_STATE_INIT) {
	if (o->agt_gen->ag_running_async) {
	o->agt_state = AWAITABLE_STATE_CLOSED;
	if (o->agt_args == NULL) {
	PyErr_SetString(
	PyExc_RuntimeError,
	"aclose(): asynchronous generator is already running");
	}
	else {
	PyErr_SetString(
	PyExc_RuntimeError,
	"athrow(): asynchronous generator is already running");
	}
	return NULL;
	}

[graingert]

probably could do with a test

[graingert]

test added in 1220aee

[encukou]

As in #117851: I'm not an expert in this area.
It makes sense to start the generator in send/throw.

What I don't get is the purpose of all the synchronization between ags_state and ags_gen->ag_running_async/ags_gen->ag_closed. What's the difference between what they represent? Should they go out of sync at some point?

[graingert]

I think it's possible for them to go out of sync, but I think I have a fix for that here #117906

[encukou]

Is it a bug if they go out of sync?
If it is, and ag_running_async & ag_closed hold the necessary info, why have ags_state at all?

[graingert]

Is it a bug if they go out of sync? If it is, and ag_running_async & ag_closed hold the necessary info, why have ags_state at all?

you can have multiple of each agen.aclose() and agen.anext() objects constructed at the same time, and they each need to know what state they are in so they can update the agen fields appropriately

[graingert]

this is where gen->ags_state and ag_running_async go out of sync:

import types
import itertools

@types.coroutine
def _async_yield(v):
    return (yield v)


async def agenfn():
    for i in itertools.count():
        await _async_yield(i)
    return
    yield

agen = agenfn()
gen = agen.__anext__()
print(f"{gen.send(None)=}")
print(f"{agen.ag_running=}")
gen.close()  # uh-oh, gen->ags_state is CLOSED
print(f"{agen.ag_running=}")  # but the async generator is still running
agen.aclose().send(None)  # and it's broken so we can't aclose it

[encukou]

Ah! Got it, thank you!

[encukou]

The PR looks good to me. But, it would be great to have a more comprehensive way to find issues like this. I commented on the issue.

Was testing the bot

[hugovk]

No need for me to review, I was only testing the bot earlier. 👍

[graingert]

driveby: oops this wasn't running

[graingert]

@encukou is there anything else I need to do on this? I'd like to start looking at fixing athrow().close() and asend().close() next which builds on this work

[encukou]

No, sorry for the delay!

[miss-islington-app]

Thanks @graingert for the PR, and @encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

[miss-islington-app]

Sorry, @graingert and @encukou, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker fc7e1aa3c001bbce25973261fba457035719a559 3.12

[bedevere-app]

GH-118458 is a backport of this pull request to the 3.12 branch.