ConnectionError (Error UNKNOWN) since updating to Channels 4.0.0

[alexanderjulmer]

Hi y'all,

we've recently upgraded our application to Django 5 and Channels 4. We've installed the following packages relevant (I think) for async:

asgiref                  3.7.2
channels                 4.0.0
channels-redis           4.2.0
Django                   5.0.3
django-redis             5.4.0
gunicorn                 21.2.0
kombu                    5.3.4
redis                    5.0.1
uvicorn                  0.27.1
uvloop                   0.19.0
websockets               12.0

We are using Redis 6.2.14 with noeviction on Heroku.

We're now experiencing errors throughout the async part of the application. Unfortunately, we haven't yet figured out how to reproduce these errors reliably. They don't seem to affect all users all the time and the user experience does not seem to be affected at all.

Using the excellent stack traces from Sentry, these is the last code on our side that triggers the errors. The first two seem to be particularly effected.

await self.channel_layer.group_discard(
    get_annotation_group_name(self._chapter_id, self._user_id),
    self.channel_name,
)

await self.channel_layer.group_add(
    get_annotation_group_name(self._chapter_id, self._user_id),
    self.channel_name,
)

async_to_sync(channel_layer.group_send)(
    get_annotation_group_name(annotation.chapter_id, annotation.created_by_id),
    notification,
)

async_to_sync(channel_layer.group_send)(
    get_annotation_group_name(annotation.chapter_id, annotation.created_by_id),
    notification,
)

async def _subscribe_to_chapter_steps_updates(self):
    self.channel_name = await self.channel_layer.new_channel()
    group_name = get_chapter_step_group_name(self.chapter.uuid)
    await self.channel_layer.group_add(group_name, self.channel_name)
    logger.debug("Subscribed to chapter's updates: %s", group_name)
    self.notification_task = asyncio.create_task(
        self._listen_for_chapter_steps_updates()
    )

From here, by pure co-occurence one might suspect get_annotation_group_name to be the culprit, but it's simply

def get_annotation_group_name(chapter_id: Union[uuid.UUID, str], user_id: int) -> str:
    return f"chapter-{chapter_id}_annotations_user-{user_id}"

The actual error seems to be raised in the send_packed_command method in redis/asyncio/connection.py. Here's an exemplary stack trace:

Prior research

• Issue on Stackoverflow from 2023: same error but channels not involved, comment suggests async_to_sync as a possible solution. This seems not to be the issue for us.
• Discussion on Channels Github page: same error, traced back to a timeout and suggested a heartbeat to fix it. This sounds plausible, yet the error has only started to occur for us once we upgraded to Channels 4.0.0 so I would assume that our previous deployment should have had the same issues. Since it didn't I suspect a different cause.
• I did observe that for some (but not all) chapters, a "sorted set" is created on Redis. (Chapters are a group containing the changes to a collaborative document.)

You help is greatly appreciated.

Attempted fixes

• Use daphne instead of uvicorn: no effect
• Upgrade to py-redis^5.0.3: no effect

[carltongibson]

It looks like a connection error of some kind. It's not anything Channels specific. Sorry to not be able to help.