Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Startup procedure needs me to manually turn on the camera #1182

Closed
balkce opened this issue May 10, 2024 · 21 comments
Closed

Startup procedure needs me to manually turn on the camera #1182

balkce opened this issue May 10, 2024 · 21 comments

Comments

@balkce
Copy link

balkce commented May 10, 2024

Describe the bug

Every time the docker wyze bridge starts, it throws several "IOTC_ER_DEVICE_OFFLINE" errors. However, if I turn on the camera during the startup procedure, everything from that point on starts up quickly with none of those errors. But, the startup procedure turns the camera off as its first step, so I need to keep an eye on the log to see when the camera has been asked to turn off, and then manually turn on the camera so the rest of the startup goes smoothly.

Environment (if applicable)

  • Affected Camera(s): Wyze v3 pan
  • Affected Camera Firmware: 4.50.4.9222
  • Affected Bridge Version: 2.8.3
  • Bridge type: Home Assistant


This is an example of the log without me intervening:

[WyzeBridge] 📚 Using 'user' from local cache...
[WyzeBridge] 📚 Using 'cameras' from local cache...
[WyzeBridge] [+] Adding Living Room Cam [HL_PAN3]
[WyzeBridge] starting MediaMTX 1.1.1
[WyzeBridge] 🎬 1 stream enabled
[WyzeBridge] [CONTROL] SET living-room-cam state=stop
[WyzeBridge] [CONTROL] ☁️ Sending power_off to living-room-cam via Wyze API
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-90] IOTC_ER_DEVICE_OFFLINE
[WyzeBridge] 👻 Living Room Cam is offline.
[WyzeBridge] Living Room Cam will cooldown for 10s.
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'cruise_point', 'payload': '1', 'response': 'timed out'}
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-90] IOTC_ER_DEVICE_OFFLINE
[WyzeBridge] 👻 Living Room Cam is offline.
[WyzeBridge] Living Room Cam will cooldown for 10s.
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'reset_rotation', 'payload': 'PRESS', 'response': 'timed out'}
[WyzeBridge] [CONTROL] SET living-room-cam state=stop
[WyzeBridge] [CONTROL] ☁️ Sending power_off to living-room-cam via Wyze API
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-90] IOTC_ER_DEVICE_OFFLINE
[WyzeBridge] 👻 Living Room Cam is offline.
[WyzeBridge] Living Room Cam will cooldown for 10s.
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'cruise_point', 'payload': '1', 'response': 'timed out'}
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-90] IOTC_ER_DEVICE_OFFLINE
[WyzeBridge] 👻 Living Room Cam is offline.
[WyzeBridge] Living Room Cam will cooldown for 10s.
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'reset_rotation', 'payload': 'PRESS', 'response': 'timed out'}
[WyzeBridge] [CONTROL] SET living-room-cam state=stop
[WyzeBridge] [CONTROL] ☁️ Sending power_off to living-room-cam via Wyze API
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-90] IOTC_ER_DEVICE_OFFLINE
[WyzeBridge] 👻 Living Room Cam is offline.
[WyzeBridge] Living Room Cam will cooldown for 10s.
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'cruise_point', 'payload': '1', 'response': 'timed out'}
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-90] IOTC_ER_DEVICE_OFFLINE
[WyzeBridge] 👻 Living Room Cam is offline.
[WyzeBridge] Living Room Cam will cooldown for 10s.
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'reset_rotation', 'payload': 'PRESS', 'response': 'timed out'}
[WyzeBridge] [CONTROL] SET living-room-cam state=stop
[WyzeBridge] [CONTROL] ☁️ Sending power_off to living-room-cam via Wyze API
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-90] IOTC_ER_DEVICE_OFFLINE
[WyzeBridge] 👻 Living Room Cam is offline.
[WyzeBridge] Living Room Cam will cooldown for 10s.
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'cruise_point', 'payload': '1', 'response': 'timed out'}
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-90] IOTC_ER_DEVICE_OFFLINE
[WyzeBridge] 👻 Living Room Cam is offline.
[WyzeBridge] Living Room Cam will cooldown for 10s.
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'reset_rotation', 'payload': 'PRESS', 'response': 'timed out'}
[WyzeBridge] [CONTROL] SET living-room-cam state=stop
[WyzeBridge] [CONTROL] ☁️ Sending power_off to living-room-cam via Wyze API
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-90] IOTC_ER_DEVICE_OFFLINE
[WyzeBridge] 👻 Living Room Cam is offline.
[WyzeBridge] Living Room Cam will cooldown for 10s.
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'cruise_point', 'payload': '1', 'response': 'timed out'}
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-90] IOTC_ER_DEVICE_OFFLINE
[WyzeBridge] 👻 Living Room Cam is offline.
[WyzeBridge] Living Room Cam will cooldown for 10s.
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'reset_rotation', 'payload': 'PRESS', 'response': 'timed out'}
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-90] IOTC_ER_DEVICE_OFFLINE
[WyzeBridge] 👻 Living Room Cam is offline.
[WyzeBridge] Living Room Cam will cooldown for 10s.
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'pan_cruise', 'payload': '2', 'response': 'timed out'}
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-90] IOTC_ER_DEVICE_OFFLINE
[WyzeBridge] 👻 Living Room Cam is offline.
[WyzeBridge] Living Room Cam will cooldown for 10s.
[WyzeBridge] [living-room-cam] Snapshot timed out



And this is an example of the log with me manually turning on the camera during the startup procedure:

[WyzeBridge] 📚 Using 'user' from local cache...
[WyzeBridge] 📚 Using 'cameras' from local cache...
[WyzeBridge] [+] Adding Living Room Cam [HL_PAN3]
[WyzeBridge] starting MediaMTX 1.1.1
[WyzeBridge] 🎬 1 stream enabled
[WyzeBridge] [CONTROL] SET living-room-cam state=stop
[WyzeBridge] [CONTROL] ☁️ Sending power_off to living-room-cam via Wyze API
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-90] IOTC_ER_DEVICE_OFFLINE
[WyzeBridge] 👻 Living Room Cam is offline.
[WyzeBridge] Living Room Cam will cooldown for 10s.
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'cruise_point', 'payload': '1', 'response': 'timed out'}
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-90] IOTC_ER_DEVICE_OFFLINE
[WyzeBridge] 👻 Living Room Cam is offline.
[WyzeBridge] Living Room Cam will cooldown for 10s.
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'reset_rotation', 'payload': 'PRESS', 'response': 'timed out'}
[WyzeBridge] [CONTROL] SET living-room-cam state=stop
[WyzeBridge] [CONTROL] ☁️ Sending power_off to living-room-cam via Wyze API
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-13] IOTC_ER_TIMEOUT
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'cruise_point', 'payload': '1', 'response': 'timed out'}

<<< AT THIS MOMENT, I MANUALLLY TURN ON THE CAMERA >>>

[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] 📡 Getting 180kb/s HD stream (H264/20fps) via LAN mode (WiFi: 65%) FW: 4.50.4.9222 🔒
[living-room-cam] [CONTROL] Attempting to SET: reset_rotation=PRESS
[living-room-cam] WARNING: Skipping wrong frame_size at start of stream [frame_size=1]
[WyzeBridge] ✅ '/living-room-cam stream is UP! (3/3)
[living-room-cam] [CONTROL] response=None
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] ❌ '/living-room-cam' stream is down
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] 📡 Getting 180kb/s HD stream (H264/20fps) via LAN mode (WiFi: 62%) FW: 4.50.4.9222 🔒
[living-room-cam] [CONTROL] Attempting to SET: pan_cruise=2
[living-room-cam] WARNING: Skipping wrong frame_size at start of stream [frame_size=1]
[living-room-cam] [CONTROL] response=1
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] ❌ '/living-room-cam' stream is down
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] 📡 Getting 180kb/s HD stream (H264/20fps) via LAN mode (WiFi: 65%) FW: 4.50.4.9222 🔒
[living-room-cam] WARNING: Skipping wrong frame_size at start of stream [frame_size=1]
[WyzeBridge] ✅ '/living-room-cam stream is UP! (3/3)
[WyzeBridge] 📖 New client reading from living-room-cam
[WyzeBridge] 📕 Client stopped reading from living-room-cam
[WyzeBridge] ❌ '/living-room-cam' stream is down

@mrlt8
Copy link
Owner

mrlt8 commented May 10, 2024

Hmm, this seems like a sticky MQTT message. Can you try turning on the camera via MQTT and see if that reverts the change?

@balkce
Copy link
Author

balkce commented May 10, 2024

Do you mean turning on the camera via MQTT and then restarting?

EDIT: I did so, and yeah it doesn't happen now. Here is the log when I turn on the camera through MQTT and then restarting the Wyze Bridge addon:

[WyzeBridge] 📚 Using 'cameras' from local cache...
[WyzeBridge] [+] Adding Living Room Cam [HL_PAN3]
[WyzeBridge] starting MediaMTX 1.1.1
[WyzeBridge] 🎬 1 stream enabled
[WyzeBridge] [CONTROL] SET living-room-cam state=stop
[WyzeBridge] [CONTROL] ☁️ Sending power_on to living-room-cam via Wyze API
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] 📡 Getting 180kb/s HD stream (H264/20fps) via LAN mode (WiFi: 65%) FW: 4.50.4.9222 🔒
[living-room-cam] [CONTROL] Attempting to SET: cruise_point=('cruise_point', '1')
[living-room-cam] WARNING: Skipping wrong frame_size at start of stream [frame_size=1]
[living-room-cam] Pan to cruise_point=0 (65, 11)
[WyzeBridge] ✅ '/living-room-cam stream is UP! (3/3)
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] ❌ '/living-room-cam' stream is down
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] 📡 Getting 180kb/s HD stream (H264/20fps) via LAN mode (WiFi: 67%) FW: 4.50.4.9222 🔒
[living-room-cam] [CONTROL] Attempting to SET: reset_rotation=PRESS
[living-room-cam] WARNING: Skipping wrong frame_size at start of stream [frame_size=1]
[WyzeBridge] ✅ '/living-room-cam stream is UP! (3/3)
[living-room-cam] [CONTROL] response=None
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[WyzeBridge] ❌ '/living-room-cam' stream is down
[living-room-cam] 📡 Getting 180kb/s HD stream (H264/20fps) via LAN mode (WiFi: 69%) FW: 4.50.4.9222 🔒
[living-room-cam] WARNING: Skipping wrong frame_size at start of stream [frame_size=1]
[living-room-cam] [CONTROL] Attempting to SET: pan_cruise=2
[living-room-cam] [CONTROL] response=1
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] ❌ '/living-room-cam' stream is down
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] 📡 Getting 180kb/s HD stream (H264/20fps) via LAN mode (WiFi: 68%) FW: 4.50.4.9222 🔒
[living-room-cam] WARNING: Skipping wrong frame_size at start of stream [frame_size=1]
[WyzeBridge] ✅ '/living-room-cam stream is UP! (3/3)
[WyzeBridge] 📖 New client reading from living-room-cam
[WyzeBridge] 📕 Client stopped reading from living-room-cam
[WyzeBridge] ❌ '/living-room-cam' stream is down



So what does this mean? If I ever need to restart, I need to turn on the camera first through MQTT? Or am I missing something?

mrlt8 added a commit that referenced this issue May 12, 2024
mrlt8 added a commit that referenced this issue May 12, 2024
mrlt8 added a commit that referenced this issue May 12, 2024
* Sign motion event request #1125

* use device id instead of mac #1125

* remove wrong flags from mkfifo #1174

* less aggressive flush #1159 #1167

* Use K10052 for setting FPS #1161

* Refactor bits for going above 255 for quality

* Revert ffmpeg changes #1159 #1167

* version is obsolete

* default quality to hd180

* Update Wyze iOS App version from v2.44.5.3 to v2.50.6.1 (#1176)

* use struct pack

* Token based auth over the webUI

* Fix redirect for Home Assistant Ingress

* Use request headers to fix redirect for HA

* Change GET to POST for webhooks data

* Deprecate ifttt_webhook in favor of webhooks

* use yml for HA config and make credentials optional

* keep trying to identify audio #1172

* Snapshot on motion and push to mqtt #709 #970

* Add event time to motion message

* refactor auth

* EVENT_API option #1125

* Add additional headers  #1125

* Audio sync with higher bitrate

* Debug api request #1125

* Update api.py

* only debug on error

* Tweak audio sync

* don't raise error on lost frame

* clear buffer if out of sync

* Unique macs only #1125

Co-Authored-By: Cameron <[email protected]>

* Require auth by default and block non-ingress access #1181

* Allow non-ingress access with auth #1181

* Remove retain flag from commands #1182

* update webrtc to work with auth streams

* Add WB_API and rename WEB to WB #1181

WEB_USERNAME > WB_USERNAME
WEB_PASSWORD > WB_PASSWORD

* HA move /config/wyze-bridge/ to /config/

* Don't notify substream event and remove v2 #1125

* WebUI Auth related config for HA

* Don't retain discovery message? #1182

* Case sensitive credentials for WebUI

* changelog and readme

---------

Co-authored-by: Cameron <[email protected]>
@mrlt8
Copy link
Owner

mrlt8 commented May 12, 2024

Any ideas @jhansche?

@jhansche
Copy link
Contributor

Seems like this might be HA (mqtt integration entities) trying to reinstate the last known state of the discovered entities, which might be done automatically via a retained message on their end. When the bridge starts up, it's receiving the retained HA command message and trying to complete that command, but it hasn't connected to the camera yet, or possibly can't perform the command at all (like resetting position if the camera is off).

I don't think retain=false in the discovery message will solve this, and actually ends up leading to the same issue as has been reported in the past, so I think we need to keep that.

I think what's more likely is we need to drop retained HA commands that we receive on initial startup - or more to the point, don't respond with a failure if HA asked the bridge to turn it off and it's already off/offline anyway. In other words, it seems like HA is saying "as far as we know, you should be off please", and the bridge replies with an error "I can't turn it off because it's off, try again". When instead it should be "ok you want it off and it's off, nothing to do here"

This is all theoretical of course. To confirm for sure what's happening I would need to see the mqtt traffic that is being received, how the bridge is handling those messages, and how the bridge is responding. It's possible that we need to ignore a command that would be impossible (or result in an error) - like trying to turn off an offline camera?

When this cooldown happens, are we also reconnecting to mqtt?

@balkce
Copy link
Author

balkce commented May 17, 2024

@jhansche:

I would need to see the mqtt traffic that is being received, how the bridge is handling those messages, and how the bridge is responding.

I wouldn't mind providing this, but I don't know how to capture MQTT traffic.

@balkce
Copy link
Author

balkce commented May 17, 2024

I tried it again having the camera on before re-starting Wyze bridge and now there are IOTC_ER_TIMEOUT errors thrown, and a very long start procedure. I also made sure to re-start the MQTT addon so as to remove any messages that could have been stuck.

I then tried again, but now with me manually turning on the camera mid-way through the start procedure, and from then on everything went smoothly. Here's the log:

🚀 DOCKER-WYZE-BRIDGE v2.9.2 X86_64

Serving Flask app 'frontend'
Debug mode: off
[WyzeBridge] 📚 Using 'auth' from local cache...
[WyzeBridge] 📚 Using 'user' from local cache...
[WyzeBridge] WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
Running on all addresses (0.0.0.0)
Running on http://127.0.0.1:5000
Running on http://172.30.33.1:5000
[WyzeBridge] Press CTRL+C to quit
[WyzeBridge] 📚 Using 'cameras' from local cache...
[WyzeBridge] [+] Adding Living Room Cam [HL_PAN3]
[WyzeBridge] starting MediaMTX 1.1.1
[WyzeBridge] 🎬 1 stream enabled
[WyzeBridge] [CONTROL] SET living-room-cam state=stop
[WyzeBridge] [CONTROL] ☁️ Sending power_off to living-room-cam via Wyze API
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'cruise_point', 'payload': '1', 'response': 'timed out'}
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-13] IOTC_ER_TIMEOUT
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'reset_rotation', 'payload': 'PRESS', 'response': 'timed out'}
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-13] IOTC_ER_TIMEOUT
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'pan_cruise', 'payload': '2', 'response': 'timed out'}
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[WyzeBridge] [living-room-cam] Snapshot timed out
[WyzeBridge] [CONTROL] SET living-room-cam state=stop
[WyzeBridge] [CONTROL] ☁️ Sending power_off to living-room-cam via Wyze API
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-20011] AV_ER_TIMEOUT
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'cruise_point', 'payload': '1', 'response': 'timed out'}
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-13] IOTC_ER_TIMEOUT
[WyzeBridge] ☁️ Fetching 'cameras' from the Wyze API...
[WyzeBridge] [API] Fetched [1] cameras
[WyzeBridge] 💾 Saving 'cameras' to local cache...
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'reset_rotation', 'payload': 'PRESS', 'response': 'timed out'}
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-13] IOTC_ER_TIMEOUT
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'pan_cruise', 'payload': '2', 'response': 'timed out'}
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[WyzeBridge] [living-room-cam] Snapshot timed out
[WyzeBridge] [CONTROL] SET living-room-cam state=stop
[WyzeBridge] [CONTROL] ☁️ Sending power_off to living-room-cam via Wyze API
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'cruise_point', 'payload': '1', 'response': 'timed out'}
[WyzeBridge] [CONTROL] Connecting to living-room-cam
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] [-13] IOTC_ER_TIMEOUT
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] [MQTT] {'status': 'error', 'command': 'reset_rotation', 'payload': 'PRESS', 'response': 'timed out'}
[WyzeBridge] [CONTROL] Connecting to living-room-cam

<<< AT THIS MOMENT, I MANUALLLY TURN ON THE CAMERA >>>

[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] 📡 Getting 180kb/s HD stream (H264/20fps) via LAN mode (WiFi: 63%) FW: 4.50.4.9222 🔒
[living-room-cam] [CONTROL] Attempting to SET: pan_cruise=2
[living-room-cam] WARNING: Skipping wrong frame_size at start of stream [frame_size=1]
[living-room-cam] [CONTROL] response=1
[WyzeBridge] [CONTROL] Disconnecting from living-room-cam
[WyzeBridge] ❌ '/living-room-cam' stream is down
[WyzeBridge] 🎉 Connecting to WyzeCam Pan V3 - Living Room Cam on xxx.xxx.xxx.xxx
[living-room-cam] 📡 Getting 180kb/s HD stream (H264/20fps) via LAN mode (WiFi: 65%) FW: 4.50.4.9222 🔒
[living-room-cam] WARNING: Skipping wrong frame_size at start of stream [frame_size=1]
[WyzeBridge] ✅ '/living-room-cam stream is UP! (3/3)
[WyzeBridge] 📖 New client reading from living-room-cam
[WyzeBridge] 📕 Client stopped reading from living-room-cam
[WyzeBridge] ❌ '/living-room-cam' stream is down

As you can see, the start procedure turns off the camera and, for whatever reason, the camera does not respond from thereon after. It isn't until I manually turn on the camera (separately, using the Wyze app) that everything starts going smoothly.

I don't want to intrude in your thought process here, but I'm not really sure that the issue only lies with MQTT. It seems to me that, for some reason, when the Wyze Bridge sends Sending power_off to living-room-cam via Wyze API command, something happens that the camera becomes unresponsive to further API requests.

Just a thought.

And, for what's it worth, the MQTT log has lots of repetitions of the following:

2024-05-17 17:09:10: New connection from 172.30.33.1:53405 on port 1883.
2024-05-17 17:09:10: New client connected from 172.30.33.1:53405 as auto-EEE3735E-2712-A732-003B-4EB1F5061319 (p2, c1, k60, u'addons').
2024-05-17 17:09:10: Client auto-EEE3735E-2712-A732-003B-4EB1F5061319 disconnected.

but with different origin ports and the name of "auto-....", such as:

2024-05-17 17:09:10: New connection from 172.30.33.1:52095 on port 1883.
2024-05-17 17:09:10: New client connected from 172.30.33.1:52095 as auto-4DB70FB5-81AB-E6D6-7FA2-665799E68C22 (p2, c1, k60, u'addons').
2024-05-17 17:09:10: Client auto-4DB70FB5-81AB-E6D6-7FA2-665799E68C22 disconnected.

Let me know if you need anything else from me to help solve this issue.

@jhansche
Copy link
Contributor

I think I agree that this is not an mqtt issue. I had been resisting updating my bridge to 2.9, because I saw the commit about removing the retain flag from discovery, and I thought that would break for me.

I updated this weekend anyway, and I'm now seeing the same behavior that you are describing. MQTT Discovery appears to be unchanged for me for now, but I can confirm that it doesn't really start working correctly until all cameras are turned on, and I'm not sure I understand why.

@mrlt8 was this an intentional change? I don't recall seeing this behavior, or the reason behind it, in the release notes

@mrlt8
Copy link
Owner

mrlt8 commented May 19, 2024

I'll have to look into it. I think the only major change was when we updated our paho mqtt client to v2.0 back in v1.8.0 of the bridge.

@jhansche
Copy link
Contributor

jhansche commented May 19, 2024

2.8 was working fine for me up until yesterday. I updated to 2.9.3 yesterday, and started seeing this. So I don't think it's the paho client change - this seems to be deliberate 🤔

@mrlt8
Copy link
Owner

mrlt8 commented May 20, 2024

Ok, so I believe the issue was actually fixed here 2fb8d80 (retain option in the MQTT discovery payload) but the retain flag was still being kept by the MQTT server.

Since there isn't really a way to clear the retain flag, I updated the on connect callback to send an empty message to each of the get/set topics to try to clear it out if you're upgrading from an older version of the bridge with the retain flag set.

I've also updated the other topics to publish with the retain flag, so they should get re-broadcasted out when reconnecting.

Changes should be in the dev branch. Would appreciate any feedback!

@jhansche
Copy link
Contributor

If that were the problem, wouldn't it have affected me on v2.8?

@mrlt8
Copy link
Owner

mrlt8 commented May 20, 2024

I tried downgrading to a few versions, including 2.8, however when reconnecting with my MQTT client, I would get the get and set topics rebroadcasted unless I changed the discovery message's retain option to false.

I believe setting the retain option to true in the MQTT discovery payload would make HA publish to the get or set topics with the retain flag causing the bridge to get the message every time it starts up.

@jhansche
Copy link
Contributor

But what you changed in that commit was the discovery message retain flag. What is rebroadcasting when you reconnect is more likely HA telling you the last expected state, following the change in availability - and that has been in place since before 2.9

What I'm getting at, is there seems to have been a change in behavior, in how the Wyze bridge interprets those commands when they are received during startup 🤔

@mrlt8
Copy link
Owner

mrlt8 commented May 20, 2024

Were you communicating with the bridge directly over MQTT? This issue seems to affect commands sent via the HA GUI which were added via MQTT discovery.

You can replicate the issue on an older version by issuing a command via HA, then connecting to the MQTT server with another MQTT client after the command was sent.

@jhansche
Copy link
Contributor

I'm not sure what the question is asking; but I do interact with the mqtt-discovered entities, daily. I have automations that toggle the power switch, night vision switch, and set the Pan-v3 cruise points.

Again, what I'm saying is not disputing the behavior of ha+mqtt that you're describing. What I'm saying is that the behavior of the Wyze bridge, when receiving those topic messages (perhaps specifically during startup?) is the behavior that changed in a problematic way.

@mrlt8
Copy link
Owner

mrlt8 commented May 21, 2024

hmm, I don't think so. Most of the recent changes were related to ffmepg and timestamps.

@balkce can you see if the dev branch still has the same issue?

@balkce
Copy link
Author

balkce commented May 22, 2024

This is the log after starting the DEV branch version of Wyze:

🚀 DOCKER-WYZE-BRIDGE v2.9.2 X86_64 DEV BUILD [2024-05-20t13:53:55.689z] f15d0d5

Serving Flask app 'frontend'
Debug mode: off
[WyzeBridge] 📚 Using 'auth' from local cache...
[WyzeBridge] 📚 Using 'user' from local cache...
[WyzeBridge] WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
Running on all addresses (0.0.0.0)
Running on http://127.0.0.1:5000
Running on http://172.30.33.1:5000
[WyzeBridge] Press CTRL+C to quit
[WyzeBridge] 📚 Using 'cameras' from local cache...
[WyzeBridge] [+] Adding Living Room Cam [HL_PAN3]
[WyzeBridge] starting MediaMTX 1.1.1
[WyzeBridge] 🎬 1 stream enabled
[WyzeBridge] API Motion Events Enabled [interval=1.5]

It doesn't seem to have that issue, since I'm able to control the webcam (power on and off, for example) right after a restart even if the camera is off.

@balkce
Copy link
Author

balkce commented May 22, 2024

Wait, this is weird. When I switched back to the stable branch (2.9.2), it seems to be working the same as the DEV branch now. Here is the log:

🚀 DOCKER-WYZE-BRIDGE v2.9.2 X86_64

Serving Flask app 'frontend'
Debug mode: off
[WyzeBridge] 📚 Using 'auth' from local cache...
[WyzeBridge] 📚 Using 'user' from local cache...
[WyzeBridge] WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
Running on all addresses (0.0.0.0)
Running on http://127.0.0.1:5000
Running on http://172.30.33.1:5000
[WyzeBridge] Press CTRL+C to quit
[WyzeBridge] 📚 Using 'cameras' from local cache...
[WyzeBridge] [+] Adding Living Room Cam [HL_PAN3]
[WyzeBridge] starting MediaMTX 1.1.1
[WyzeBridge] 🎬 1 stream enabled

Could the DEV branch done something to the MQTT messaging buffer?

@mrlt8 mrlt8 mentioned this issue May 22, 2024
mrlt8 added a commit that referenced this issue May 22, 2024
* Clear retain flag from MQTT discovery #1182

* Ignore commands when connection is stopping

* changelog
@mrlt8
Copy link
Owner

mrlt8 commented May 24, 2024

For some reason, the retain flag was getting stuck on all the set/get topics which was causing the messages to get re-sent on start up. Each subsequent message would then update the topic, but not clear the retain flag.

v.2.9.3 basically sends a blank message with a retain flag to all the set/get topics which effectively clears that retain flag.

Will close this issue for now, but feel free to open a new issue if you run into other bugs.

@mrlt8 mrlt8 closed this as completed May 24, 2024
@balkce
Copy link
Author

balkce commented May 24, 2024

I've updated to 2.9.3, and it doesn't seem to have this issue.

Thank you so much!

@jhansche
Copy link
Contributor

Things seem to be working ok for me too - I was expecting to encounter issues with the retain flag removed for discovery.

But that might have been specifically when restarting HA, which I don't think I've done yet. Reloading the mqtt integration seems to work ok, so I think we're good 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants