New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Companion] Documentation Dumps #2325
Comments
Apparently there is no need to do pair-verify if pair-setup has just been successfully completed. thiccaxe/CompanionGames#2 I doubt this will be viable in pyatv due to how the "pairing" session is treated seperately from the "connect" session. However, I guess this repo has a bit more visibility in case anyone runs across this! |
TVSystemStatus event is used instead of SystemStatus. |
Old ios (12) seems to allow siri of all things through the companion remote.
Record the time this was sent. then the device sends
then stop the voice input Likely a solution for #170 , that issue links to protobufs that document what goes inside the |
I have been investigating touchpad too. Having some trouble to fully mimic iOS remote - especially swipe up to close apps. Other than that seems like I managed to make it work. |
I can take a look at it from my end. Is your remote public? |
No, not public - it's iOS app to control Mac/PC and Apple TV, and I'm just adding touchpad for Apple TV to it - so not even released. I checked your game and now see that you running a sever and controlling the game using iOS Apple TV remote. I'm working from another end :) trying to be a client that control it. |
Someone found that double clicking up will close apps when in the app switcher. |
Using _tPh value 5? |
No, I believe it's just two sets of press-unpress |
Yes. I see this as well, in atvproxy. I am investigating if anything unusual happens when closing apps now. Having some issues with atvproxy. |
Was able to fix it, Apple TV closing the socket, by sending _systemInfo frame. |
@cherpake how are you calculating the _ns value? |
It seems that siri has worked its way into the latest update? not sure exactly. You can trigger siri using homebutton press and hold on phones with home buttons and the power button on phones without homebutton.
unfortunately the mangling that |
What to investigate?
With my goal being to setup a companion server so that one can use built-in remote app to control any arbitrary interface (for example, use it as a mouse with the touch pad, or just control media), I thought it prudent to essentially dump any logs and progress I might encounter here, notably anything that is missing in pyatv (even though it may not be useful to add such a thing...)
note, on latest 17.3 beta ios
Captions
related to #720
Enable
Set Captions Settings (13)
{'_i': '_mcc', '_x': 0, '_btHP': False, '_c': {'_mcs': 4, '_mcc': 13}, '_t': 2}
Get Captions Settings (12)
{'_i': '_mcc', '_x': 0, '_btHP': False, '_c': {'_mcc': 12}, '_t': 2}
->
{'_c': {'_mcs': 2}, '_t': 3, '_x': 0}
Disable
Set Captions Settings (13)
{'_i': '_mcc', '_x': 0, '_btHP': False, '_c': {'_mcs': 4, '_mcc': 13}, '_t': 2}
Get Captions Settings (12)
'_i': '_mcc', '_x': 0, '_btHP': False, '_c': {'_mcc': 12}, '_t': 2}
->
{'_c': {'_mcs': 3}, '_t': 3, '_x': 0}
NO CLUE how this works, seems to be unlike what is described in 720. Sending the set caption settings (same message each time) seems to toggle _mcs from 2 and 3. Has 0 effect in Netflix, youtube, zee
Note that the _mcf in a video is 1806. In this case bits 0x0040 and 0x0080 are set to 0, whereas in music for example mcf is 511, in which both bits are set to 1. Related?
mcf while watching a youtube ad is 270, 256 while switching from one ad to the next ad. reverts to 1806 in regular operation. 1805 when you go to home screen, then 256 a second later.
Siri Remote bluetooth
Related to new feature where you can find Siri/Apple TV remote from ios
Get keys/info about siri remote.
Not sure about behavior when there 2+ remotes connected
{'_i': 'FetchSiriRemoteInfo', '_x': 0, '_btHP': False, '_c': {}, '_t': 2}
{'_c': {'SiriRemoteInfoKey': BPLIST}, '_t': 3, '_x': 0}
BPLIST:
'6 bytes' is mac address.
tangentially related to discussion in #720 ? It seems that a NSObject is wrapped in a Bplist is wrapped in a OPACK frame...
Events
{'_i': '_interest', '_x': 0, '_c': {'_regEvents': ['PushSiriRemoteInfo']}, '_t': 1}
same as fetch. sent on airpods connect for some reason.
Enable searching:
{'_i': 'ToggleFindingMode', '_x': 0, '_btHP': False, '_c': {'FindingModeEnabledKey': 'YES'}, '_t': 2}
Disable searching:
{'_i': 'ToggleFindingMode', '_x': 0, '_btHP': False, '_c': {'FindingModeEnabledKey': 'NO'}, '_t': 2}
Supported Actions events
Register:
{'_i': '_interest', '_x': 0, '_c': {'_regEvents': ['SupportedActions']}, '_t':
GuideSupportedKey
seems to be updated on any major window change (state changes (power on, screensaver off), entering the quick menu (press and hold home button) and entering/exiting apps). related to accesibility?
{'_i': 'SupportedActions', '_x': 0, '_c': {'GuideSupportedKey': False}, '_t': 1}
NowPlayingInfo
seems to be somewhat handled in MRP protocol.
Register:
{'_i': '_interest', '_x': 0, '_c': {'_regEvents': ['NowPlayingInfo']}, '_t': 1}
Play
Note that is NOT A GUARANTEE that video/audio is being played or not. for example, netflix preview triggers this. zee preview does not trigger this.
{'_i': 'NowPlayingInfo', '_x': 4159442948, '_c': {'NowPlayingInfoKey': BPLIST}, '_t': 1}
BPLIST:
netflix tv show, and
apple music song
have same. There is no difference between audio and video, seemingly, to tv remote app. note _mcf bit field
Pause
BPLIST
Touch interface
it is likely critical to figure out how NS is calculated before this could be used.
_tPh values:
5: "click" action; tap, press and hold, has (ended?)
1: swipe started
3: swipe progressing
4: swipe ended
x, y is cx, cy. x ranges left to right, 0-1000
y ranges top to bottom, 0-1000.
Swipe situation
simply send the hidT (timestamped?) with _ns and the appropriate x and y coordinates as the swipe happens.
send the appropriate _tPh values as well. In case that there is more complex situations you need to deal with, see below.
hidT will not recieve OPACK ack from server. hidC functions as you would expect.
Tap
quickly aftger each other
{'_i': '_hidC', '_x': 0, '_btHP': False, '_c': {'_hBtS': 1, '_hidC': 6}, '_t': 2}
{'_i': '_hidC', '_x': 0, '_btHP': False, '_c': {'_hBtS': 2, '_hidC': 6}, '_t': 2}
{'_i': '_hidT', '_x': 0, '_c': {'_ns': 41004766480958, '_tFg': 1, '_cx': 1000, '_tPh': 5, '_cy': 1000}, '_t': 1}
x and y as _cx _cy, will always bee 1000,1000 no matter where it is actually pressed
Press and hold
quickly after each other
{'_i': '_hidC', '_x': 0, '_btHP': False, '_c': {'_hBtS': 1, '_hidC': 6}, '_t': 2}
{'_i': '_hidT', '_x': 0, '_c': {'_ns': timestamp? big number at least 46 bits, maybe 64?, '_tFg': 1, '_cx': 1000, '_tPh': 5, '_cy': 1000}, '_t': 1}
x and y as _cx _cy, will always bee 1000,1000 no matter where it is actually pressed
release
{'_i': '_hidC', '_x': 0, '_btHP': False, '_c': {'_hBtS': 2, '_hidC': 6}, '_t': 2
Press and hold then swipe to right
{'_i': '_hidT', '_x': 0, '_c': {'_ns': 40673733253583, '_tFg': 1, '_cx': 335, '_tPh': 1, '_cy': 611}, '_t': 1}
{'_i': '_hidT', '_x': 0, '_c': {'_ns': 40674033085625, '_tFg': 1, '_cx': 340, '_tPh': 3, '_cy': 598}, '_t': 1}
{'_i': '_hidC', '_x': 0, '_btHP': False, '_c': {'_hBtS': 1, '_hidC': 6}, '_t': 2}
-> recieved ack for 3, not sure if it is needed to wait for this?
{'_i': '_hidT', '_x': 0, '_c': {'_ns': 40674033085625, '_tFg': 1, '_cx': 340, '_tPh': 5, '_cy': 598}, '_t': 1}
as you can see, x changed from 335 to 340
dial action
the physical remote supports circular motion at the border of its touch surface to wind forward/backwards. This is not supported in ios apple tv remote app in general.
tap with two+ fingers.
regular tap but tph is set to 4. Does no action in tvos, but you could still detect it I guess.
Pressing the power button, mute button
regular hidC button call
{'_i': '_hidC', '_x': 0, '_btHP': False, '_c': {'_hBtS': 1, '_hidC': NUMBER}, '_t': 2}
{'_i': '_hidC', '_x': 0, '_btHP': False, '_c': {'_hBtS': 2, '_hidC': NUMBER}, '_t': 2}
Power button: NUMBER = 19
Mute button: NUMBER = 18
Skip forward and backward
{'_i': '_mcc', '_x': 1725146213, '_btHP': False, '_c': {'_skpS': SKIP AMOUNT IN SECONDS AS A FLOAT, '_mcc': 7}, '_t': 2}
examples:
10.0
,-10.0
press and hold left arrow button
returns to home screen.
Expected outcome
written above is pretty much everything that I saw missing in pyatv.
Based on this I will begin work on my server. I hope to make updates to pyatv as time allows (but then I have to do testing and make sure my code is good quality and other silly things like that /s)
The text was updated successfully, but these errors were encountered: