Mouse actions can also trigger keys underneath the zoomed-in view, when using floating dynamic keyboards

[annakirkpatrick]

I've seen this behavior in 3.2.4 pre-release 3, but I don't know if it exists elsewhere. I only recently started using floating keyboards, so it's possible that this isn't a new issue.

When performing a mouse action with a zoom step, if the zoomed-in area overlaps with the floating keyboard, a keyboard key 'underneath' the zoomed-in view can be pressed when a point is selected.

It's a little tricky to explain, so I made a screen capture video: https://youtu.be/nONq2962e2s

Here's the simple dynamic keyboard used in the video: keyboard.zip

[kmcnaught]

This seems to be specific to the gamepad trigger again, very similar to the behaviour I mentioned in an email. I have a candidate fix but need to check there aren't any unintended consequences in the various selection modes.

[kmcnaught]

@JuliusSweetland I'd appreciate some input here please.

This is an issue that came in with my changes to enable continuous mouse actions (e.g. locking down left click). It used to be that you were either in "key selection mode" or "point selection mode" whereas now we need both subscriptions to be active at the same time to allow for selecting a key to lock it down, or a point selection. This leads to some complications with regarding when you get a key selection trigger but it's not currently relevant, so you need to filter it out at some point down the pipeline. We discussed this logic at the time and tested all the various combinations of key/point selection, magnifier, etc. It turns out some of the behaviours are not working correctly with a gamepad trigger source in particular.

The issue arises because the same trigger (e.g. Xbox Key A) is used for both point selection and key selection. When you are clicking on a magnified popup to make a point selection, Optikey receives/processes a "simultaneous" point and key trigger. For some reason that I can't pin down, the order of these events is different with a gamepad source compared to with a keyboard key source. The end result which @annakirkpatrick is describing is that a single click of the gamepad causes the intended click action to occur, and then immediately clicks on the key also.

In the case of a keyboard key trigger (such as F9), the point event is handled first, and in the process of being handled the selection mode is changed and the key event is ignored as the input source is re-started. This gives the correct behaviour.

In the case of a gamepad button trigger, the key event is handled first. When the Selection event is handled, it is correctly identified as an invalid event for the current state. However, by the time the SelectionResult event is handled, the app state (selection mode) has changed while the point event was being processed and so a key result is incorrectly processed. The reason we get this situation is that there is interleaving of the handling of the two events, due to an async delay in ProcessKeySelectionTrigger.

This comes in here:

OptiKey/src/JuliusSweetland.OptiKey.Core/Services/InputService.subscriptions.cs

Lines 217 to 221 in 9fde581

	PublishSelection(TriggerTypes.Key, triggerSignal.PointAndKeyValue);
	await Task.Delay(20); //Add a short delay to give time for the selection animation
	PublishSelectionResult(new Tuple<TriggerTypes, List<Point>, KeyValue, List<string>>(

While waiting at line 219, the point event is being processed and changing the selection mode, which means that the selection result handling thinks the event is valid again.

If I remove the async delay on L219 here everything is fine, because the events are handled serially and we can trust the logic for whether or not it's appropriate to process the event. I can't see any difference in the animations, but I'm not sure what I'm looking for. Can you remember why this was necessary?

[kmcnaught]

Ah, just realised it came in here 2839130

@AdamRoden, what was the visual effect that made this necessary? Does it only apply in certain circumstances, such as specific dynamic keys, or dwell animations vs switch triggers?