Keyboard layout for shortcuts #2428
Comments
RastislavKish
added
the
enhancement
|
Agreed this is a difficult one - it falls in the unfortunate gap between bug (because there are incorrect/misleading/unreachable key combinations), and feature (because fixing it requires a localisation framework of some kind). Let me say from the outset that I definitely want to fix this. The lack of support for non-US keyboards is entirely a function of my own cultural ignorance, not an intentional design choice. I want Toga to have good support for internationalization, localisation and accessibility (i18n, l10n and a11y) out of the box. However, the tyranny of being a English speaking, non-impaired developer means my practical experience with these features is often lacking; and in an attempt to get something working, i18n, l10 and a11y concerns will often get neglected. The assistance and experience of non-English-native developers like yourself is invaluable when it comes to addressing i18n and l10n issues. You've mentioned that keyboard shortcuts should be defined through hardware keys. Conceptually, I can see what you're saying, and agree; the complication is that we need to be cross platform, and there isn't a cross-platform representation for hardware keys - and some platforms don't seem be entirely compatible with this concept. Toga's key definitions essentially are "hardware" representations, although they're defined in text form in an enumeration - The problem we hit is that this mapping is both platform and locale dependent - especially when it comes to "shifted" keys on some platforms. GTK seems to do this reasonably well - there's a separate GDK key code for However, on macOS, the mapping is tied to physical keys - so the "1" key is "key 18", and it's up to Toga to map from "key 18 + shift" to The situation on Windows is much the same, except even more confusing because the Winforms Key enum has some... very weird design decisions - key names seem somewhat arbitrary in some cases, and I haven't seen any way to represent locale-specific keys like Ľ and Ô. I'm open to suggestions. As I said at the start - the lack of support for non-US keyboards wasn't an intentional choice, it's purely a function of not having access to non-US keyboard to develop/test with. Any and all assistance on this issue would be most welcome. |
|
Thanks for the quick and detailed response. There is no need to worry, feedback is what community is here for. :) Also, I wouldn't view keyboard-related stuff as necessarily a pure a11y matter, indeed keyboard control is an important feature for people who can not use the mouse, but also for powerusers who want to use their programs efficiently, Vim with its community is albeit a bit extreme, but great example of the power a good keyboard input can provide. This is the viewpoint I would be primarily considering when designing keyboard shortcuts system specifically, i.e. powerusers, because the aimed benefit is speed and convenience, increased a11y is just a nice benefit on top. Perhaps it would be good to clarify the terminology a bit, I feel we're using the same terms but actually referring to the opposite ends of the problem:
By hardware representation, or hardware key, I imagine a locale-independend identifier of a physical key on a "Generic 105 key PC" keyboard. However, with this definition, the Key enum shouldn't contain EXCLAMATION and _1 at once (I didn't in fact notice the _1 exists because help(toga.Key) doesn't list it likely due to the _ being interpreted as access indicator, thanks for pointing this out). I would imagine something like the X keycodes, note for sake of convenience, the symbols of the English keyboard layout are often used for labelling the individual keys so they can be referred to in a convenient way, this is used by the win32 libraries, Android keyboard input, WPF, ... And, what I'm suggesting is that these hardware keycodes would be used for setting shortcuts by the programmer. Because, to give an example, right now, I'm creating a program, where the user is expected to quickly place timemarks of different categories while doing other things by pressing Alt+1,, Alt+2, ... (the number reprresents a category) Basically, what I'm trying to say is that keyboard shortcuts are primarily about physiology of hand movements and maximizing their efficiency, therefore, it makes much more sense to define them based on the keyboard geometry (which is mostly generic) rather than layout (which is language dependent). The only aspect of shortcuts related to the user's language I can think of is memorability, i.e. if the programmer wanted to have a shortcut for say "Go to page" function, they may choose Ctrl+P to make it easier to remember, in such case, a Slovak variant could be Ctrl+S (like strana - page). But this is something that's very rarely done, and even if it is, I believe it should be a matter of localisation stack rather than the GUI one. These are basically my thoughts on approaching this. I believe it's the method used by most of GUI frameworks I've worked with, albeit some like GTK in particular do have quite confusing systems, but I think most of them should offer a reasonably low-level key representation. |
Oh, sure - and accessibility doesn't end with keyboard shortcuts. All I was saying is that Toga tries to do the right thing for accessibility as a general policy. We sometimes (often?) fall short, but we want to do better.
But that level of representation doesn't help - at least, not without another layer telling you what those physical keys mean. Knowing "the key three from the left on the top row was pressed" isn't a helpful identifier unless I know what that key represents. That's always going to be a symbol of some kind, or something non-printable like "delete", "enter", or a modifier. And that's before you've gotten into what "three from the top left" even means if you've got a Dvorak-mapped 42 key Corne keyboard :-) Knowing "the user pressed 1" or "the user pressed !" is a useful representation, and one that is tied to the hardware, because it represents what the user actually typed, independent of whether they needed to press shift or any other modifier to get that key; and if a shift or modifier was involved, it isn't dependent on what the unshifted or unmodified key was. In this representation, the only thing that "shift" is useful for is for differentiating "A" from "a".
I think you're convolving two different (but equally important) concepts in your analysis. The first is the key itself. "As a developer, I want to bind a key shortcut to the CTRL-'+' key combination.". This is keyboard dependent, and has a locale component. for a Slovak keyboard, "+" is an unshifted key on the top left of the keyboard; for me, it's a shifted "=" on the top right of the keyboard. As a developer, I don't particularly care where the key is on the keyboard - I just want the "+" symbol to activate it. It's Toga's job to work out where that key is, and how to bind to it, in a locale independent way. The second is locale-dependent shortcuts - "As a developer, I want the 'go to page' command to be bound to CTRL+p for English, CTRL-s for Slovak, ...". This is an important l10n problem to solve - but it also requires a solution the first, as you can't use "ctrl-s" unless you can reliably describe the "s" key. The shortcut choices could reflect physical keyboard geometry, language, or anything else. Interestingly, the second problem is a variant on one that Toga already handles - platform-specific shortcuts. For example, does the shortcut to quit the app involve the Q key (for Quit), the E key (for Exit), or the X key (for eXit)? This is the same underlying command, but different platform style guides dictate different shortcuts. We're able to make a split here by defining app commands at the backend level, but that doesn't help users who want to define "system" commands; and it doesn't help the l10n case where "Q" isn't the right shortcut in the current language (I don't know if Slovak apps use "Q/E/X" for quitting apps, but hopefully the point here is clear). |
I would say this depends on the application. If the purpose of capturing keys is typing, let's say I would be programming my own TextInput component, then yes, I do need to know the symbol the user wanted to put in the field, because that's what they want to see there. However, if I'm introducing keyboard shortcuts, my interest is not the symbol on the physical key, but it's location. For example, Vim had mapped the next line command to the J key, because it's on the home finger position, under the strongest finger of right-handed people, therefore easy to press. The symbol is not as much of a deal in this case, it's about the location. Indeed when you can, you select keys with symbols that evoke the functionality in some way to make them rememberable, but the location, number of keys to press, how spread apart they are always gets priority.
For me it's the exact opposite, I don't care what letter is printed on the key I want to use for a shortcut, I just want it's position to be the same as mine, to make sure it will be convenient and comfortable to use. Of course, there will always be physical keyboards that won't satisfy the requirements, there are plenty of them that don't contain some of the modifiers, lack the functional row, don't have special keys like Home, End, but then, such a keyboard is likely not the right tool for powerusing my application and the users who would like to do so likely know that very well, they can use the menus or action icons if they don't have a suitable hardware. It seems we're talking about two different approaches of choosing shortcuts, a location based and meaning based. Perhaps Toga could implement geometrical shortcuts and textual shortcuts, which would technically be the same, just the textual shortcut would at runtime go through a reverse-translation process to find out what is it's geometrical position in respect to the currently active layout, what needs to be done anyway. Note the names are what just came on my mind. |
It may be your interest - but I strongly challenge the assertion that this is the common case. I would argue that the vast majority of app keyboard shortcuts are assigned based on, in order:
Taking macOS in an English locale as an example - Save is CMD-S, Quit is CMD-Q, Hide is CMD-H, Minimize is CMD-M. "Preferences" might seem like it should be CMD-P, but it's CMD-, because Print is more significant as a "P" command. The position of these keys has no real bearing on anything. The shortcuts have their names because of the command names, followed by the availability of keys. An app like vim is such an extreme case that it really shouldn't really be used as a primary exemplar here. vim is an app that comes from an era where mice - and even keyboards with arrow keys - effectively didn't exist. "Microsoft Word" is a better exemplar for the types of behavior we're enabling here. It's certainly possible to build vim with Toga... but that's not the use case we should be optimising for. The other thing to keep in mind is how this will be implemented. How are you going to label the "sixth key from the left on the second row"? That's Y for me, but Z for you. What meaningful name/label can you apply as a constant in code? More importantly - how do the underlying GUI toolkits represent things? Every single one of them has a constant for "!". None of them have a constant for "the key at the top left of the keyboard".
I agree a translation layer would be possible; however, given that the underlying platform implementations are universally textual/symbol based, it would seem to make a lot more sense to map geometry to symbol than the other way around. |
|
One possible counterpoint is that while copy is Ctrl/Cmd C, cut and paste are X and V. I don't think those mean anything beyond the fact that they're physically next to C on a US keyboard. But all these examples may be somewhat beside the point... What exactly do other common apps do across different-language keyboards? @RastislavKish, in most apps, where are, for example, cut, copy, and paste for you? Are they where your X, C, and V are? Are they in the same spot that X, C, and C are on a US keyboard, even though that's Y, X, and C for you? (Or are they in some other spot entirely?) What about Undo — is it modifier + Z, or + \? What about more application-specific shortcuts, if you have a way of telling what the same app uses on a US keyboard? Whatever approach "normal" apps use is probably what Toga should do, at least by default. Somewhat separately, re: OS-dependent shortcuts for things like quit: we could do something like what Qt does and have a simulated "Quit" keycode that maps to the system default. If I recall, it has those for most common actions like cut, paste, close, etc. I'm not sure how it handles collisions with manually-defined shortcuts, though. |
Ok - you've got me there. If I was to retcon a rationalisation into my original description, I'd say "C" was picked because of the name; X and V were picked for memory reasons because it's easy to remember they're physically close to C.
The complication that Qt doesn't handle (AFAIK) is that Windows puts Quit in the File menu, and macOS puts it in the Application menu. However, you're right that system commands could be defined in an abstract "Whatever the key for Cut is" manner. However, that doesn't help us with user-defined commands that need to be locale dependent. There's essentially 2 issues here - one is a bug, and one is a feature. The bug is that I can't currently define a keyboard shortcut for CMD-! that is reachable for Slovak (and other non-US keyboard) users on macOS and Windows, because the keyboard handling on those platforms assumes ! is a shifted 1. This requires us to modify the "shift identification" code to be locale dependent. I don't believe GTK isn't affected by this because the Gdk keycode for ! is independent to 1; shifting is only used to differentiate A from a. It also requires us to add key definitions for keys that aren't currently reachable at all by Toga, like all the accent keys on a Slovak keyboard. The missing feature is that I can't define a localized keyboard shortcut. This is the "Go to Page" being CMD-P in English, but CMD-S for Slovak users. This would also be accompanied by translating the text of the command itself from English to Slovak. Localization is a big task, but one that I definitely want Toga to support. From there, @RastislavKish's "geometrical layout" is a separate API for specifying keycodes other that localization, mapping from some abstract geometrical representation to Toga keycodes in a locale specific way. I can't say this is a high priority for me personally, but if someone wants to tackle the problem and comes up with a good geometrical representation, then I guess we might as well support it. |
|
It is true I personally usually handle shortcuts symply by listening on window key presses, so I'm not sure how are various backends handling the question of specifically keyboard shortcuts in this regard. And they may actually do some fiddling there, for example Audacity I believe used to have ( and ) keys for selecting audio, the problem was that on Slovak keyboard, these two characters are not beside each other, but they're placed on ) and \ keys, plus you need to press Shift, so doing accurate selections was quite a drag. The special keys were causing also a lot of issues in programs like Excel, VS Code, the letter approach for picking shortcuts works reasonably well if the app has few functions, but as their amount starts to grow, there's always a need to use a different logic for their placement, these programs either used combinations like Ctrl+Shift+`, which can't even be pressed on Windows with a Slovak layout, because the ` key / character simply doesn't exist there, and, there were combinations like Ctrl+Shift+;, which could be technically done but they usually didn't work likely for similar problems that Toga currently has with Shift modifying the outcoming character. So, judging by apps, there were different approaches on this topic. And we have even more funny situations, because, Windows (at least on the Slovak keyboard, not sure about the English-one), translates right alt into Ctrl+LeftAlt. It is a system-level translation, meaning there's nothing like AltGr on Linux, the apps will simply see a Ctrl+Alt combination. While at the same time, because many keys on their own cary accented characters (take the number row, as an example, which is ; + Ľ Š Č Ť Ž Ý Á Í É =), we have many special keys assigned for Ctrl+Alt modifiers. For example, Ctrl+Alt+P produces ' on Windows. Right now, if toga defined one command with a Ctrl+Alt+P shortcut and another command with Ctrl+Alt+', pressing Ctrl+Alt+P would likely trigger either both of them, or, depending on the internal handling, would lead to undefined behavior. Keyboard layouts are a mess. :) That's why I don't like to mess with them. But this apparently is a personal preference, so sticking to what the underlying Toga frameworks offer sounds reasonable. Which should be also the starting point for possible implementation of any other approaches. If say GTK understands shortcuts like some modifiers plus a textual character (not physical key), then it indeed makes sense to have this as the base and implement a function, which would take physical key (marked by its position on English layout) + modifiers and return the textual character they produce, this could be used then to assemble shortcuts that are layout independend and Toga wouldn't actually need to introduce any new API except the translation function. With GDK, this can be done pretty easily, even though the naming and differences between hardware_keycode, keycode, keyval, ... are super-confusing, basically there is a Keymap structure exactly for this purpose, it can be obtained from the GDK display object and provides translate_keyboard_state method which can do the trick. I've done this in one of my projects and it worked really well. However, I don't know the state of this with Winforms and Cocoa. Those would need further research. |
Sure - but all these cases would be caught by the ability to define locale-specific key shortcuts. Whether the locale-specific shortcut has been selected for linguistic reasons or because of basic keyboard ergonomics doesn't matter - as long as every key on the keyboard can be mapped on any given locale's keyboard, and there's an ability to define locale-specific key codes, then the problems you describe can be avoided. That's why I've described this as a combination of a bug and a feature.
No disagreement on that :-)
It's not just that it's "reasonable" - it's that it's the only option that is available to us. A Winforms menu item can only be specified in terms of a
The current GTK implementation is entirely based on keyval; if there's a more appropriate API that we should be using that allows us to continue to define GTK keyboard shortcuts, feel free to suggest a change in a PR.
Sure - if you don't have expertise or access to Windows or macOS, we don't expect you to submit patches for those platforms (unless you're interested, of course!) As long as the GTK-specific changes don't impose API requirements on other platforms for an existing feature, we're happy to accept single-platform fixes. |
What is the problem or limitation you are having?
I was not sure whether to classify this as a feature request or a bug, I think it depends on the viewpoint.
Basically, right now, if I assign a hotkey for a command, say Key.MOD_1+"1", what I actually need to press to get it triggered is Ctrl+Shift+1, because the Slovak keyboard layout I'm using has the character "+" where the English keyboard has 1, and Shift+"+" produces 1 on our keyboard.
I would guess that Key.MOD_1+Key.EXCLAMATION yields a similar behavior on the English layout, i.e. you need to press Ctrl+Shift+1. Which sort of makes sense in this particular case, but normally, I would rather define this like Key.MOD_1+Key.SHIFT+"1" (it seems there is actually no value for 1 on the Key enum, I do see Key.NUMPAD_1 there but the number row is only represented by the special symbols, so this is what works instead).
Does Toga have a guidelined way of approaching this?
Describe the solution you'd like
For me personally, it makes more sense to define shortcuts through hardware keys, because various countries have very differing stickers on their keyboards, but actually very similar hardware layouts, so when I'm designing a keyboard-centric app, I can select shortcuts based on what's physically most convenient to press and makes most sense, so when my users install my app, most of them will have the fine-tuned logical control system I designed for them out of the box.
While, if I used localized shortcuts, they could be giving perfect sense on my keyboard, but be absolutely out of reah for many other countries, or not possible to press at all (like Ctrl+Ľ or Ctrl+Ô on an English keyboard).
Describe alternatives you've considered
_
Additional context
Environment
The text was updated successfully, but these errors were encountered: