Rockbox Technical Forums

Support and General Use => User Interface and Voice => Topic started by: salival on March 15, 2006, 09:47:01 AM

Title: usability: On-screen keyboard
Post by: salival on March 15, 2006, 09:47:01 AM
On Misticriver I started a thread (http://www.misticriver.net/showthread.php?t=38613) discussing the onscreen keyboards.

Currently there are at least two different layouts, the original rockbox keyboard and the keyboard used in rockword amongst others.

Although these keyboards do what they supposed to do, in my opinion they are not the most optimal solution. In my opinion there should be a single layout for all targets (ofcourse they differ slightly due to screen limitations).

My initial proposal for a layout is as show in the picture hereafter:
(http://img137.imageshack.us/img137/8729/keybmockup0kp.png) (http://img137.imageshack.us/img137/8729/keybmockup0kp.png)
(edit: the image appears transformed, click on it to show the original image)

other characters are displayed by using the "sym"(symbols) and "acc" (accents) buttons. Optionally it could be possible to cycle through these character sets by button on the player.

I'd like to hear your thoughts on this.

edit: wiki page (http://www.rockbox.org/twiki/bin/view/Main/VKeyboardDesignProposal)
Title: Re: usability: On-screen keyboard
Post by: Bagder on March 15, 2006, 09:51:00 AM
Any thoughts on how this would work for our friends that use Korean, Cyrillic, Greek or other non-latin letters?
Title: Re: usability: On-screen keyboard
Post by: salival on March 15, 2006, 09:56:11 AM
I think this kind of layout could be easily adapted to acccomodate other character sets. If the basic alphabet of such set has more than the current 26 characters an extra row could be added, an extra button to change the set.

So there is definitely room for those languages. Unfortunately I don't know the characteristics of those alphabets.
Title: Re: usability: On-screen keyboard
Post by: salival on March 15, 2006, 10:38:22 AM
I did some quick search on some alphabets and foud the following list (correct me if i'm wrong):
The text between () is how I think it could be displayed.

arabic: 28 (same as latin, plus two keys next to the spacebar)
cyrillic: 66 (three screens/sets of 22 characters each)
korean: 24 elements (same as latin, consonants and vowels grouped)
hiragana/katakana(japanese): 48 each (two screens each, one button to change between the two sets [hiragana/katakana], there could be an option to write kanji by inputting hiragana)
thai: ~70 (three screens)
Title: Re: usability: On-screen keyboard
Post by: kenshin on March 15, 2006, 10:54:27 AM
To fully support the Japanese Hiragana and Katakana base characters you need 71 characters in each set. There's an additional 36 each called "glides" where two characters are squashed together to make a single sound. For example, "ki" and "yo" squish together to form "kyo" and this symbol is written differently than "ki" and "yo" separately. You could probably cheat by making the user type in both characters, though. That would leave you with 71 base characters (in both sets) plus 3 glides in each set. Minimum total? 74*2=148.
Title: Re: usability: On-screen keyboard
Post by: pabouk on March 16, 2006, 05:38:49 AM
Salival thank you for your ideas. I think there is a need for an on-screen keyboard not occupying the whole screen. Here are my thoughts:

Why are there the boxes around the characters? I think they are completely useless and they take a lot of valuable space.

What are the cursor keys for? The cursor can be much easily moved using button combinations (e.g. ON/PLAY + RIGHT on H100s).

What about other useful characters like
Code: [Select]
!@#$%^&*()-_,<.>/?;:' "\|`~[{]}=+
I have an idea for accented characters:
Double press or long press on a letter can show accented variants for the letter. I.e. for "e" it could be é ě è ë ē ė ę ẹ ẻ ẽ ế ề ể ễ ệ etc.

To reduce loading of new unicode characters to the font cache it would be usefull to add an option to restrict available characters. It would be fine if the user could select his preferred ISO charset sets as an unicode subset. For example one could enable ISO 8859-1 + 8859-2 for the on-screen keyboard.
Title: Re: usability: On-screen keyboard
Post by: JdGordon on March 16, 2006, 05:53:28 AM
i had an idea of a single line at the bottom (or top) of the screen, which is bassiaclly a slider of all available charachters and u use left/right to choose the char and enter to select it...
much simpler and easier to use than the current one...
Title: Re: usability: On-screen keyboard
Post by: Zoide on March 16, 2006, 05:57:39 AM
I think there could also be Han Zi (Chinese characters), but we'd need something like Pinyin (where you type the romanization and it gives you a list of characters to choose from) and that would probably take a lot of programming...  Has anyone thought about this?
Title: Re: usability: On-screen keyboard
Post by: salival on March 16, 2006, 06:08:24 AM
Why are there the boxes around the characters? I think they are completely useless and they take a lot of valuable space.
To make it a little more pleasing to the eye. obviously this may be subject to change.

Quote
What are the cursor keys for? The cursor can be much easily moved using button combinations (e.g. ON/PLAY + RIGHT on H100s).

Some programs, like rockword and rocalendar use these to navigate around the "paper". So I thought they would come in handy. Another option would be to be able to quickly toggle between the keyboard and paper. That way these keys become obsolete

Quote
What about other useful characters like
Code: [Select]
!@#$%^&*()-_,<.>/?;:' "\|`~[{]}=+
I have an idea for accented characters:
Double press or long press on a letter can show accented variants for the letter. I.e. for "e" it could be é ě è ë ē ė ę ẹ ẻ ẽ ế ề ể ễ ệ etc.

Those are located under the "acc" (for accents) and "sym" (symbols) buttons.

Ofcourse my version is only a setup and I intended this thread to come to a widely accepted, easy to use, pleasing to the eye (etc. ) solution.

i had an idea of a single line at the bottom (or top) of the screen, which is bassiaclly a slider of all available charachters and u use left/right to choose the char and enter to select it...
much simpler and easier to use than the current one...

That would mean an awful lot of unnessecary keypresses. IMHO this is only an option to the target where you only have one line available, like remotes.

--------------------------------------------------------------------------------------------------

I played around a little with the japanese layout. As a reference I used the onscreen keyboard which comes with the windows IME. this can use the JIS layout.
(http://img79.imageshack.us/img79/3029/keybmockupjis6jx.png) (http://img79.imageshack.us/img79/3029/keybmockupjis6jx.png)
(click for the full image)(the characters are less readable because of antialiasing)
Title: Re: usability: On-screen keyboard
Post by: Llorean on March 16, 2006, 06:14:35 AM
Just as a note:

Rockbox is a music player. The first concern should be ease of use for inputting filenames on any of the music player targets. Rockword isn't even in CVS yet, and should it become so it's still not part of the objective of Rockbox to be a PDA replacement.

Any virtual keyboard ideas should be focused around maintaining as much simplicity and ease of use across all targets, and not increasing the binary size any more than absolutely necessary.

I'm of the opinion that the current vkeyboard doesn't have any serious disadvantages other than the characters being too small for some people on the larger screen or higher resolution targets. I think maybe reconsidering the font choice of those and maybe having a larger font and just showing the subset that the H1x0s show is a better solution.

I think any complicated keyboard ideas should be heavily weighed against the "how much bigger will it make the core on Archos" question, because if it doesn't add functionality (If it *just* looks better, and doesn't significantly increase ease of use) is it really worth sacrificing some of the limited binary size you already have?
Title: Re: usability: On-screen keyboard
Post by: JdGordon on March 16, 2006, 06:22:34 AM
while i agree... there isnt really anything stopping us useing the current code for archos and new "niceer" stuff for the rest of the players...
Title: Re: usability: On-screen keyboard
Post by: Llorean on March 16, 2006, 08:00:37 AM
Yeah, there's nothing to stop that. I'm just suggesting that, if a newer one is used for newer players, it should be as similar to the one used for Archos' as possible in button assignments and key layout.
Title: Re: usability: On-screen keyboard
Post by: moozooh on March 16, 2006, 08:23:58 AM
cyrillic: 66 (three screens/sets of 22 characters each)
Wrong. The language-specific characters are: АБВГДЕЁЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯ (and you may even discard Ё, as it is optional). That sums up to 33 (32 without Ё) characters (66/64 with both regular and capital sets, but they shouldn't be all on the same screen).
Title: Re: usability: On-screen keyboard
Post by: salival on March 16, 2006, 08:36:08 AM
I took a quick look (with my limited programming knowledge) in to how the current keyboard works. To me it seems the kind of layout I suggested wouldn't be too hard to implement, since there already is support for multiple screens implemented.

The remainder is a matter of generating the characters, some buttons and, for the more graphical players, the optional bitmaps, which should be basic shapes (in the case I presented, squares with rounded corners).

The display of a selected character would IMO a little harder to implement, but inverting the "button" should work.
Title: Re: usability: On-screen keyboard
Post by: kenshin on March 16, 2006, 11:49:02 AM
After looking closely at the screenshots I think I prefer the Archos layout better than the H1x0 layout. The graphics are just getting in the way.

After looking at the Japanese onscreen IME keyboard you'll have to implement a "shift" button to handle the "glides" I mentioned before. Also, some intelligence will have to be built into the keyboard to make sure a few characters are treated properly. There's 25 characters that use the "double quote" and "circle" to change the meaning of the symbol (i.e. from "ha" to "ba" (double quotes) or from "ha" to "pa" (circle)). I'd be happy to help define any rules for validation and character combinations for Japanese.
Title: Re: usability: On-screen keyboard
Post by: pabouk on March 16, 2006, 12:02:57 PM
cyrillic: 66 (three screens/sets of 22 characters each)
Wrong. The language-specific characters are: АБВГДЕЁЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯ (and you may even discard Ё, as it is optional). That sums up to 33 (32 without Ё) characters...
Yes but that is just for Russian ;) What about other languages using cyrillic like Belarusian, Serbian, Macedonian, Ukrainian, Bulgarian and many non-slavic languages from ex-soviet union countries? Are they inferior? ???

I'm of the opinion that the current vkeyboard doesn't have any serious disadvantages other than the characters being too small for some people on the larger screen or higher resolution targets. I think maybe reconsidering the font choice of those and maybe having a larger font and just showing the subset that the H1x0s show is a better solution.
I agree that the keyboard should be simple and I am not supporter of merely fancy-and-non-functional features like boxes aroung characters but I think that on-screen keyboard with space for more than one line of edited text would be very useful.

I do not agree that text editor is not important for Rockbox. It would be very useful for editing configuration files, writing notes about music I am listening on radio or podcast etc. Editing ID3 tags with more than one line visible would be really cool too.
Title: Re: usability: On-screen keyboard
Post by: salival on March 16, 2006, 12:43:31 PM
Some people have stated they didn't like the borders around the characters. Since I could understand why I created a cleaner mockup.

(http://img127.imageshack.us/img127/9624/keyboardmockupv30km.png) (http://img127.imageshack.us/img127/9624/keyboardmockupv30km.png)
(click for original)

I firmly believe that the bigger screens need some sort of container for the keyboard, so it may be implemented more easily into plugins. Plus it gives the oppertunity to group buttons, in this case numeric and function keys (shift etc.)
Title: Re: usability: On-screen keyboard
Post by: salival on March 16, 2006, 03:50:32 PM
When you have a button to toggle between navigating in the keyboard and moving around on the "paper" the on screen arrow keys can be removed, which frees up space for character heavy languages.

I think a selection-button hold would work best for this. (navi-hold/ joystickpress-hold/...)
Title: Re: usability: On-screen keyboard
Post by: salival on March 17, 2006, 04:39:48 AM
...and it would look something like this for non latin char sets.
(http://img62.imageshack.us/img62/8518/keyboardmockupv3jis7wa.png) (http://img62.imageshack.us/img62/8518/keyboardmockupv3jis7wa.png)
(click for original)

note: in this mockup I excluded the arrows. In my current opinion a button-hold to toggle between screen and keyboard seems the best solution.
Title: Re: usability: On-screen keyboard
Post by: salival on March 18, 2006, 03:32:34 PM
I have started a design document: link (http://www.rockbox.org/twiki/bin/view/Main/VKeyboardDesignProposal)

I need help in defining the character sets and rules for non-latin character sets.

Since I posted it in the rockbox wiki, feel free to edit the document. Although this document is intended as a functional design, if someone has suggestions on the technical implementation of this all don't hesitate to add them.

One friendly request though: Until we have come to a design which contain very little flaws, I would like you to put suggested changes in a note (ie. "Note: more columns") or at least tell which section you edited. (I know there's revisioning, this is IMO a little clearer and faster :) )

When the design is as complete as possible I plan to add it as a feature request.
Title: Re: usability: On-screen keyboard
Post by: moozooh on March 19, 2006, 01:02:13 PM
Yes but that is just for Russian ;) What about other languages using cyrillic like Belarusian, Serbian, Macedonian, Ukrainian, Bulgarian and many non-slavic languages from ex-soviet union countries? Are they inferior? ???
No, they aren't. Still, look at your desktop keyboard and think, why would you need all the characters of a given codepage appear simultaneously on it, wasting precious space? ;) I told you about Russian, and I assure you that the russian keyboard layout needs only 33 language-specific characters at max. If I want something else (eg. Macedonian), I'll switch to it, as I can (and must) do on my desktop. Got my point? :)
Title: Re: usability: On-screen keyboard
Post by: salival on March 19, 2006, 01:19:02 PM
At the moment I think we have two options for this issue:
1. create menu options on the keyboard to switch between the different cyrillic character groups (i.e. russian, macedonian, etc.)
-pro's: only one cyrillian keyboard layout needed.
-con's: you probably end up having more than 4 menu options, which means you need to scroll the menu (which should be supported)

2. create multiple cyrillic layouts. Which one is shown depends on the language the person has chosen for rockbox.
-pro's: only the character set the person typically would need is shown.
-con's: more layouts to define/design

edit: I forgot the option of one single layout which shows all characters.
-pro's: only one layout
-con's: a lot of characters on screen which the user might not use.
Title: Re: usability: On-screen keyboard
Post by: moozooh on March 19, 2006, 02:35:43 PM
-con's: more layouts to define/design
Actually, it's easier to design a layout that has fewer characters than otherwise. Moreover, the keyboard layout map is already defined (…many years ago, yeah) and is widely available.
I'd personally go with this option, as it is more reliable to the end user. Other than that, the only option I'd use is the whole UTF-8 range of practically usable symbols combined on the same screen (it would require very small font though, about 6 to 8 px in height)
Title: Re: usability: On-screen keyboard
Post by: salival on March 19, 2006, 03:29:48 PM
I read my reply again and noticed that the options 1 and 2 are basically the same. the only difference would be how the are accessed (automatic or menu selection.)

As you might have seen, the layout for the onscreen keyboard differs from a standard keyboard. Therefore a standard layout most likely has to be altered in some way. IMHO an alphabetic order is the best option.
Title: Re: usability: On-screen keyboard
Post by: phaedrus961 on March 20, 2006, 02:27:56 AM
I created a patch which allows customizable keyboard layouts. It's not quite what you've been discussing, but maybe some of you will find it useful? Let me know what you think of this approach.

http://www.rockbox.org/tracker/task/4856
Title: Re: usability: On-screen keyboard
Post by: salival on March 20, 2006, 04:42:09 AM
I haven't tested the patch, but:
The description says it creates new pages when needed. Can the user decide what's on every page, or does the code decides whether to use a new page or not?
If it doesn't, I think using a (multidimensional?) array (layout=[a,b,..,y,z][A,B,..,Y,Z]) would do the trick.

This can bring the "final solution" closer, since most of the code needed for the keyboard as described in the design document is already there.

What that solution IMO needs is: (hardcoded or not)
- definable layouts
- definable menu options
- multipage suport, where the pages are chosen by a menu.
- the option to allow a page to return to the default or previous page when a key is pressed. an example would be "shift", where the layout should return to lowercase when a character is entered.

I am not much of a programmer, but I think the following could work:
One array for the menu options where every position corresponds with a page. So, position [ 0 ], which for instance has the value "sym" (for symbols), corresponds to page 0 in the pagelayout array, which contains the set for the symbols page.

example (pseudo code):
array menu=[default]["shift"]["Caps"]["acc"]["sym"]; (the first position of the array is not shown in the menu.)
array layout=[a,b,c,...,x,y,z][A,B,C,...,X,Y,Z][á,ä,..][@,#,...]

There are some other points, but I think it would be better if those are discussed in a technical design doc, so they can be explained in greater detail.
Title: Re: usability: On-screen keyboard
Post by: markun on March 20, 2006, 08:14:41 AM
salival: I think your ideas are too complex. phaedrus961's patch works very well regardless of screen size and font size (which I think your probosal does not). Try it and tell us what you think of it.
Title: Re: usability: On-screen keyboard
Post by: salival on March 20, 2006, 09:27:12 AM
I don't think my proposal is much more complex then the current implementation or the mentioned patch.

The reason they might look complex might be of they way I explain things. (I have the tendency to make things look more complex than they are.)

I try to explain the "philosophy" of my proposal in a few sentences.

1. There should be one keyboard which is optimised on a per target basis, while the basic layout remains the same.
2. There should be one single keyboard that can easily be implemented in any program/plugin, while maintaining the feel of that program.
3. The keyboard should have support for non-latin character sets.
4. The keyboard should have support for multiple character subsets, like accented characters, which can be selected by a menu.

It is about the same concept as used in onscreen keyboards on game consoles.

That's the basic premisse.

The main functions needed for this are:
- multi page support, which is already available in the current keyboard.
- custom layouts, which are (partially) supported by the current keyboard and phaedrus961's patch.

All in all it is more a redesign than a completely new design. I always try to keep in mind how easy or difficult an option is to implement.
Title: Re: usability: On-screen keyboard
Post by: kenshin on March 20, 2006, 02:37:50 PM
I adjusted the wiki with a more complete Japanese character set. There are 10 small characters in both the Hiragana and Katakana, not just the 1 Hiragana and 4 Katakana that was already there.
Title: Re: usability: On-screen keyboard
Post by: salival on March 20, 2006, 03:15:26 PM
The hira/katakana layout now occupies 5 rows, which IMO is one too many. Either the small characters have to be fitted between the large ones or, as you suggested and in my eyes the best solution, use a shift button to show a layout with only the small characters.
Title: Re: usability: On-screen keyboard
Post by: phaedrus961 on March 20, 2006, 05:23:17 PM
salival:

As I said, my patch is not really what you're looking for. It was designed to allow the user to create their own layout which works for their screen size and font, instead of having pre-defined layouts. This way, everyone can have only the chars that they need/want and a layout that is intuitive for them. Granted, it's not as pretty as what you're suggesting, but I believe my approach offers the user much more flexibility.

Don't get me wrong, I don't think yours is a bad idea. It's just:
1. Mine should work for most everybody's needs.
2. It's already implemented.
Title: Re: usability: On-screen keyboard
Post by: salival on March 24, 2006, 06:10:57 AM
Korean and thai layouts added.

These need to be checked and verified. (link (http://www.rockbox.org/twiki/bin/view/Main/VKeyboardDesignProposal))

edit: Greek layout added. Also needs to be verified.
edit2: Russian and Macedonian layouts added. Needs verification
Title: Re: usability: On-screen keyboard
Post by: XavierGr on March 25, 2006, 10:05:52 PM
I think salival's idea for a unified virtual keyboard is very good, though very hard to implement on current rockbox code.

One major problem is the default font of rockbox. Is this font a unifont?

Will the virtual keayboard be drawed as a (1)"b/w picture" or as (2)"text"?

1. If we are going to display the keyboard as a picture we need many pictures for each language that contain the characters.
-Good thing with the picture idea is that we can make specific bmp files per target and per language. (a lot of pictures though)
-I still don't know though how the user will be able to input those characters, if the default font can't accept those....

2.Displaying the keybard with text is very difficult on this current state. Remember that we are restricted to font selection. We have the default font that is the same for all targets. Making a keybaord with this font is difficult to match all screens. E.g. On archos it will fit fine but then on H300 (or even iPod Video) it won't. It will be tiny.

So unless we change the font system I can't  see how we can make this work. Either we need pictures for each target or we have to load a different font.

Maybe some things are unclear to me but implementing the idea needs major rework on some parts.
Title: Re: usability: On-screen keyboard
Post by: salival on March 26, 2006, 06:08:47 AM
A text based keyboard has my preference, since that way you only have to define the layouts once. The font has to be the right size offcourse.
As for the font issue: I believe there are patches being developed for multiple font support.

If a bitmap based layout is chosen, sprites could be a solution. That way you only need one (or a few) image(s) for every target. On selection the character color could be inverted.

An other option would be to generate the bitmaps with somthing like the gd graphics library. (that would even be convienient wen using a sprite)

I am no programmer and as such have limited understanding of what is really possible or not. However, most of the things in my proposal are already there in one way or another and only need to be optimised. (multiple/custom layouts for example)

If someone thinks some parts of my proposal could be done easier or better, feel free to add a note or suggestion to the design document.
Title: Re: usability: On-screen keyboard
Post by: salival on March 26, 2006, 07:42:11 AM
Hebrew layout added, probably incomplete. Needs to be verified and completed.
Title: Re: usability: On-screen keyboard
Post by: salival on March 27, 2006, 04:15:57 AM
I submitted a feature request as the basics of the document are done, IMO.

That doesn't mean there won't be any changes, but the basics should remain the same. So if there are changes they will be minor. (like character layouts)
Title: Re: usability: On-screen keyboard
Post by: moozooh on March 27, 2006, 04:43:06 AM
edit2: Russian and Macedonian layouts added. Needs verification
Thanks for the effort. It's very nice of you to care about the cyrillic layouts. :)

Russian is ok (can't really verify Macedonian), but one thing I think I should say about the symbol layout is that —> ¯ <— symbol isn't used anywhere. On the other hand, some of us (not particularly Russians though) use three different types of dash ("-", "–", "—") that could be present (maybe). However, there are two "-"s that are the same. You could delete "¯" and one of the "-"s, and replace them with "–" & "—". Another symbol I haven't seen used at all is "¨". Should be replaced with the trademark ("™"), maybe? The rest is quite good.
Title: Re: usability: On-screen keyboard
Post by: salival on March 27, 2006, 05:31:18 AM
The russian character set was re-defined by Andrey Nosich, including adding the symbols.

If you think the symbol layout for russian contains obsolete characters or misses some, feel free to modify it. That way the layouts will be as good and complete as possible.
Title: Re: usability: On-screen keyboard
Post by: NuJew on April 25, 2006, 11:26:04 PM
Korean and thai layouts added.

These need to be checked and verified. (link (http://www.rockbox.org/twiki/bin/view/Main/VKeyboardDesignProposal))

edit: Greek layout added. Also needs to be verified.
edit2: Russian and Macedonian layouts added. Needs verification

kedmanee is widely used and most popular Thai keyboard, where as pattachote
is rare (mostly old fashioned,
Title: Re: usability: On-screen keyboard
Post by: rastaman01 on July 07, 2009, 10:57:17 AM
One suggestion I had as far as the virtual keyboard on the iPod targets:

It is just too complex to navigate through it... my suggestion would be to:
Use the scrollwheel to navigate left and right, and once the end of line is reached it should move to the next line (instead of using the scrollwheel to go up and down). 

The select button for the 'enter' comand is fine, and I guess the 'backspace' command when the file name is selected I can live with... but it would also be nice to use the << button to as backspace and >> as a forward-space. 

Menu to cancel and >|| to apply/accept work well.

Just a thought...