vkeyboard suggestion (corrected for dodgy diagram alignment)

[seani]

I've been using the vkeyboard a fair bit for the wps announcement patch.

Although it can seem clumsy at first, I think it's an excellent effort at a non-trivial problem.

However it suffers from having a couple of mode switches in there, and voicing the current character under selection / edited content seems like it would be difficult to follow for visually impaired users.


So a suggestion - quite long, sorry, but it's mostly diagrams;

This represents a charcell version of a new kbd_input screen, based on the approx 18 x 9 charcells available on a C240. The shaded blocks were just for alignment, obviously:




Some points:

1) The edit string is always in the centre of the screen. The focus alway stays on the character currently being edited at the centre of the edited string, everything moves around that point.

2) Left and right scroll the whole input string left and right so that the character under consideration is always at the centre of the screen.




3) Up and down scroll up and down through the available strip of characters. There's no select to pick the character; the character is whatever it is when you press left or right to bring a different character into focus




4) Scrolling "past" either end adds an implicit character to either end. Off the end and one is added to the end, off the beginning and one is inserted at the beginning. If you move off that character without selecting anything, the insert is "undone"




5) A short select / play anywhere inserts a new blank character. Again, moving off that space without selecting anything will undo that insert action:



6) A separate key may be required to do a deletion - not certain what that is / should be, but deleting the character in focus moves the edited string and position in the selectable character list as you would expect.

Another option would be to have the "null" character right at the start of the list. So you fast scroll right to the start with the up key, the null character is in focus, and then press left or right, and the deletion effected.




7) A long select commits, and power (or whatever) cancels.


8 ) it would also be particularly well suited to entering time and date, pin numbers etc; maybe some sort of input mask characters would be handy to indicate separator such as "/" and ":" to skip when scrolling left and right.




9) Although the selection is from a one-dimensional list as opposed to a two dimensional array, I think this would be made up for by the speed at which you could scroll through alpha / numeric chars. Maybe slightly worse for punctuation as there's no "natural" order. Not certain. Scrollwheel users may be best off.


10) Custom keymaps become simple lists of characters; omit / repeat / group as you like.


11) I think voicing becomes more consistent:

a) Announce the whole string once when you enter the editor

b) Announce the character under the cursor. This is always part of the edited text, you never need to establish context or make a mental flip between the entered text and available keyboard as this method of entry is essentially modeless.

c) Voice the new string if you confirm, or "CANCELLED" if you don't


12) Offsetting the focus character gives an opportunity for short on-screen explanatory text (although of very limited use on smaller screened targets.




13) Issues with targets with smaller screens, or the fact that there's no "natural" sequence for punctuation etc. could be mitigated in one of two ways:

a) by having a section of the characters scrolling vertically, but "snaking" off to scroll horizontally at the extremes. That way you can fit in a decent preview of the characters available in either direction:



I think this would be quite effective when in motion.


b) And / or by reserving a special character that always occupies the "end" of the available character sequence. Hitting select when this character is in focus would not insert a new character, but would cycle through the available sets of characters - Upper, Lower, Numeric, punctuation etc. These could be soft delimited in the available character string map:

[[Saint]]

seani,


Perhaps add some bitmap screenshots/mockups instead....this looks totally warped without a fixed width font (which I assume quite few people are using in their web browsers).

It took me some time to figure out that the alignment I'm seeing isn't intentional.

EDIT: You talk about adding another button instance for "delete"...the iPods don't *have* a spare button in the vkeyboard.

EDIT2: I'm really not 100% convinced this would speed anything up...I actually think it would be quite awkward only having 1 dimension to scroll through...have you seen how many chars are in the default (not using a .kbd file) vkeyboard? I certainly wouldn't like to have to scroll through them all in a 1 dimensional list....personally.



[St.]

[seani]

seani,


Perhaps add some bitmap screenshots/mockups instead....this looks totally warped without a fixed width font (which I assume quite few people are using in their web browsers).

It took me some time to figure out that the alignment I'm seeing isn't intentional.



[St.]

Hmm, odd. I'm using the "tt" tag in the post there to indicate it's fixed width, and that's just how it renders here (Firefox / Ubuntu 10.04). It took long time to do, so that's a real irritation.

[[Saint]]

Hmm, odd. I'm using the "tt" tag in the post there to indicate it's fixed width, and that's just how it renders here (Firefox / Ubuntu 10.04). It took long time to do, so that's a real irritation.

Just tried it in IE8 and it looks a *lot* better there
Google Chrome however makes it look quite warped indeed.


[St.]

[seani]

seani,


Perhaps add some bitmap screenshots/mockups instead....this looks totally warped without a fixed width font (which I assume quite few people are using in their web browsers).

It took me some time to figure out that the alignment I'm seeing isn't intentional.

EDIT: You talk about adding another button instance for "delete"...the iPods don't *have* a spare button in the vkeyboard.

EDIT2: I'm really not 100% convinced this would speed anything up...I actually think it would be quite awkward only having 1 dimension to scroll through...have you seen how many chars are in the default (not using a .kbd file) vkeyboard? I certainly wouldn't like to have to scroll through them all in a 1 dimensional list....personally.



[St.]

There are two aims:

1) to make the method of entry generally easier for visually impaired users. At the moment, I think the way that the keyboard is voiced makes is confusing to understand. This is partly because you can either be voicing information from the edited line, or information from the keyboard as you choose a character, and making that distinction is difficult on the basis of announcements.


2) to make the general method easier and more consistent.

I don't think it would take long for it to work in that regard. 4 keys and select would replace the current 4 keys for navigating the keymap, 2 keys for moving left and right through the text and select to insert the chosen character. If anything it should free keys.

I think this would be a lot more intuitive in use; actually navigating to the chosen character for selection is only one part of the experience of editing text. If the *way* that you're meant to achieve something isn't intuitive, that all adds to the time you'll take to do it. The current method means flipping between different sets of controls in a fairly arbitrary (although well defined) way.


3) No way of knowing until I make it work somewhere, but I believe it will still turn out to be faster with a 1d list that scrolls with repeat and appropriate feedback against a 2d map that requires changes in direction. I understand your concern with the number of characters in there by default, but I think that can be mitigated against.


Thanks for the feedback!

[JdGordon]

I *really* Like this idea! I'm not sure how well it owuld work on non-wheel targets (presumably better than the current keyboard though)

point 13 about the snaking is a very interesting idea but I'm not sure it wouldn't make it more confusing. Also it isnt a very good use of screen space. (on the small screen targets anyway)

having a paging system is probably more important than a delete key, so my suggestion would be that it only needs (minimum), left/right, up/down (or wheel), ok/cancel, "change page".

By pages I mean a small set of common chars (lower case, upper, number, punctuation, symbols, etc), where there is enough screen space we can display the page number/total.

If there is no spare key for del we can maybe add a new page which has "keys" for del, insert, copy/paste?

One thing to consider is touchscreens, if we can do very simple up/down sliding then it should be workable with a few on screen buttons.


For the speed argument, I think that the vast majority of uses is just naming a file which cuts alot of possible letters out.

[seani]

I think the wheel targets are a no brainer - certainly the characteristics of the scroll wheel on an e280 make me think it would be a lot faster. Other acceleration characteristics for non wheel targets and a lot of small details would close the gap there I think.

I'm now thinking in terms of short select for insert and short power for delete, with long versions for accept and cancel. That just leaves a key to find for "cycle chosen character page" to alter the current vertically scrolling list.

If you can find a cycle key for all targets, the snaking may be unnecessary anyway (although I might just try for fun)

[JdGordon]

My suggestion would be "Bloody go for it!" and we can figure out the buttons later. the ondio might cause problems, but I think all others should have enough buttons.

[soap]

Very nicely presented, and fleshed out, proposal.

[seani]

Well I had a go at a quick and dirty patch to feel things out a bit. It *was* actually surprisingly easy to bend the current kbd_input method in an unpleasant way to at least get an idea, but I wandered down a few rabbit holes and wasted a load of time.

The disposable frig I've attached is really designed to work on an E280 only, although other screens could be made to work if the attached keyboard definition was amended to suit.

You need to load the keyboard file before trying.


1) The "#"  and "<" are symbols reserved at the start and end of each list. Hitting Select on "<" will backspace and delete the character to the left of the cursor on the edit line, "#" will flip pages.

Yes, that means you can't insert those characters, but remember this is just a convenience to play with the feel of the thing. Special chars / symbols await in the "real" version.


2) Wheel scroll up and down advances through the character selected, changing the current edit character


3) Cursor left and right moves left and right through the edit line.


4) Enter will insert the character currently under selection in the picker at the current point in the edit string


5) You can't save your changes!



Although it's been interesting to think through some of the issues and get a bit more familiar with the code, a few things have become very apparent.

1) The block cursor moving through the picker and the character changing in the edit string to the left of the insert cursor in synch. makes for a disjointed experience. Attention is constantly flicking between the two; up to the picker to see where the block cursor is, down to the edit line to see what the change looks like.

2) The mismatch between the block cursor operating *on* a specific character and the edit cursor *between* two characters with an implicit focus character is also distracting.


But both of these issues are artefacts of it being a sandbox and not real code; I'm still convinced that a single point of intersection between picker and editor will play much more nicely; one point of focus.


3) The method for insertion in this patch is meh. Hitting Enter selects and inserts the current focus char on the picker and then wheel-scrolling changes it, and this doesn't flow well at all

4) Inserting at the end of a string is also counterintuitive; you need to cursor to the end, hit Select to effectively insert a copy of the last character and then scroll to change.


But again, 3 and 4 are explicitly addressed in a different way in the originally post (insertion of a "null" character and changing or moving off) and I think doing this properly and getting the detail right will sort it out.


I also think that it would be easy to accommodate more "special" actions in the picker to address the issues of restricted controls where required; accept entry, cancel entry, delete-to-end, delete-to-beginning, restore-to-original, etc. etc. Also pretty easily taken care of in the keyboard definition.


So now worth spending the time to do it properly I think.


(files may need extensions renaming - only .txt accepted)

[JdGordon]

bump wondering if you are still working on this or someone else should pick it up?

[gbl08ma]

I'm also really interested in seeing a vkeyboard like this one (not the prototype made by seani where many things aren't as described), to get an idea of its usability. From thinking how it would be without seeing it in action, I think it would make text input much more practical.

[seani]

bump wondering if you are still working on this or someone else should pick it up?

Yes, I've been trying it out with decent success on a C240, E280 and Clip+ (sim. and target), but I've stalled a bit due to a fairly crippling amount "Real Life" work - I find I need to have a good few hours run at a time to make decent progress.

I've tried a few different methods of flipping between character groups, inserting, deleting etc. and there don't seem to be any problems on even the players with the most restricted controls.


The control model I've gone for is:


LEFT and RIGHT:
Move the focus character left and right through the input string. This moves the input string "around" the focus character which remains in the  middle, and this is highlighted by one of: an inverse square, a solid border or "chevrons" depending how I feel.

When you move left and right and bring a char into focus, it synchronises the vertical keyboard correctly. I thought this might feel "flickery" but not so. Even so I'm thinking of fading out characters in the keyboard progressively where the display would support it.


UP and DOWN
Move up and down through the characters in the currently selected keyboard changing the one at the focus


I then use the DEL and SEL buttons, but always as part of a two keystroke sequence, as opposed to simultaneously, and am using variations on this scheme;

SEL -> RIGHT
Insert an empty character to the right of the current focus character.

SEL-> LEFT
Insert an empty character to the left of the current focus character

SEL -> UP
Cycle to the next keyboard set in the sequence (each keyboard set is just a line of characters. There are no "special" characters any more. I have Upper, Lower, Numeric and Symbols as a test)

SEL -> DOWN
Cycle to the previous keyboard set in the sequence

SEL -> SEL
Accept the current string

DEL -> DEL
Delete the current focus character

DEL -> SEL
Abandon the current edit


This still leaves DEL->LEFT, DEL->RIGHT, DEL->UP, DEL->DOWN - possibly delete to end, beginning etc. although I haven't tried those out yet.


Moving "past" the end of the current string performs an implicit SEL->RIGHT to allow chars to be added to the end.


I'm toying with adding an extra explicit SEL press to confirm a change if UP and DOWN have changed the character. This isn't necessary, but I just keep *expecting* to do it, rather than just moving off when it's correct.


I've also tried a number of ways of flowing characters straight-line or round-the-corner as above and a number of methods of indicating focus etc. Not really any clear winners, more personal taste.

One thing I've found is that players without a scrollwheel are a lot better than I thought they'd be - really quite fluid - but the e280 with a scrollwheel isn't quite as good as imagined (although I have a few tweaks pending and it's *all* details with this). Part of the problem for me is that I'm using the current vkeyboard a lot more so my muscle memory for it is a lot better, and it's skewing my judgement a bit.


I haven't put any new voicing in for accessibility yet which was one main driver for me, that's next up.


As far as other efforts are to do this are concerned, I'd like to post at least one starter patch for discussion, but equally wouldn't want to stop anyone from doing it - I'm not getting through it as quickly as I'd hoped.