Home

Usability and RIA user test video

Monday, 21 September 2009

Joshue O Connor was invited by the Web Accessibility Initiative (WAI) to give a presentation on accessibility, WAI-ARIA and Rich Internet Applications (RIA) at the recent Techshare 2009 pre-conference workshop.

The workshop was entitled “Web Applications Enabled with WCAG 2.0 and WAI-ARIA" and involved a video presentation of an informal user test (with friend and colleague Stuart Lawler) of a well known RIA (Google Reader).

Watch the video!

The video footage got some great feedback from people at the conference so we have decided to publish the results on our site. Download the "User Testing of RIA" Video [WMV,104.5 Mb]. This video is not captioned.

The video presentation aimed to examine the usability of rich internet applications as technologies such as WAI-ARIA, and APIs like Google's Axsjax - become an increasingly accepted feature of the developer landscape. How will users of Assistive Technology respond to these changes? Will they have to learn more complex ways of navigation or different modes of interaction?

The presentation and video hoped to demonstrate the importance of user testing RIAs to ensure that the reality lives up to the promise.

Methodology

The user test took place in the CFIT usability lab and is very informal. There is open discussion between test facilitator and test participant as issues arise while the user interface and functionality are explored.

While the purpose of the test was originally to explore the use of WAI-ARIA in the wild, in this case during the test it was found that the Google Reader contains only a small amount of WAI-ARIA functionality. Therefore is best to view the test as an examination of the user experience for a blind screen reader user when using an RIA, and not solely as an exploration of WAI-ARIA.

The test participant used JAWS 10, Internet Explorer 8 and Firefox version 3.5 and 3.2.

Test Outcomes and Observations

• Labeling of elements was generally poor. This is totally avoidable and a surprise considering the interface was developed by Google.
• Google reader is an interesting example of a great idea that is poorly executed. In terms of intuitive RIA usability it does not perform well. When the WAI-ARIA enhanced version is activated there is nothing to indicate what you have to do to get to the extra functionality. Even if the link for ‘Click here for ARIA enhanced interface” had the extra text “Press ? for list of commands” that would have made the whole experience easier.
• However, once the user has climbed the steep learning curve and has been orientated it does come into its own – but that is a steep curve!
• The keyboard command to give the user the list of short cut keys in Google Reader does not work with JAWS 10, in virtual PC mode. Or in the common mode used by many blind screen reader users. You must de-activate the PC Cursor (Insert+Z) and toggle between different modes to successfully interact with the interface itself and to take advantage of the Axsjax/ARIA stuff. Way too complicated! But is it the shape of things to come, as other RIAs follow suit?
• Will the AT user may be put off using these kind of applications due to this switching of modes? Very likely, initially anyway.
• Problems with help: Insert F1 – added to virtual buffer. Text not placed in virtual buffer- so where is it coming from? Therefore there is an inconsistency with how screen reader help and support works (with JAWS) and any other screen readers that use a virtual buffer.
• Stuart comments It’s a different experience – turning off the virtual cursor and I would like to see a way of interacting with the virtual cursor were very interesting and gave an insight into how the user feels like he is moving into a new mode of browsing.
• Interesting behaviour – in different browsers – and Virtual Cursor settings gave certain elements focus and others where ignored/invisible.
• Use of the text “Add a subscription button on the left” is useless.
• Also notifying the user somewhere that the Virtual Cursor had to be off – is vital.
• The average user will not be able to deal with changing Virtual Cursor – on and off – on top of the complexity of learning extra commands.
• Focus issues – the screen reader user cannot read character by character easily, due to virtual cursor being turned off.
• Totally Different experience – or realm of browsing.
• Will layered commands be an option to deal with more complex Web Applications (such as in JAWS 11).
• In Internet Explorer when the virtual cursor is off the ARIA/Axsjax stuff won’t work. IN Firefox it is working, but the buttons won’t work properly with the Virtual Cursor on.
• JAWS didn’t go into ‘Forms mode’ when the “add a subscription” button was activated in Firefox.