My first experience using an accessible touch screen device

Yes, you read correctly: An accessible touch screen device! This morning, I went to a retail store carrying mostly Apple products and had a look at the new iPhone 3G S that was released in Germany on Friday. Apple revealed during the WWDC keynote two weeks ago that it would have a built-in screen reader named the same as is included in Mac OS X: VoiceOver. This is a feature not available on the regular iPhone 3G, as its hardware capacity is insufficient.

I was not at all sure what to expect. From reading a bunch of posts on the VIPHone Google Group, I knew that people were going through a learning curve, a steep one at times. Up to now, something usable via a touch screen or touch-only keys would always mean a dead-end to me and other blind people. The iPhone 3G and the iPod Touch are not usable for me. Likewise, elevators that have keys you only need to touch, not press, to get toa different floor, are a real challenge. In fact I once tripped an alarm while trying to use such an elevator, alone int he cabin and touching the emergency button accidentally.

When I arrived at the store, I had already made arrangements with them to be allowed to take an in-depth look at the 3G S. As we went over to the iPhone stand, one of the sales assistants already knew how to turn on VoiceOver. Apple are documenting this in the regular iPhone user’s manual, no special docs needed. The assistant helping me turned it on, and a clear crisp voice came out of the built-in speakers. She was a bit confused by the changed gestures. I had done some reading, and took over from there.

And I must say this was an amazing experience! My fingers definitely need to get used to gestures such as flicking or tapping, or using a rotor. But having an iPod Nano 4th generation helped with that, since moving the finger over the screen like on a dialer felt most like tracking around the iPod’s click wheel. Even the sound the rotor makes is the same. 🙂

Responsiveness to gestures was amazing. I own an Nokia N82, which is to date probably the handset that reacts fastest to keyboard commands with the Talks or MobileSpeak screen readers, but the responsiveness on the iPhone beats that by lengths!

Finding my way around the iPhone’s UI took some getting used to. Traditional mobile screen readers, also like most Windows or Linux screen reader solutions, give the blind user a filtered view of the world, by default constrained to the focus location. Only on demand can one explore the screen using mouse emulation or similar techniques. On the iPhone, you interact with the real thing right from the start. You touch the screen in the lower half, somewhere on the right, and you’re told that the Safari or iPod symbol is there on the Home screen. You move your finger to the left, and you’re told what’s right next to it. To interact with the menu bar of the Phone app, you need to move your finger down to the bottom and move from left to right to hear the options such as “Contacts” or “Phone pad”. Yes, there are VoiceOver gestures to explore the screen top to bottom, left to right. You do this by flicking left to right anywhere, and the accessible controls are being walked one by one. But the interaction model is very close to the actual screen layout most of the time. This tremendously helped when I walked through a couple of applications with the sales assistant standing next to me. She could literally point me to the correct spot, and VoiceOver would speak what I needed to hear. Or she could give me verbal directions, and my finger would find the controls.

Typing is probably going to take the most adjusting. It is nothing like typing on the number pad of my N82. James Craig’s typing tip for VoiceOver on iPhone helps a lot: You look for the correct key with one hand, keep your finger there, then tap somewhere on the screen with another finger from the same or the other hand, and the character is input. Gladly, the keyboard doesn’t change position, and after a few letters I had a very good idea where each letter should be, and my typing sped up within 10 typed letters already. In addition, one can turn on word prediction/completion, which is another accessibility feature that can also aid people with motor impairments make typing easier. It plays nicely with VoiceOver.

This is by far not a comprehensive review or comparison. I couldn’t use many of the features since the SIM card in that exhibited model was locked, and I don’t have my own model yet.

Apple are speeding ahead and breaking down conventions in accessibility, or as Mike Calvo of Serotek wrote: They’re getting to the future first. They’re the first to include a screen reader for the blind on one of their mainstream models. Google are going to do something similar with their G1 efforts. The API is there, and some basic console work seems to be working already, but this is by far not as comprehensive as what Apple are doing. RIM also have an accessibility API, but from what I’m told, the screen reading solution that has been hinted every now and then over the past couple of weeks is going to cost extra money, which Apple’s solution does not. The traditional mobile accessibility solutions on Windows Mobile and Symbian S60 all require an additional payment of $200 to $350 for a screen reading solution, or in some cases even proprietary hardware that then costs $2000 or even more.

And this, of course, opens up other possibilities for future implementations of touch screen use cases, not just by Apple, but by other companies as well.

And one more bit of info: The gestures and touchy interface also come to VoiceOver in Snow Leopard with compatible new MacBooks with the multi-finger trackpad. So whenever a colleague tells me to lok for something in the top right quadrant of the screen, I can do that once I have Snow Leopard running on my MacBook. I’ll just put my finger there and let VoiceOver tell me what’s there!

Now my only problem is to get an iPhone. It would appear that my current contract doesn’t allow me to upgrade, since I upgraded it only recently, but too long before I knew the 3G S was coming. We’ll see how I get my hands on a device, it’s not freely available without contract in Germany.

My first touch screen experience was an amazing one!


104 thoughts on “My first experience using an accessible touch screen device

  1. Pingback: chechar (chess)
  2. I’m glad to hear Apple is making good progress in this area. High quality screen readers should be free or affordable on all devices. Thanks for the post Marco.

  3. Pingback: jcsteh (James Teh)
  4. Pingback: roatnek (kentaro)
  5. I bought an iPhone a few days ago. The learnign curve is steep but very satisfying, and the free screenreader compensates for the priciness of the phone. Thanks for the typing tip – that will help a lot.

    The one thing I’m missing is a GPS app that’ll give me turn by turn instructions, as WayFinder did on my old Symbian phone. TomTom for iPhone should be out this summer – let’s see how much that will cost…

  6. Hi Lynn,

    yes, a good GPS solution will definitely be appreciated! And I must say I have no high expectations right now given my poor experiences with Wayfinder Access on both an N73 and an N82. I found it to be mostly useless when giving me directions, since the turn by turn instructions came either far too early or far too late, and finding the correct house number on a street was almost always a hit and miss game, mostly off by far more than 5 meters, often even 10.

    In addition, the information given on my current location and the nearby points of interests are pretty useless when giving directions like “20 meters south east” if I don’t know which direction I’m actually facing. A 400 Euros I would not spend again.

    So I share your sentiments and hope that TomTom will be accessible on the 3G S with VoiceOver!

  7. Pingback: thirtysix (Nat)
  8. Pingback: noj (Jonathan Mao)
  9. Pingback: jim0203 (jim0203)
  10. Pingback: ucdit (UCanDoIT)
  11. Pingback: jim0203 (jim0203)
  12. Hello Marco,

    thank you so much for these insights! I would never have expected a touch screen device reacting so sleek, if you can’t even see anything on it!

    Now, I’m crossing fingers, that you will hopefully soon find a way out of your current contract to get one of these shiny new iPhones. 🙂

    I’m really excited about this.

    Nice to hear that Google has similar plans with the G1. I got mine on me all the time, so next time around, feel free to check it out. I’m sure there will be news. 🙂

    One last Question:
    You’re a power user. How do you think the iPhone will work for newbies?


  13. Hello, Marco.

    I’m from brazil and I’m not so acostumed speaking english. besides this little problem I read a lot of post in this blog and that’s amazing this one!

    I’ve read one week ago that apple that developed an acessible screem reader, and the most important: it’s included on the iphone 3gs.

    I will wait this product come to brazil to take a look on it and have my own impressions, but I think I will like.

    I found your e-mail to enter in contact with you in particular, but I didn’t find.
    If possible, I would talk to you about some tips to build a blog using wordpress because I’m having problems.

    If it’s possible:
    this is my messenger, if you can add me to change some ideas, I will be glad to you.

    thanks for atention, Lucas.

  14. Hi Sandra:

    You’re a power user. How do you think the iPhone will work for newbies?

    To be honest, I think it’ll work just fine. Primarily because a newbie isn’t biased, set in his/her ways of having used a Symbian or Windows Mobile-based phone for ages and being used to typing on a number pad or some QWERTY keyboards. For them, I think learning the iPhone will be totally different from a newbie learning a Symbian phone, but since there is no previous knowledge, it’ll feel natural I would think.

  15. @Lucas, you can find my contact info on the “About” page. I didn’t do anything special setting up WordPress, and when I stumbled I had the support forums to help me out.

  16. Hi Kaj,

    I used it with the German language voice at first, being from Germany and in a German store. But during that time, I also brought up an application that by default speaks in English. The voices used are those of Nuance, originally RealSpeak Solo, but nowadays called Vocalizer I believe. The English voice is that of Samantha, which you can also find in the Victor Reader Stream. The German voice is Yannick, the newest male voice offered by Nuance for this language. I would think that in Dutch, it would use one of those voices available from Nuance.

  17. Hi.
    well I do some unofficialarticles for the jared rimer network tech site.
    your article was linked to from that site.
    to be honest I have got those that say keyboards are still the way to go.
    however the touch screen thing does put out a few more things.
    the main one being that there will never have to be any translation to text, of say computer language.
    when someone says click the black circle in the middle of the screen on the access toolbar or something well its not like accessible at the moment but with this tech I could find it and tap it.
    will really improve accessibility.
    then there is gaming in general a touch screen with this function would really rock

  18. Excellent review, this is exciting stuff!

    I am wondering if you could receive the added benefit of a longer battery life by shutting off the screen but still have it’s touch interface. This would also be helpful for privacy reasons.

  19. Hi Eitan,

    I don’t have my iPhone yet (still!), but read on the VIPhone mailing list that people are using it with a screen that is completely darkened. So I think this is possible.

  20. Pingback: Touch Usability
  21. Pingback: chrisjf (ChrisJF)

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.