Biased testing and micro-coaching

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Biased testing and micro-coaching

Richmond Mathewson via use-livecode
I think my experience of the last two days has taught me something - I have been micro-coaching my friends when they try my app.

Just the littlest input, like saying "oh, just press the button again to submit" comes so easily and, apparently mucks up testing entirely.

While the harm will be minimal in this case, I can see where it could be disastrous for a large company.

What steps do you guys follow for accurate testing when you don't have a budget for proper official testing procedures?

Sent from my iPhone
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
Jonathon,
I feel your pain. In my case, I was initiated by my students and very quickly learned how to ask the questions a newbie would ask. I also paid small amounts to graduate students to get their feedback.

One of my very effective testers is my grandson, my wife, any of my colleagues who might be enticed to use the app. Looking over the shoulder while these folks use the app can be very illuminating.

In summary:
1. Ask friends and relatives first.
2. Perhaps there would be volunteers from the live ode users group.
3. Hire high school students who might have a tech interest. Look over their shoulders as they use the app and dialog to themselves. Actually watching users is invaluable.

Good luck,
Bill P


William Prothero
http://es.earthednet.org

> On Jul 7, 2017, at 4:57 AM, Jonathan Lynch via use-livecode <[hidden email]> wrote:
>
> I think my experience of the last two days has taught me something - I have been micro-coaching my friends when they try my app.
>
> Just the littlest input, like saying "oh, just press the button again to submit" comes so easily and, apparently mucks up testing entirely.
>
> While the harm will be minimal in this case, I can see where it could be disastrous for a large company.
>
> What steps do you guys follow for accurate testing when you don't have a budget for proper official testing procedures?
>
> Sent from my iPhone
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
On 07/07/2017 07:29 AM, prothero--- via use-livecode wrote:

> Actually watching users is invaluable.

Absolutely.
And try to resist the urge to help them along.
And take notes about where they hesitate or stumble.

As a QA engineer, I find that when I'm faced with a new application I
have about two weeks where I can reasonably find workflow problems.
During that period I try to write down every nitpicking thing I can find
and make notes about where I find myself unsure about what comes next,
etc. After that time I have become so used to the way it's supposed to
work that it's hard to step outside the box and think like a new user. I
get into the developer mindset then, and while that's essential for
finding deeper bugs it means missing some UX issues.

--
  Mark Wieder
  [hidden email]


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
In reply to this post by Richmond Mathewson via use-livecode
Jonathan,

Here’s a link to my lesson outline when I teach my students about conducting user evaluations of software. It’s still a bit sparse and needs to be fleshed out, but I include some links to a couple of really good articles on evaluation techniques, including one by our very own Jacque Gay.

http://livecode.byu.edu/userevals/UserEvals.php

Devin

On Jul 7, 2017, at 8:29 AM, prothero--- via use-livecode <[hidden email]<mailto:[hidden email]>> wrote:

Jonathon,
I feel your pain. In my case, I was initiated by my students and very quickly learned how to ask the questions a newbie would ask. I also paid small amounts to graduate students to get their feedback.

One of my very effective testers is my grandson, my wife, any of my colleagues who might be enticed to use the app. Looking over the shoulder while these folks use the app can be very illuminating.

In summary:
1. Ask friends and relatives first.
2. Perhaps there would be volunteers from the live ode users group.
3. Hire high school students who might have a tech interest. Look over their shoulders as they use the app and dialog to themselves. Actually watching users is invaluable.

Good luck,
Bill P


William Prothero
http://es.earthednet.org

On Jul 7, 2017, at 4:57 AM, Jonathan Lynch via use-livecode <[hidden email]> wrote:

I think my experience of the last two days has taught me something - I have been micro-coaching my friends when they try my app.

Just the littlest input, like saying "oh, just press the button again to submit" comes so easily and, apparently mucks up testing entirely.

While the harm will be minimal in this case, I can see where it could be disastrous for a large company.

What steps do you guys follow for accurate testing when you don't have a budget for proper official testing procedures?

Sent from my iPhone
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Devin Asay
Director
Office of Digital Humanities
Brigham Young University

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
In reply to this post by Richmond Mathewson via use-livecode
In  another era, Heathkit reportedly used secretaries to test the
instructions to their electronics kits.

They found that they could only use any given secretary three times, as she
would pick up enough doing the first three to overcome errors in future
runs without being stopped by them . . .

On Fri, Jul 7, 2017 at 7:29 AM, prothero--- via use-livecode <
[hidden email]> wrote:

> Jonathon,
> I feel your pain. In my case, I was initiated by my students and very
> quickly learned how to ask the questions a newbie would ask. I also paid
> small amounts to graduate students to get their feedback.
>
> One of my very effective testers is my grandson, my wife, any of my
> colleagues who might be enticed to use the app. Looking over the shoulder
> while these folks use the app can be very illuminating.
>
> In summary:
> 1. Ask friends and relatives first.
> 2. Perhaps there would be volunteers from the live ode users group.
> 3. Hire high school students who might have a tech interest. Look over
> their shoulders as they use the app and dialog to themselves. Actually
> watching users is invaluable.
>
> Good luck,
> Bill P
>
>
> William Prothero
> http://es.earthednet.org
>
> > On Jul 7, 2017, at 4:57 AM, Jonathan Lynch via use-livecode <
> [hidden email]> wrote:
> >
> > I think my experience of the last two days has taught me something - I
> have been micro-coaching my friends when they try my app.
> >
> > Just the littlest input, like saying "oh, just press the button again to
> submit" comes so easily and, apparently mucks up testing entirely.
> >
> > While the harm will be minimal in this case, I can see where it could be
> disastrous for a large company.
> >
> > What steps do you guys follow for accurate testing when you don't have a
> budget for proper official testing procedures?
> >
> > Sent from my iPhone
> > _______________________________________________
> > use-livecode mailing list
> > [hidden email]
> > Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> > http://lists.runrev.com/mailman/listinfo/use-livecode
>
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>



--
Dr. Richard E. Hawkins, Esq.
(702) 508-8462
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
In reply to this post by Richmond Mathewson via use-livecode
Thank you, Devin!

Sent from my iPhone

> On Jul 7, 2017, at 12:01 PM, Devin Asay via use-livecode <[hidden email]> wrote:
>
> Jonathan,
>
> Here’s a link to my lesson outline when I teach my students about conducting user evaluations of software. It’s still a bit sparse and needs to be fleshed out, but I include some links to a couple of really good articles on evaluation techniques, including one by our very own Jacque Gay.
>
> http://livecode.byu.edu/userevals/UserEvals.php
>
> Devin
>
> On Jul 7, 2017, at 8:29 AM, prothero--- via use-livecode <[hidden email]<mailto:[hidden email]>> wrote:
>
> Jonathon,
> I feel your pain. In my case, I was initiated by my students and very quickly learned how to ask the questions a newbie would ask. I also paid small amounts to graduate students to get their feedback.
>
> One of my very effective testers is my grandson, my wife, any of my colleagues who might be enticed to use the app. Looking over the shoulder while these folks use the app can be very illuminating.
>
> In summary:
> 1. Ask friends and relatives first.
> 2. Perhaps there would be volunteers from the live ode users group.
> 3. Hire high school students who might have a tech interest. Look over their shoulders as they use the app and dialog to themselves. Actually watching users is invaluable.
>
> Good luck,
> Bill P
>
>
> William Prothero
> http://es.earthednet.org
>
> On Jul 7, 2017, at 4:57 AM, Jonathan Lynch via use-livecode <[hidden email]> wrote:
>
> I think my experience of the last two days has taught me something - I have been micro-coaching my friends when they try my app.
>
> Just the littlest input, like saying "oh, just press the button again to submit" comes so easily and, apparently mucks up testing entirely.
>
> While the harm will be minimal in this case, I can see where it could be disastrous for a large company.
>
> What steps do you guys follow for accurate testing when you don't have a budget for proper official testing procedures?
>
> Sent from my iPhone
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
> Devin Asay
> Director
> Office of Digital Humanities
> Brigham Young University
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
In reply to this post by Richmond Mathewson via use-livecode
http://hyperactivesw.com/resources_testing.html


On July 7, 2017 6:59:52 AM Jonathan Lynch via use-livecode
<[hidden email]> wrote:
>
> What steps do you guys follow for accurate testing when you don't have a
> budget for proper official testing procedures?

--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com



_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
Thank you, Jacqueline

Sent from my iPhone

> On Jul 7, 2017, at 1:39 PM, J. Landman Gay via use-livecode <[hidden email]> wrote:
>
> http://hyperactivesw.com/resources_testing.html
>
>
>> On July 7, 2017 6:59:52 AM Jonathan Lynch via use-livecode <[hidden email]> wrote:
>>
>> What steps do you guys follow for accurate testing when you don't have a budget for proper official testing procedures?
>
> --
> Jacqueline Landman Gay         |     [hidden email]
> HyperActive Software           |     http://www.hyperactivesw.com
>
>
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
From reading these, it looks like my basic steps are these:

1. Make changes to the app

2. Test for usability myself a dozen times, trying things in different orders  and in different ways to make it fail

3. Have my testers, which is really about 3 family members, test it to make it fail

No coaching, no hints

Directly observe their tests very closely

Make notes on any moments of confusion, even if they minor

Interview them, asking what they were thinking at each step

Adjust the help file and add hints - and test those as well

4. Fix as needed and retest

5. Publish

6. Try to find virgin testers for next time, varying in age and mindset

Does that sound about right?

Sent from my iPhone

> On Jul 7, 2017, at 1:53 PM, [hidden email] wrote:
>
> Thank you, Jacqueline
>
> Sent from my iPhone
>
>> On Jul 7, 2017, at 1:39 PM, J. Landman Gay via use-livecode <[hidden email]> wrote:
>>
>> http://hyperactivesw.com/resources_testing.html
>>
>>
>>> On July 7, 2017 6:59:52 AM Jonathan Lynch via use-livecode <[hidden email]> wrote:
>>>
>>> What steps do you guys follow for accurate testing when you don't have a budget for proper official testing procedures?
>>
>> --
>> Jacqueline Landman Gay         |     [hidden email]
>> HyperActive Software           |     http://www.hyperactivesw.com
>>
>>
>>
>> _______________________________________________
>> use-livecode mailing list
>> [hidden email]
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
That's a good summary. It's tedious sometimes but essential. The hardest
part by far is keeping your mouth shut. If I had a one-way mirror in a
sound-proofed room I'd use that. Another method might be to have the
user share their screen and turn off your microphone, but screen sharing
is not easy on mobile apps.

On 7/7/17 2:49 PM, Jonathan Lynch via use-livecode wrote:

>>From reading these, it looks like my basic steps are these:
>
> 1. Make changes to the app
>
> 2. Test for usability myself a dozen times, trying things in different orders  and in different ways to make it fail
>
> 3. Have my testers, which is really about 3 family members, test it to make it fail
>
> No coaching, no hints
>
> Directly observe their tests very closely
>
> Make notes on any moments of confusion, even if they minor
>
> Interview them, asking what they were thinking at each step
>
> Adjust the help file and add hints - and test those as well
>
> 4. Fix as needed and retest
>
> 5. Publish
>
> 6. Try to find virgin testers for next time, varying in age and mindset
>
> Does that sound about right?
--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
Well, 3 out of 3 people who tested my app from this list got stuck signing up, but no one got stuck when I was there to prompt them to use the Universal Options button.

So, I have to discipline myself to keep my mouth shut. Everything depends on it.

This is my second time reading your article, Jacqueline- but this time I appreciate it much much more!

Sent from my iPhone

> On Jul 7, 2017, at 4:09 PM, J. Landman Gay via use-livecode <[hidden email]> wrote:
>
> That's a good summary. It's tedious sometimes but essential. The hardest part by far is keeping your mouth shut. If I had a one-way mirror in a sound-proofed room I'd use that. Another method might be to have the user share their screen and turn off your microphone, but screen sharing is not easy on mobile apps.
>
> On 7/7/17 2:49 PM, Jonathan Lynch via use-livecode wrote:
>>> From reading these, it looks like my basic steps are these:
>> 1. Make changes to the app
>> 2. Test for usability myself a dozen times, trying things in different orders  and in different ways to make it fail
>> 3. Have my testers, which is really about 3 family members, test it to make it fail
>> No coaching, no hints
>> Directly observe their tests very closely
>> Make notes on any moments of confusion, even if they minor
>> Interview them, asking what they were thinking at each step
>> Adjust the help file and add hints - and test those as well
>> 4. Fix as needed and retest
>> 5. Publish
>> 6. Try to find virgin testers for next time, varying in age and mindset
>> Does that sound about right?
> --
> Jacqueline Landman Gay         |     [hidden email]
> HyperActive Software           |     http://www.hyperactivesw.com
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
In reply to this post by Richmond Mathewson via use-livecode
Jonathan,

I second bill's approach of watching folks use the app. Years of educational software creation taught me this. I would always make friends with a local teacher that was into tech and they usually were happy to get a period to try something on the kids if it only took one period to do in the lab and was something they thought good first. Things were so self evident on what just worked and what crashed and burned. I really found that the designs that were forced (usually by marketing) always crashed and burned, but the just good ideas that came out of what was it we were really trying to do somehow avoided most all the little design eddies that folks would get a little hung up by. But watching you could quickly see those eddies w.o having to do hard core testing. Sadly this is hard to do for free in a school anymore but hiring some kids or adults will do.

It's funny as I've found the same thing with exhibit design. I would always spend a few hours just watching folks after we finished an exhibit. I found it really invaluable to find the little issues and the big ones and you could see so easily what folks were getting and what they were not, what they were looking and and not looking at and how they felt about the exhibit in the whole. Many of these exhibits got very expensive summative evaluations and I found that my just watching observations were right in line with heavy testing and many times a bit more complete and useful for potentially fixing things and learning for the future.

Cheers

Jeff

> On Jul 7, 2017, at 1:53 PM, [hidden email] wrote:
>
> Jonathon,
> I feel your pain. In my case, I was initiated by my students and very quickly learned how to ask the questions a newbie would ask. I also paid small amounts to graduate students to get their feedback.
>
> One of my very effective testers is my grandson, my wife, any of my colleagues who might be enticed to use the app. Looking over the shoulder while these folks use the app can be very illuminating.
>
> In summary:
> 1. Ask friends and relatives first.
> 2. Perhaps there would be volunteers from the live ode users group.
> 3. Hire high school students who might have a tech interest. Look over their shoulders as they use the app and dialog to themselves. Actually watching users is invaluable.
>
> Good luck,
> Bill P


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
In reply to this post by Richmond Mathewson via use-livecode
You don’t have to keep your mouth shut.  In fact, you should be vocal, but you want your tester to be more vocal.

Generalized suggestions from past experience…

- First, explain to the tester in general terms what your app does.  Avoid getting into operating specifics.

- Tell the tester you want them to verbalize as much as possible their thought process when encountering each screen/interaction process.  You goal is to get a sense of what the tester is thinking and why, not just whether or not they exhibit expected behavior (you will have to prompt the tester repeatedly to explain their thinking without scolding or leading).

- Explain to your tester there are no right or wrong actions/answers while using your app — you are trying to observe real world behavior and initial responses to what they see/experience, and their interaction (or lack of it) in no way reflects on their “intelligence”.

- Give the tester one or more planned tasks to complete.  Remind them to describe their thinking as they attempt to complete each task.

- Each time the tester is shown a new screen/process, ask them what they think they need to do at that point. Ask why.  Keep all requests/comments neutral, never correct the tester.  If their response doesn’t fit with your intended behavior, ask the tester what they would suggest to improve interaction/outcome, or make the process more intelligible.  Avoid allowing the tester complete too many tasks in a row without describing their thought process.

- If the tester can’t figure out how to proceed to a next step, give them a hint (if possible) and determine if they are able to understand the interaction. Again, ask for suggestions on what could be improved.  Ask why.

- Rinse and repeat.

- Ask the tester at the end of the test what they felt was the biggest issue with the app.  Ask the tester to reiterate how they would correct the problem.  Review your list of problems encountered by the tester to confirm your understanding of the issues.

- Record/note all responses.  Keep written notes at a minimum, use audio and/or video recording to collect more detailed/nuanced responses. In an ideal world, you would record the tester and the screen they interact with concurrently.

Hope this helps.

Regards,

Scott Rossi
Creative Director
Tactile Media, UX/UI Design



> On Jul 7, 2017, at 12:49 PM, Jonathan Lynch via use-livecode <[hidden email]> wrote:
>
> From reading these, it looks like my basic steps are these:
>
> 1. Make changes to the app
>
> 2. Test for usability myself a dozen times, trying things in different orders  and in different ways to make it fail
>
> 3. Have my testers, which is really about 3 family members, test it to make it fail
>
> No coaching, no hints
>
> Directly observe their tests very closely
>
> Make notes on any moments of confusion, even if they minor
>
> Interview them, asking what they were thinking at each step
>
> Adjust the help file and add hints - and test those as well
>
> 4. Fix as needed and retest
>
> 5. Publish
>
> 6. Try to find virgin testers for next time, varying in age and mindset
>
> Does that sound about right?
>
> Sent from my iPhone
>
>> On Jul 7, 2017, at 1:53 PM, [hidden email] wrote:
>>
>> Thank you, Jacqueline
>>
>> Sent from my iPhone
>>
>>> On Jul 7, 2017, at 1:39 PM, J. Landman Gay via use-livecode <[hidden email]> wrote:
>>>
>>> http://hyperactivesw.com/resources_testing.html
>>>
>>>
>>>> On July 7, 2017 6:59:52 AM Jonathan Lynch via use-livecode <[hidden email]> wrote:
>>>>
>>>> What steps do you guys follow for accurate testing when you don't have a budget for proper official testing procedures?
>>>
>>> --
>>> Jacqueline Landman Gay         |     [hidden email]
>>> HyperActive Software           |     http://www.hyperactivesw.com
>>>
>>>
>>>
>>> _______________________________________________
>>> use-livecode mailing list
>>> [hidden email]
>>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
In reply to this post by Richmond Mathewson via use-livecode
It sounds like a little bit of direct, intensive observation is worth a lot of testing a a distance.

Thanks Jeff

Sent from my iPhone

> On Jul 7, 2017, at 5:31 PM, Jeff Reynolds via use-livecode <[hidden email]> wrote:
>
> Jonathan,
>
> I second bill's approach of watching folks use the app. Years of educational software creation taught me this. I would always make friends with a local teacher that was into tech and they usually were happy to get a period to try something on the kids if it only took one period to do in the lab and was something they thought good first. Things were so self evident on what just worked and what crashed and burned. I really found that the designs that were forced (usually by marketing) always crashed and burned, but the just good ideas that came out of what was it we were really trying to do somehow avoided most all the little design eddies that folks would get a little hung up by. But watching you could quickly see those eddies w.o having to do hard core testing. Sadly this is hard to do for free in a school anymore but hiring some kids or adults will do.
>
> It's funny as I've found the same thing with exhibit design. I would always spend a few hours just watching folks after we finished an exhibit. I found it really invaluable to find the little issues and the big ones and you could see so easily what folks were getting and what they were not, what they were looking and and not looking at and how they felt about the exhibit in the whole. Many of these exhibits got very expensive summative evaluations and I found that my just watching observations were right in line with heavy testing and many times a bit more complete and useful for potentially fixing things and learning for the future.
>
> Cheers
>
> Jeff
>
>> On Jul 7, 2017, at 1:53 PM, [hidden email] wrote:
>>
>> Jonathon,
>> I feel your pain. In my case, I was initiated by my students and very quickly learned how to ask the questions a newbie would ask. I also paid small amounts to graduate students to get their feedback.
>>
>> One of my very effective testers is my grandson, my wife, any of my colleagues who might be enticed to use the app. Looking over the shoulder while these folks use the app can be very illuminating.
>>
>> In summary:
>> 1. Ask friends and relatives first.
>> 2. Perhaps there would be volunteers from the live ode users group.
>> 3. Hire high school students who might have a tech interest. Look over their shoulders as they use the app and dialog to themselves. Actually watching users is invaluable.
>>
>> Good luck,
>> Bill P
>
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
It does help, Scott - sounds like I should segment the testing process with a cycle, running through the test, observe, discuss, note cycle for each group of functionalities. Not unlike PM methodology.

Because I am looking to perfect and grow a single app over many years, I should be able to reliably group the functional areas for testing.

Thanks!

Sent from my iPhone

> On Jul 7, 2017, at 5:56 PM, [hidden email] wrote:
>
> It sounds like a little bit of direct, intensive observation is worth a lot of testing a a distance.
>
> Thanks Jeff
>
> Sent from my iPhone
>
>> On Jul 7, 2017, at 5:31 PM, Jeff Reynolds via use-livecode <[hidden email]> wrote:
>>
>> Jonathan,
>>
>> I second bill's approach of watching folks use the app. Years of educational software creation taught me this. I would always make friends with a local teacher that was into tech and they usually were happy to get a period to try something on the kids if it only took one period to do in the lab and was something they thought good first. Things were so self evident on what just worked and what crashed and burned. I really found that the designs that were forced (usually by marketing) always crashed and burned, but the just good ideas that came out of what was it we were really trying to do somehow avoided most all the little design eddies that folks would get a little hung up by. But watching you could quickly see those eddies w.o having to do hard core testing. Sadly this is hard to do for free in a school anymore but hiring some kids or adults will do.
>>
>> It's funny as I've found the same thing with exhibit design. I would always spend a few hours just watching folks after we finished an exhibit. I found it really invaluable to find the little issues and the big ones and you could see so easily what folks were getting and what they were not, what they were looking and and not looking at and how they felt about the exhibit in the whole. Many of these exhibits got very expensive summative evaluations and I found that my just watching observations were right in line with heavy testing and many times a bit more complete and useful for potentially fixing things and learning for the future.
>>
>> Cheers
>>
>> Jeff
>>
>>> On Jul 7, 2017, at 1:53 PM, [hidden email] wrote:
>>>
>>> Jonathon,
>>> I feel your pain. In my case, I was initiated by my students and very quickly learned how to ask the questions a newbie would ask. I also paid small amounts to graduate students to get their feedback.
>>>
>>> One of my very effective testers is my grandson, my wife, any of my colleagues who might be enticed to use the app. Looking over the shoulder while these folks use the app can be very illuminating.
>>>
>>> In summary:
>>> 1. Ask friends and relatives first.
>>> 2. Perhaps there would be volunteers from the live ode users group.
>>> 3. Hire high school students who might have a tech interest. Look over their shoulders as they use the app and dialog to themselves. Actually watching users is invaluable.
>>>
>>> Good luck,
>>> Bill P
>>
>>
>> _______________________________________________
>> use-livecode mailing list
>> [hidden email]
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
Jonathon:
There are two learning processes going on. One is for the person testing the software, the second is for you, learning what kinds of interface approaches hang up new users. As you learn, by observing users, you will gain approaches that minimize future user problems, and you will find that you will be able to code in a way that avoids them.

If it were me, I would start small with the evaluation, and do it first by informal observation, encouraging the user to think out loud as he/she uses the app. You will get a feel for obstacles pretty quickly. You may run out of test users quickly if you use many of them at once, so put as much common sense as you can into changes that you make between new testers. If this is unsuccessful, then you will have to expend more of your resources on testing.

Another good thing is to download and try other apps, checking to see how their UI is set up. For example, almost every web delivered login page is the same or similar. Why? Because they work. When numerous apps take a similar approach, learn from them.

Good luck. Please post what you learn from your testing.

Another piece of advice (worth what it costs you??). Your application is actually huge. Think Facebook and the other biggies. Maintaining it, should it be successful, will be HUGE! Think trollers, spammers, whackos, etc, etc. I had a site where I allowed anybody to create an account (but I had to approve the account to activate it), and got loads of trial logins from spammers and bots. Finally, I just disabled new accounts. I wonder if you might want to consider narrowing the scope of your app, perhaps to a specific education segment. Or, maybe a particular travel segment or for a specific tour company. This would let you get your app out there and identify early issues. A tour company might find a custom branded app that supports their tour company to be appealing.

Good luck,

Best,
Bill P.


> On Jul 7, 2017, at 3:01 PM, Jonathan Lynch via use-livecode <[hidden email]> wrote:
>
> It does help, Scott - sounds like I should segment the testing process with a cycle, running through the test, observe, discuss, note cycle for each group of functionalities. Not unlike PM methodology.
>
> Because I am looking to perfect and grow a single app over many years, I should be able to reliably group the functional areas for testing.
>
> Thanks!
>
> Sent from my iPhone
>
>> On Jul 7, 2017, at 5:56 PM, [hidden email] wrote:
>>
>> It sounds like a little bit of direct, intensive observation is worth a lot of testing a a distance.
>>
>> Thanks Jeff
>>
>> Sent from my iPhone
>>
>>> On Jul 7, 2017, at 5:31 PM, Jeff Reynolds via use-livecode <[hidden email]> wrote:
>>>
>>> Jonathan,
>>>
>>> I second bill's approach of watching folks use the app. Years of educational software creation taught me this. I would always make friends with a local teacher that was into tech and they usually were happy to get a period to try something on the kids if it only took one period to do in the lab and was something they thought good first. Things were so self evident on what just worked and what crashed and burned. I really found that the designs that were forced (usually by marketing) always crashed and burned, but the just good ideas that came out of what was it we were really trying to do somehow avoided most all the little design eddies that folks would get a little hung up by. But watching you could quickly see those eddies w.o having to do hard core testing. Sadly this is hard to do for free in a school anymore but hiring some kids or adults will do.
>>>
>>> It's funny as I've found the same thing with exhibit design. I would always spend a few hours just watching folks after we finished an exhibit. I found it really invaluable to find the little issues and the big ones and you could see so easily what folks were getting and what they were not, what they were looking and and not looking at and how they felt about the exhibit in the whole. Many of these exhibits got very expensive summative evaluations and I found that my just watching observations were right in line with heavy testing and many times a bit more complete and useful for potentially fixing things and learning for the future.
>>>
>>> Cheers
>>>
>>> Jeff
>>>
>>>> On Jul 7, 2017, at 1:53 PM, [hidden email] wrote:
>>>>
>>>> Jonathon,
>>>> I feel your pain. In my case, I was initiated by my students and very quickly learned how to ask the questions a newbie would ask. I also paid small amounts to graduate students to get their feedback.
>>>>
>>>> One of my very effective testers is my grandson, my wife, any of my colleagues who might be enticed to use the app. Looking over the shoulder while these folks use the app can be very illuminating.
>>>>
>>>> In summary:
>>>> 1. Ask friends and relatives first.
>>>> 2. Perhaps there would be volunteers from the live ode users group.
>>>> 3. Hire high school students who might have a tech interest. Look over their shoulders as they use the app and dialog to themselves. Actually watching users is invaluable.
>>>>
>>>> Good luck,
>>>> Bill P
>>>
>>>
>>> _______________________________________________
>>> use-livecode mailing list
>>> [hidden email]
>>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
Thanks Bill

I am thinking of reaching out to Trip Advisor after version 1.5, but focusing on high schools initially.

Trolls are a big concern. I have a lot ideas on dealing with that, including taking advantage of the self-correcting nature of social media. I am going to add in a rating system for reports and enable users to exclude poorly rated reports from appearing on their maps. I also want to have a class of users called documentarians, and enable users to see only reports from them. Documentarians will earn half the sponsorship income from their reports, so that will hopefully lead to a bunch of high- quality postings. I also have some ideas for a report review system.

All of that will be a start. If the trolling gets too be too much, I could resort to requiring that postings be reviewed before being being posted to certain categories.

I have some ideas on how to use trolls as an asset, but that is not fully thought out yet.

Sent from my iPhone

> On Jul 7, 2017, at 6:24 PM, William Prothero via use-livecode <[hidden email]> wrote:
>
> Jonathon:
> There are two learning processes going on. One is for the person testing the software, the second is for you, learning what kinds of interface approaches hang up new users. As you learn, by observing users, you will gain approaches that minimize future user problems, and you will find that you will be able to code in a way that avoids them.
>
> If it were me, I would start small with the evaluation, and do it first by informal observation, encouraging the user to think out loud as he/she uses the app. You will get a feel for obstacles pretty quickly. You may run out of test users quickly if you use many of them at once, so put as much common sense as you can into changes that you make between new testers. If this is unsuccessful, then you will have to expend more of your resources on testing.
>
> Another good thing is to download and try other apps, checking to see how their UI is set up. For example, almost every web delivered login page is the same or similar. Why? Because they work. When numerous apps take a similar approach, learn from them.
>
> Good luck. Please post what you learn from your testing.
>
> Another piece of advice (worth what it costs you??). Your application is actually huge. Think Facebook and the other biggies. Maintaining it, should it be successful, will be HUGE! Think trollers, spammers, whackos, etc, etc. I had a site where I allowed anybody to create an account (but I had to approve the account to activate it), and got loads of trial logins from spammers and bots. Finally, I just disabled new accounts. I wonder if you might want to consider narrowing the scope of your app, perhaps to a specific education segment. Or, maybe a particular travel segment or for a specific tour company. This would let you get your app out there and identify early issues. A tour company might find a custom branded app that supports their tour company to be appealing.
>
> Good luck,
>
> Best,
> Bill P.
>
>
>> On Jul 7, 2017, at 3:01 PM, Jonathan Lynch via use-livecode <[hidden email]> wrote:
>>
>> It does help, Scott - sounds like I should segment the testing process with a cycle, running through the test, observe, discuss, note cycle for each group of functionalities. Not unlike PM methodology.
>>
>> Because I am looking to perfect and grow a single app over many years, I should be able to reliably group the functional areas for testing.
>>
>> Thanks!
>>
>> Sent from my iPhone
>>
>>> On Jul 7, 2017, at 5:56 PM, [hidden email] wrote:
>>>
>>> It sounds like a little bit of direct, intensive observation is worth a lot of testing a a distance.
>>>
>>> Thanks Jeff
>>>
>>> Sent from my iPhone
>>>
>>>> On Jul 7, 2017, at 5:31 PM, Jeff Reynolds via use-livecode <[hidden email]> wrote:
>>>>
>>>> Jonathan,
>>>>
>>>> I second bill's approach of watching folks use the app. Years of educational software creation taught me this. I would always make friends with a local teacher that was into tech and they usually were happy to get a period to try something on the kids if it only took one period to do in the lab and was something they thought good first. Things were so self evident on what just worked and what crashed and burned. I really found that the designs that were forced (usually by marketing) always crashed and burned, but the just good ideas that came out of what was it we were really trying to do somehow avoided most all the little design eddies that folks would get a little hung up by. But watching you could quickly see those eddies w.o having to do hard core testing. Sadly this is hard to do for free in a school anymore but hiring some kids or adults will do.
>>>>
>>>> It's funny as I've found the same thing with exhibit design. I would always spend a few hours just watching folks after we finished an exhibit. I found it really invaluable to find the little issues and the big ones and you could see so easily what folks were getting and what they were not, what they were looking and and not looking at and how they felt about the exhibit in the whole. Many of these exhibits got very expensive summative evaluations and I found that my just watching observations were right in line with heavy testing and many times a bit more complete and useful for potentially fixing things and learning for the future.
>>>>
>>>> Cheers
>>>>
>>>> Jeff
>>>>
>>>>> On Jul 7, 2017, at 1:53 PM, [hidden email] wrote:
>>>>>
>>>>> Jonathon,
>>>>> I feel your pain. In my case, I was initiated by my students and very quickly learned how to ask the questions a newbie would ask. I also paid small amounts to graduate students to get their feedback.
>>>>>
>>>>> One of my very effective testers is my grandson, my wife, any of my colleagues who might be enticed to use the app. Looking over the shoulder while these folks use the app can be very illuminating.
>>>>>
>>>>> In summary:
>>>>> 1. Ask friends and relatives first.
>>>>> 2. Perhaps there would be volunteers from the live ode users group.
>>>>> 3. Hire high school students who might have a tech interest. Look over their shoulders as they use the app and dialog to themselves. Actually watching users is invaluable.
>>>>>
>>>>> Good luck,
>>>>> Bill P
>>>>
>>>>
>>>> _______________________________________________
>>>> use-livecode mailing list
>>>> [hidden email]
>>>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>
>> _______________________________________________
>> use-livecode mailing list
>> [hidden email]
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
In reply to this post by Richmond Mathewson via use-livecode
I would echo Jacqueline's advice here:

" There's no secret to creating a great application that people will use and enjoy. Test the application and everything in it, again and again. Run everything through its paces several times and in different orders. Click every button, read and scroll every field, run every script. Show every dialog—and when you do, try every possible response button. Then get all your friends to do the same thing too.

That's it. Everything else I'm about to tell you is just commentary."

I myself will run through the app, over and over and OVER again… as Mark says, you will end up in the developer minds set and miss the UX stuff;   so you need to step away mentally and do a "reset" on your head space.. then come back and run through it all again. I probably spend 50 % of my time working on content. 30% on code and a full 20% just "thinking about it, how it works, from a user point of view." And you do this away from your computer!

Like today little girl told me "I love doing the word puzzles, but when I first came to that screen it was a bit confusing how it was supposed to work. Not everyone will think to click on the "I" icon [supposed to be for "info/help")  Maybe you could show instructions the first time?"  and "it would be great if you could save the puzzle in a semi-finished state, so I can come back and finish it." This "little girl" in a 10 year-odl body with a 19year old brain that actually has the patience to take a 300 character quote and put the words in order, even if it takes her 20 minutes!  Even I won'd do that… but she will, but she might have to run off to school and stop half way through….

Another more mature response to the first beta: "It's all very lovely, but I'm not clued in to what, where we are going …."

In V2 or V3 I'm planning to have a little more curated entry to the app… There's got to be more of "a story here"

so you will never get this UX feedback on your own.

I have about 40 beta testers.. even they are, frankly terrible about give us feedback. …   if you are non-profit or on shoe string budgets, you can't pay for testing, and volunteers are often super busy people. So you have to be very pro-active in engaging feedback.  I'm a bit "in your face" with guests here when I spot someone in my "target market" (young 15-40 educated, Hindu background modern mind set)  I've come to know who will be engaged and who will not.

But this type of users (Free beta testers) are not UX experts either… so you get  either silence or "it works great lovely very handy and responsive on my iPhone" or "it's slow" [android]  without telling me exactly what is slow and when. I have deliberately hold mini "focus meetings" here with some of them and drag out from them what is "wrong" with the app, because not being up on UX, these users don't know how to articulate their experience very well.

I'm constantly collecting possible beta testers. Apple devices on the developer account are running up close to 45 for this year , with about as many turned off from last year… and we have at least another 30 android uses… but, maybe only 3-5 of these will actually communicate with me. Once I actually find someone who is interested enough and articulate enough and has the time to stop and talk about it, e.g.

"I really don't like that timer giving a count down sound. It ruins the experience. Just let me decide how long, then let it run and at the end you can "ding""

OK Aha! then I "got one!" … i.e. someone  you can spend on to help with the UX and milk that dialog about the UX as much as a I can.

I have two meetings on the latest beta this afternoon: one with a professional health care lady and  her daughter who both test the app and another with a 19year old brilliant university studen, who flew to Kauai just to go on a "study retreat" for some crazy advance bio-genetic engineering exams.. who is very open and articulate about  what they think about how it should work. And I have on my schedule to talk to 4 different beta testers remotely

This doesn't happen by itself, you have to be super pro active. Once yet get a team of volunteer testers who are engaged, even just 4-5, who communicate well with you, you are good to go. But you might have to work through 100 different individuals before you find those 5.

That's my experience so far here…




 

On 7/7/17, 7:53 AM, "use-livecode on behalf of Jonathan Lynch via use-livecode" <[hidden email] on behalf of [hidden email]> wrote:

    Thank you, Jacqueline
   
    Sent from my iPhone
   
    > On Jul 7, 2017, at 1:39 PM, J. Landman Gay via use-livecode <[hidden email]> wrote:
    >
    > http://hyperactivesw.com/resources_testing.html
    >
    >
    >> On July 7, 2017 6:59:52 AM Jonathan Lynch via use-livecode <[hidden email]> wrote:
    >>
    >> What steps do you guys follow for accurate testing when you don't have a budget for proper official testing procedures?
    >
    > --
    > Jacqueline Landman Gay         |     [hidden email]
    > HyperActive Software           |     http://www.hyperactivesw.com
    >
    >
    >
    > _______________________________________________
    > use-livecode mailing list
    > [hidden email]
    > Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
    > http://lists.runrev.com/mailman/listinfo/use-livecode
   
    _______________________________________________
    use-livecode mailing list
    [hidden email]
    Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
    http://lists.runrev.com/mailman/listinfo/use-livecode
   

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
I am impressed by your circle of beta testers!

I have plenty of friends and family that would help, but they are spread around the country.

Point taken about folks not communicating so well. It seems like direct observation of them using it could help.

The way my brain works, I am decent at figuring out clever solutions, but terrible at realizing what others will find intuitive. It looks like I will need to put in extra effort to cultivate my testers.

My plan for very slow early growth is partly to compensate for this issue. It needs to be a near flawless user experience before I start promoting.

I can be a beta tester for you, if that would help. I don't resemble your target market in the least, and my UI preferences are odd, but I might have some good ideas.

Sent from my iPhone

> On Jul 7, 2017, at 8:07 PM, Sannyasin Brahmanathaswami via use-livecode <[hidden email]> wrote:
>
> I would echo Jacqueline's advice here:
>
> " There's no secret to creating a great application that people will use and enjoy. Test the application and everything in it, again and again. Run everything through its paces several times and in different orders. Click every button, read and scroll every field, run every script. Show every dialog—and when you do, try every possible response button. Then get all your friends to do the same thing too.
>
> That's it. Everything else I'm about to tell you is just commentary."
>
> I myself will run through the app, over and over and OVER again… as Mark says, you will end up in the developer minds set and miss the UX stuff;   so you need to step away mentally and do a "reset" on your head space.. then come back and run through it all again. I probably spend 50 % of my time working on content. 30% on code and a full 20% just "thinking about it, how it works, from a user point of view." And you do this away from your computer!
>
> Like today little girl told me "I love doing the word puzzles, but when I first came to that screen it was a bit confusing how it was supposed to work. Not everyone will think to click on the "I" icon [supposed to be for "info/help")  Maybe you could show instructions the first time?"  and "it would be great if you could save the puzzle in a semi-finished state, so I can come back and finish it." This "little girl" in a 10 year-odl body with a 19year old brain that actually has the patience to take a 300 character quote and put the words in order, even if it takes her 20 minutes!  Even I won'd do that… but she will, but she might have to run off to school and stop half way through….
>
> Another more mature response to the first beta: "It's all very lovely, but I'm not clued in to what, where we are going …."
>
> In V2 or V3 I'm planning to have a little more curated entry to the app… There's got to be more of "a story here"
>
> so you will never get this UX feedback on your own.
>
> I have about 40 beta testers.. even they are, frankly terrible about give us feedback. …   if you are non-profit or on shoe string budgets, you can't pay for testing, and volunteers are often super busy people. So you have to be very pro-active in engaging feedback.  I'm a bit "in your face" with guests here when I spot someone in my "target market" (young 15-40 educated, Hindu background modern mind set)  I've come to know who will be engaged and who will not.
>
> But this type of users (Free beta testers) are not UX experts either… so you get  either silence or "it works great lovely very handy and responsive on my iPhone" or "it's slow" [android]  without telling me exactly what is slow and when. I have deliberately hold mini "focus meetings" here with some of them and drag out from them what is "wrong" with the app, because not being up on UX, these users don't know how to articulate their experience very well.
>
> I'm constantly collecting possible beta testers. Apple devices on the developer account are running up close to 45 for this year , with about as many turned off from last year… and we have at least another 30 android uses… but, maybe only 3-5 of these will actually communicate with me. Once I actually find someone who is interested enough and articulate enough and has the time to stop and talk about it, e.g.
>
> "I really don't like that timer giving a count down sound. It ruins the experience. Just let me decide how long, then let it run and at the end you can "ding""
>
> OK Aha! then I "got one!" … i.e. someone  you can spend on to help with the UX and milk that dialog about the UX as much as a I can.
>
> I have two meetings on the latest beta this afternoon: one with a professional health care lady and  her daughter who both test the app and another with a 19year old brilliant university studen, who flew to Kauai just to go on a "study retreat" for some crazy advance bio-genetic engineering exams.. who is very open and articulate about  what they think about how it should work. And I have on my schedule to talk to 4 different beta testers remotely
>
> This doesn't happen by itself, you have to be super pro active. Once yet get a team of volunteer testers who are engaged, even just 4-5, who communicate well with you, you are good to go. But you might have to work through 100 different individuals before you find those 5.
>
> That's my experience so far here…
>
>
>
>
>
>
> On 7/7/17, 7:53 AM, "use-livecode on behalf of Jonathan Lynch via use-livecode" <[hidden email] on behalf of [hidden email]> wrote:
>
>    Thank you, Jacqueline
>
>    Sent from my iPhone
>
>> On Jul 7, 2017, at 1:39 PM, J. Landman Gay via use-livecode <[hidden email]> wrote:
>>
>> http://hyperactivesw.com/resources_testing.html
>>
>>
>>> On July 7, 2017 6:59:52 AM Jonathan Lynch via use-livecode <[hidden email]> wrote:
>>>
>>> What steps do you guys follow for accurate testing when you don't have a budget for proper official testing procedures?
>>
>> --
>> Jacqueline Landman Gay         |     [hidden email]
>> HyperActive Software           |     http://www.hyperactivesw.com
>>
>>
>>
>> _______________________________________________
>> use-livecode mailing list
>> [hidden email]
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>
>    _______________________________________________
>    use-livecode mailing list
>    [hidden email]
>    Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>    http://lists.runrev.com/mailman/listinfo/use-livecode
>
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Biased testing and micro-coaching

Richmond Mathewson via use-livecode
On 07/07/2017 05:50 PM, Jonathan Lynch via use-livecode wrote:

> The way my brain works, I am decent at figuring out clever solutions, but terrible at realizing what others will find intuitive.

Heh. You're not alone there. There's a truism that you can't test your
own software - you're way to close to the way it *should* work to ever
figure out how users are going to try to use it.

> my UI preferences are odd,

You make that sound like a bad thing... <g>

--
  Mark Wieder
  [hidden email]


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
12
Loading...