Uncontrollable Errors

24 Mar 2006 - 3:18pm
8 years ago
3 replies
245 reads
Jeremy Wood
2006

I have a question... I wonder if any research has been put into this
topic:

Suppose we have a software application that interacts with a scanner, a
camera, FTP server, etc. Some external agent beyond our control.
Now suppose there's a problem, such as:
1. Their scanner is broken
2. Their FTP server is down
3. Their proxy server is failing
4. Their camera is not turned on
... etc.

The user then contacts us, and after several emails/calls we eventually
get an idea what the real problem is. It turns out it's completely
beyond our realm of control. (If we're REALLY lucky, we can advise
them how to fix it. If we're mildly lucky, we get to talk to a network
administrator, or tech-savvy individual who can take steps to really
fix the problem, or contact the right people to help. But sometimes
we're not that lucky, and all we can do is explain to the user it's a
problem we can't fix.)

And even if we fix it, the user doesn't always catch on to the fact
that it wasn't a bug in our software. Sometimes they still walk away
thinking our application caused their problem.

I'm wondering: does this leave irrevocable negativity in the user's
mind towards our software? Is it better to refrain from these "risky"
features, and not lose face when they fail?

That's my basic question, right there. Other tidbits about our
situation:

As we draft a list of new features for upcoming version of our
products, things like scanner support, USB camera playthrough, etc.,
are trickling in. These are great ideas, but we're a small company. I
wonder if trying to add these will cause more headache, and in some
cases bad experiences, than simply, say, asking people to use the
software that came with their scanner... then import those images into
our software.
(When the hardware changes protocol somehow, will our software break?
Will we get calls from people when their images are gray, only to find
out their scanner is to blame?)

On the other hand, I know it's always great if people don't have to
switch apps to get their goals accomplished, so I can see both sides.
And our user base is mostly students and teachers; younger students
especially need everything gift-wrapped in one tidy application to
really get anything accomplished.

But, for example, we've had a lot of calls/complaints from people using
our web-authoring tool that usually boil down to server problems on
their end. At this point I'd like to encourage them simply to export
their webpages to their local hard disk, and THEN copy them to their
server. If they can't connect to their server from Windows/the Finder,
that should make it more clear that it's something for their network
administrator to deal with. If they see the error in our application
instead, does that subtly poison them against our software/company?

P.S. I'm in no way implying our software is otherwise perfect and
bug-free. :) But we do happen to field a lot of calls from users
about things outside our control.

Comments

28 Mar 2006 - 7:39am
jbellis
2005

Jeremy,
Some thoughts, below.
www.jackBellis.com, www.UsabilityInstitute.com
----- Original Message -----
From: "Jeremy Wood" <jeremy at tech4learning.com>

> I'm wondering: does this leave irrevocable negativity in the user's mind
> towards our software?

If they're of an aptitude level that renders them incapable of appreciating
the several interdependent layers of modern PCs then, yes, they probably
blame you. But, being somewhat in-astute, their displeasure might not be
significant and certainly not irrovocable. They are not the type who says to
themselves, "Well, I'll just go to the next camera-scanner-FTP integrator.
Ah, here's one right here in my address book."

>Is it better to refrain from these "risky" features, and not lose face when
>they fail?

No. End-to-end solutions are the very essence of usability, no matter how
tenuous our balance on the "high wire."

> I was wonder if trying to add these will cause more headache...

Yes, providing more functionality for users will cause lots of headaches for
you. That is the business of software development... almost the tautlogical
definition of the work (words were free today on Dictionary.com). If you
simply don't have the resources, you've answered your own question.

> (When the hardware changes protocol somehow, will our software break?
Isn't that what Twain is supposed to combat??? I understand such middleware
isn't perfect.

> But, for example, we've had a lot of calls/complaints from people using
> our web-authoring tool that usually boil down to server problems on
> their end.

This is the real issue and the reason for my reply. I call it the "recipe
vs. ingredients" problem. Installation instructions typically tell you the
recipe. After the recipe fails you need to step through the ingredients to
see what's missing.

I believe you need to do everything you can- - -communicating through the
user
interface- - -to enable the user to separate the layers of technology when
there's trouble... and/or maybe BEFORE?:::

1) Within the code you do own, trap failure with fine-grained error messages
at every point that you possibly can.

2) When all the traps fall through, provide a visual error message that
shows your layer as merely the first link in the chain. If you can't even
communicate with the user once you contact the outside world, then is it
possible to provide a "portal" (through-the-looking-glass?) message..."Now
sending/receiving... if your
page does not refresh in a few moments <underline>Show Our Diagnostic
Tests</underline>"?

3) Provide a link (or hyperlink the pictures of the other "links" in the
chain) directly to diagnostics that test every dependent layer... or mere
"cue card" panels that tell the user how to test FTP, ping the server, etc.
The age or aptitude of the user has no bearing on the value of providing
diagnostic tools. The usual retort here is that "they" (the mythical,
diminutized [?] persona) won't know how to do this. I've fought against this
argument forever. What good does it do to deprive resourceful users of tools
simply because not everyone can use them?

Thanks, Jack

28 Mar 2006 - 2:45pm
Jeremy Wood
2006

Thanks for replying. :)

>> I'm wondering: does this leave irrevocable negativity in the user's
>> mind
>> towards our software?
>
> If they're of an aptitude level that renders them incapable of
> appreciating
> the several interdependent layers of modern PCs then, yes, they
> probably
> blame you. But, being somewhat in-astute, their displeasure might not
> be
> significant and certainly not irrovocable.

Hmmm. I see your point...

At the same time, just today we got an email from a teacher. She's
having trouble running our installer, because disk permissions on Mac
OS X are the devil. (We don't write the installer, we have a contract
with VICE and use their installers.) Her email ends with:

"Any suggestions? I like this software, I want my students to use it,
but right now the frustration level is rising fast." Already one
strike against us... and they haven't even launched our application
yet.

>> Is it better to refrain from these "risky" features, and not lose
>> face when
>> they fail?
>
> No. End-to-end solutions are the very essence of usability, no matter
> how
> tenuous our balance on the "high wire."

I see your point, but I disagree that it's the "essence" of usability.
:) There are certainly occasions when end-to-end solutions are nice,
but there's a lot to be said for self-contained tools. Many times I
would rather have a series of well-designed, small tools than 1 program
that integrates everything. (For example, I don't like Microsoft Word;
they integrated way too many features, in my opinion.)

The whole model of the modern OS is to have several well-defined
applications that do separate things. When Application X adds support
for a particular feature, we may just be reinventing a very
well-designed wheel somewhere... bloating our software (which increases
file size, the amount of testing we require, code to maintain, etc.).

I think end-to-end solutions are sometimes very useful, but need to be
evaluated on a case-by-case basis.

>> I was wonder if trying to add these will cause more headache...
> Yes, providing more functionality for users will cause lots of
> headaches for
> you. That is the business of software development... almost the
> tautlogical
> definition of the work
Hey now, don't be sneaky with your words. :) Your argument is great
rhetoric, but I was talking about a very specific subset of features
(those that depend on external hardware), and you started talking about
features in general. Of course we embrace features and work. (And
other things like puppies and freedom and pregnant women.)

But I was referring to a very specific type of "headache". When a user
reports a problem and:
1. We can't reproduce it. To pick a recent example, because we don't
have a FirstClass server.
2. We can't change the error-producing code. (FirstClass, for
example, didn't let go of cached images when we replace files.)

>> (When the hardware changes protocol somehow, will our software break?
> Isn't that what Twain is supposed to combat??? I understand such
> middleware
> isn't perfect.
You're right, Twain will help. But you're also right: it isn't
perfect. Neither is the hardware. And middleware solutions don't
always exist for everything. And it's been a while since I looked at
Twain, but will it really provide informative feedback if the driver
isn't installed on one computer? Or if the ribbons in the scanner are
out of alignment, because it fell off the desk? We'll get calls about
these sort of things. :)

>> But, for example, we've had a lot of calls/complaints from people
>> using
>> our web-authoring tool that usually boil down to server problems on
>> their end.
>
> This is the real issue and the reason for my reply. I call it the
> "recipe
> vs. ingredients" problem. Installation instructions typically tell you
> the
> recipe. After the recipe fails you need to step through the
> ingredients to
> see what's missing.
>
> I believe you need to do everything you can- - -communicating through
> the
> user
> interface- - -to enable the user to separate the layers of technology
> when
> there's trouble... and/or maybe BEFORE?:::

An interesting idea.
I ran this by our head of technical support... he disagrees, to an
extent. At the point when something is failing, he'd rather they call,
and talk to a person. On the phone they can better diagnose what the
problem is.

On the flip side, I half like the idea, knowing that users don't always
call tech support when they have problems. Maybe we could help users
who would normally stop trying. (Another subject I'd be interested in
research on.)

But, another con is: that sounds like as much -- if not more -- work to
develop and test than the feature itself. (Did I mention that our two
supported platforms (Mac OS and Windows) have built in FTP support? So
we can spend weeks making our own integrated FTP support, and
diagnostic tools, and save the user a few keystrokes... or just teach
them how to use their OS and keep the scope of our application
simpler.)

> 3) Provide a link (or hyperlink the pictures of the other "links" in
> the
> chain) directly to diagnostics that test every dependent layer... or
> mere
> "cue card" panels that tell the user how to test FTP, ping the server,
> etc.
> The age or aptitude of the user has no bearing on the value of
> providing
> diagnostic tools. The usual retort here is that "they" (the mythical,
> diminutized [?] persona) won't know how to do this. I've fought
> against this
> argument forever. What good does it do to deprive resourceful users of
> tools
> simply because not everyone can use them?

Hmmm... I can find several reasons for both agreeing and disagreeing,
but in our case, here's the bottom line:

My original email was about whether we should develop & support a
feature K. Let's say it would take X many weeks to implement this
feature. Your response is: "Yes, and also support feature K-2." This
may take an *extra* X many weeks to implement.

It does sound like a nice idea, but in our case we simply don't have
the resources. So that answers the question for us right there. If we
did have the resources... we'd still have to think about it. Every
menu item, every button, every dialog should be scrutinized and
carefully weighed. Just because we can add a feature doesn't mean its
in the user's best interest. If we add a feature, is it used by more
than 1% of our user base? (So the other 99% will see that as an
"advanced" or "useless" feature.) At what percent does that decision
change?

Thanks for the response. Gives food for thought.

- Jeremy

28 Mar 2006 - 7:30pm
jbellis
2005

Jeremy, we don't really disagree about much, but this being the web, I'm
honor-bound to try to extend this thread beyond "Func specs, long and
getting longer."

> bloating our software (which increases
> file size, the amount of testing we require, code to maintain

Those are vendor values (that can impact user values)... but they are not
user values.

>he'd rather they call,
> and talk to a person.

You have the resources to handle support calls with newbies but can't
justify robust code? If ever there was a poster child for
small-corporate-America, that's it.

Look, you know what you have to do. The most compelling arguments here are
not yours or mine but the screaming dearth of answers from others: you're up
against pure business tradeoffs in the midst of a technological civil war
(no, it's not a civil war), not interaction design issues.

Kind regards and you can have the last word, I promise,

www.jackBellis.com, www.UsabilityInstitute.com

Syndicate content Get the feed