Evaluating a script...

21 Sep 2005 - 9:47am
9 years ago
1 reply
359 reads
Ted Boren
2005

Hi All *

Like most of you, I work in the software arena. However, I've recently been asked to organize audience evaluations for audiovisual productions as well (potentially film, TV, DVD, etc.). Some are training productions, some are persuasive, some have an entertainment component (though that is never the primary motivation for productions in my case).

I've done a quick web search, and most script evaluation seems to be of the "expert review" type ("pay me lots of money and I'll tell you where it stinks"). I'd to involve potential audience members instead of doing this kind of professional review. Short of full-blown audience previews & screenings (which usually come too late in most production processes to have much impact), what can I do early on to evaluate an AV production's direction? Anyone have any experience or ideas?

I have a few ideas that may cross over somewhat from my experience in usability research. Examples:
Plus-minus studies of the script. (See various articles by Menno de Jong, Sienot, and other Dutch researchers in text evaluation.)Focus groups after a individual or group reading of scripts or storyboards. (Normally I'm not a big fan of focus groups for software evaluation... but this seems different with AV, since participants will have a specific, subjective, recent experience to reflect on, and since in most cases there will not be a lot of objectively measurable outcomes, such as task success, time on task, etc.)Anybody have any experience with the above-mentioned methods, or with other evaluation methods, in an AV context? Thanks in advance!

Ted Boren

------------------------------------------------------------------------------
This message may contain confidential information, and is
intended only for the use of the individual(s) to whom it
is addressed.
------------------------------------------------------------------------------

Comments

21 Sep 2005 - 12:45pm
John Schrag
2005

At 10:47 AM 9/21/2005, Ted Boren wrote:
>Like most of you, I work in the software arena. However, I've recently
>been asked to organize audience evaluations for audiovisual productions as
>well (potentially film, TV, DVD, etc.). Some are training productions,
>some are persuasive, some have an entertainment component (though that is
>never the primary motivation for productions in my case).

I've been involved in evaluation of training films. We had clearly stated
goals, so we could evaluate them based on their effectiveness. That is, we
could compare the task behaviour of people who hadn't seen the film to the
behaviour of those who had. If people who saw the film manifested the
desired training behaviours, then the film was successful. We could
specifically measure which concepts were retained and which weren't.

If the goal of a production is to persuade, perhaps you could survey
attititudes of a control group versus a group who watched the film; or
compare attitudes of the same people before and after.

If the goal is to keep people entertained and engaged, you might look at
what they did at the Children's Television Workshop for Sesame
Street. According to Malcolm Gladwell in "The Tipping Point", every Sesame
Street clip was tested by showing it to children on one screen while
distracting slides were being shown on nearby screens. If the clip held
the children's attention against the distractors, it was considered a
success. I haven't tried this one myself.

I think your evaluation techniques will really be entirely dependent on the
goals of the production.

-john

----------------------------------------------------
John Schrag Alias
Interaction Designer 210 King Street East
jschrag at alias.com Toronto, Canada M5A 1J7

Syndicate content Get the feed