Supporting Wizard of Oz experimentation for language technology applications
Citation:
Stephan Schl_gl, 'Supporting Wizard of Oz experimentation for language technology applications', [thesis], Trinity College (Dublin, Ireland). School of Computer Science & Statistics, 2013, pp 141Download Item:
Abstract:
Wizard of OZ (WOZ) is a well-established method used by researchers and product designers
to simulate the functionality and user experience of future systems. Using a human wizard to
mimic possible operations is particularly useful in situations where extensive engineering effort
would otherwise be required to explore a given design space. The method has been widely used
in connection with speech and language technologies, but advances in sensor technology and
pattern recognition as well as new application areas such as human-robot interaction have made
WOZ increasingly relevant to the design of interactive systems.
Even though it is recognised as a valuable prototyping technique, surprisingly little effort
has been devoted to exploring WOZ from a wizard perspective. The goal of this thesis is to expand
upon the existing knowledge by presenting a systematic investigation and analysis of the
potential design space for WOZ prototyping as well as an empirical exploration of appropriate
support features for wizards through an iteratively developed web-based prototyping platform.
The presented research employed a systematic analysis approach in which the construction
and consequent evaluation of a maturing system created new insight into WOZ support and
some of its challenges. As part of these evaluations, WOZ characteristics such as wizard workload,
wizard consistencies as well as the creation of WOZ experiments, were analysed. The
results show that wizards gradually adapt to their task, as would be expected, resulting in faster
response times. Consistency, however, is more difficult to achieve and therefore requires additional
support. Furthermore, it was found that researchers/designers are generally able to create
WOZ experiments, even without upfront training, validating our efforts of providing a platform
for more generic support.
In summary, this thesis expands the existing body of knowledge on WOZ prototyping by
presenting a structured analysis of the method. Informed by both the literature and a set of
investigations (including a series of interviews with researchers from industry and academia),
it demonstrates the multitude of motivations for using WOZ, highlights its challenges and investigates
potential solutions. Being undertaken alongside the development of a new WOZ
prototyping tool, this thesis furthermore explores technical problems and presents possibilities
for more generic tool support. Even though the presented work has a strong focus on evaluating
language technologies, we believe that the generated insight is applicable to the whole domain
of WOZ prototyping, and therefore should be seen as a substantial contribution to the field of
Human-Computer Interaction.
Author: Schlogl, Stephan
Advisor:
Doherty, GavinPublisher:
Trinity College (Dublin, Ireland). School of Computer Science & StatisticsNote:
TARA (Trinity's Access to Research Archive) has a robust takedown policy. Please contact us if you have any concerns: rssadmin@tcd.ieType of material:
thesisCollections
Availability:
Full text availableMetadata
Show full item recordLicences: