For User Study. The Implications of Design

Summary of: R. Chow, “For User Study. The Implications of Design,” 2005.

In her 2005 Ph.D. thesis, Rosan Chow succeeds in providing a theoretical framework that gives user study a proper place inside the design process. She first draws a distincition between design and science to define design as an «inquiry oriented toward a specification that fits». She then uses this definition to explore the purpose of user study in the context of design. According to her, the outcome of a user study should not be a generalized understanding but rather a reasonable narrative.

Introduction

The belief that user study is important is only based on informal observation and lacks a theoretical justification rooted in design theory.

Study has shown that user study is well known but not well understood 1.

User study is not new: 2, 3. It is just more privileged today.

User-centered design (UCD) places user study at the beginning of the design project 4. This is necessary because market research doesn't produce the type of information needed by the designer Applicability gap.

Originally design was closer to the humanities 5. UCD moves it closer to the sciences. UCD research methods are grounded in the social sciences.

Most evaluations of user study focus on how information is collected and not on how this information is used to assist designing, the infamous Applicability gap. 6

Contrary the goal of generalization in Science, raw (ungeneralized) data seems more useful to designers:

More abstracted user study data have been found to be less likely to provide design teams in this study with material for the development of problem- solving themes; they are also less likely to be sources of organizing structures that teams use to frame the design problem, generate solution concept, and, in many cases, communicate those concepts. 7

The nature of design

Many definitions

The question then arises, what form of information is useful to the designer.
For this it is necessary to understand the design process, and not the concrete realization that is practiced today (design practice) but a more abstracted version of it, called here the nature of design (a definition).

p.48: Love8 has compiled a 30,000-word annotated bibliography on the definitions of ‘design’ made between 1962 and 1995 in the field of Engineering Design alone.

Chow here establishes her background to inform the reader where her definition of design is coming from. She goes on to lay out four approaches used to define "design" in the past:

  1. Cognitive Problem Solving: Describes the cognitive processes of designing
  2. Knowledge processing: Design as a bridging discipline in business
  3. Communication interaction: Sees the design process as an interaction between products and environments
  4. Philosophic Intellectual: Design is seen as practical reasoning

Comparing Design with Science

A definition should be useful as a map and help with organization. It is less concerned with truth.

Two criteria are chosen for a good definition: It should be consistent and it should distinguish design from science. The second criterion is based on the Chow's goal to distinguish user study for design from studies in scientific disciplines and on a current discussions.

The relationship between design and science has been discussed in each generation: Modernism (1920s), Design Methods Movement (1960s), User Centered Design (2000s)

Views on the relationship between design and science 9

  1. Design is reduced to science.
    Design is an intuitive, instinctive activity for meeting human needs that needs to be better understood by science to become more efficient and effective.
  2. Design is different in kind from science.
    Design is the intellectual discipline of forethought and planning
  3. Science is reduced to Design
    Design is the art of operations and performance, the art of the practical and the possible, the art of making things work as practiced in the elaboration of scientific knowledge.
  4. Design and science are inseparable union of theory and practice
    Design is the dialectical interplay of science and art in action to shape the man-made world.

Cognitive Problem Solving (CPS) research sees design as a different way of knowing, as the "third discipline" next to the sciences and the arts. But CPS doesn't show precisely how designing differs from other problem solving activities using for example Abductive reasoning.

Chow argues that the distinction should not be made based on the cognitive processes employed. Although cognitive processes are involved in design and science, both are not cognitive processes.

Chow also excludes view 3 ("Science is reduced to Design") because it compares the way of thinking called "design" which is also employed in scientific practice with the social practice of science as a whole.

To better distinguish design and science it is established that both are a form of inquiry. The distinction then lies in the different objectives of the two inquiries.

Inquiry oriented toward a specification that fits

Design as the mediator

To clarify the objective of the design inquiry, Chow sketches her version of the nature of design. The object of design is the artificial world as opposed to the natural world (science).

The artificial world created by design is a mediator between humans and their environment. It is the interface between the world as it is and the world as we would like it to be (horizontal axis).

At the same time, design is concerned with matching an artifact and its context (vertical axis).

(D)esign is an interface between the artefactual and the contextual that is an activity humans engage to change the environment to fit their intention.

Indeterminateness of the artifact

Yet the artifact doesn't exist when the designer starts her work. The designer must conceive it out of the problem (which is somehow related to the context). And how this happens is precisely one of the central points in design discourse.

Even though the problem seems fixed at first, it is redefined during the design process. A constant negotiation between goal, problem and solution. Which could be seen as a form of inquiry:

Inquiry

(I)nquiry is the controlled or directed transformation of an indeterminate situation into one that is so determinate in its constituent distinctions and relations as to convert the elements of the original situation into a unified whole (1991: p.108)

Design and science are inquiries that differ in objective

Science aims for a general description, design aims for a specification that fits a context.

Science has recently been invaded by social, political and economical objectives which demand from it contextualized specifications. By this it becomes more like design.

The design process involves the creation of description but always with the aim of informing the construction of a specification.

User study creates context

Context as a narrative

When designing something concrete (a specification), the designers needs to set a lot of parameters. The design space is huge. The more specific the context, the more the design space is reduced. A "table for children" reduces the options for the "height" parameter as compared to simply "a table".

Participants for user study are therefore often selected because the represent a specific population (lead users). As opposed to social sciences.

For designing, it is not about finding out the general needs and wants of a general population, it is always specific needs and wants of a specific group of people that interests designers. This conclusion is consistent with research findings that designers have found general information such as the average ergonomic data not useful. It might also explain why the designers in Melican's7 study find abstract data less useful.

But there exists a gap between specification and what any user study can provide:

(T)he final product of use study is ‘statements of customers needs’ not product specifications. 10

The following methodology is proposed:

  • Step 1: Gather raw data from customers
  • Step 2: Interpret raw data in terms of customer needs
  • Step 3: Organize the needs into a hierarchy
  • Step 4: Establish the relative importance of the needs
  • Step 5: Reflect on the results and the process

Example (SD = Screw Driver):

Creating a user context may appear to be a formalized and straightforward practice, but time and again Ulrich and Eppinger10 mention that this is not an exact science and must rely on the experiences and intuition of the development team to carry out properly. As Sugar's11 study shows, for novice design students who are less experience in making sense of what they observe, user study has little effect on the quality of designs.

Although this might seem non-rigorous, Chow argues, based on Toulmin 12, that the narrative form of reasoning is not less rigorous than rational reasoning:

‘(F)rom early on, the word ‘philosophy’ referred to the systematic and methodical treatment of any subject. The spectrum reached from geometry and astronomy at one pole to autobiography and historical narrative at the other. In all these human activities “reasons” play a central part’. It is therefore a historical accident if we believe that reason is secondary to rationality. ‘Substantive arguments are historically situated and rely on the evidence of experience’ the best they can claim to do is to put a conclusion ‘beyond a reasonable doubt’ and establish the ‘strongest possible presumption on its behalf’.

User study therefore aims to establish context through a reasonable narrative of a particular user. This narrative then becomes the justification for design decisions. If it serves this function, its objective is fulfilled even though it might not be seen as satisfactory in other disciplines.

Context doesn't guarantee design

Aristotle's four causes:

  • Formal cause (what is it?)
  • Material cause (what is it made of?)
  • Efficient cause (what made it?)
  • Final cause (what is it made for?)

St. Thomas addition:

  • Exemplary cause (what could it be?)

For a table, the wood is its matter, the table is its form, the carpenter is its efficient cause, the client has the final cause, and the designer provides the exemplary cause. Purpose (final cause) is not the same as an idea (exemplary cause) that leads to a design a specific form that fulfills the purpose.

Identifying the final cause is therefore not the same as as the exemplary cause. And context, although related, not the same as the specifications. Again the Applicability gap.

Creating a context for designing a table is different from generating possible forms of table and these two activities are also different from judging whether a particular table is fitting to the context. Description or explanation of any kind, as important as they are in giving directions, only sets the problem to which a design addresses. Designs can not be deduced from them. No amount of information generated from user study is sufficient in bringing out designs. There is always a gap between description and creation, problem and solution or context and specifications. Recognizing this gap is important for it directs us to suggest different approaches to construct user study in relation to designing.

Design-driven user study

The creation of a context is also a design project. This is because the context needs to be a specification that fits the needs of the designer and not a general description of reality. So there is no defined end and no beginning in creating context. The initial context is mostly arbitrary.

The project can not start with establishing a sufficient context because no amount of context information can be the cause of a specification (as described above). Additionally, a specification will have change the context in unforeseeable ways.

User study is therefore something that is part of the design process and not apart from it.

Design typically starts from what exists and goes in one of two directions:

  • More general: Mobile communication
  • Existing: Mobile phone
  • More specific: Mobile phone for elderly ← The strength of UCD

All design is therefore essentially redesign.

Beyer & Holzblatt13 propose a methodology based on contextual inquiry that allows to move more towards the general. Based on interviews and observations they create work models for a person. After creating multiple work models, they create consolidation work models from which more abstract statements of needs are induced. (similar to Ulrich's finding 10).

[T]he level of interpretation of raw data increases as the innovative level of design increases. Raw data is abstracted to a more general level. However, this abstraction is still not a scientific causal explanation, but rather a more general narrative of a particular user context.

Chow calls this Essential design.

Design knowledge

Design knowledge that is categorically comparable to scientific knowledge. It is the knowledge that results from design inquiry. Just like scientific knowledge results from scientific inquiry.

Design uses knowledge from scientific disciplines like ergonomics. But this doesn't make it "design knowledge". It is still scientific knowledge.

Historical

Design knowledge then is the specification. The specification is contextual. Because context changes and is changed by design, this knowledge is not generalizable. Rather, design is a historical discipline 5.

Transferable

It is not able to predict but allows implications to be drawn so others can anticipate what might arise in other contexts.

Design knowledge is being transferred from one context to another. However, what designers have not done so successfully is to systematically reflect, document and disseminate this knowledge. In other words, the design field has fallen short of constructing explicit and public knowledge to build consensibility and consensuality.

Socially robust

[Socially robust knowledge] will be context-sensitive, and instead of assisting prediction, will provide a domain of possible implications that others can draw upon in order to establish the grounds for anticipated outcomes. 14

Design knowledge must help the designer to take action and to bridge the applicability gap.


  1. Rothstein, Paul and Michelle T. Shirey. 2004. "User-Centered Research: a Status Report." Design Philosophy Papers(1). 

  2. Dreyfuss, Henry. 1955. Designing for People. New York: Simon and Schuster. 

  3. Jones, John C. 1992. Design Methods: Seeds of Human Futures. New York: John Wiley & Sons, Inc. 

  4. Koskinen, Ilpo and Katja Battarbee. 2003. "Introduction to User Experience and Empathic Design." Pp. 37-50 in Empathic Design: User Experience in Product Design, Ed. Ilpo Koskinen, Katja Battarbee, and Tuuli Mattelmäki. Finland: IT Press, Edita Prima Ltd. 

  5. Jonas, Wolfgang. 2000. "The Paradox Endeavour to Design a Foundation for a Groundless Field." International Conference on Design Education in the University. Perth. 

  6. Zwaga, Harm J. G., Boersema Theo, and Hoonhout Henriëtte C.M. 1999. Visual Information for Everyday Use: Design and Research Perspective. London, Philadelphia: Taylor & Francis Ltd.  

  7. Melican, James P. 2000. "Describing User-Centered Designing: How Design Teams Apply User Research Data in Creative Problem Solving." Dissertation. Illinois Institute of Technology, Chicago.  

  8. Love, Terrence. 1998. Social, Environmental and Ethical Factors in Engineering Design Theory: A Post-Positivist Approach. Perth, Western Australia: Praxis Education.  

  9. Buchanan, Richard. 1993. "Rhetoric and the Productive Sciences: Towards a New Program for Research in Design." Design Methodology and Relationships With Science(Eindhoven, The Netherlands, M. J. de Vries, Nigel Cross, and D. P. Grant. Dordrecht: Kluwer Academic Publishers.  

  10. Ulrich, K. T. and S. D. Eppinger. 2004. Product Design and Development. New York: McGraw- Hill/Irwin.van Lill, Cecilia F. M. 1999. "A Model for User Needs and Information Use Studies in Library and Information Services." Dissertation. University of Pretoria, South Africa.  

  11. Sugar, William A. 1998. "What Is So Good About User-Centered Design? Examining the Relative
    Effectiveness of Usability Sessions." Dissertation. Indiana University. 

  12. Toulmin, Stephen. 2001. Return to Reason. Cambridge, London: Harvard University Press.  

  13. Beyer, Hugh and Karen Holtzblatt. 1998. Contextual Design. Defining the Customer-Centered Systems. San Francisco: Morgan Kaufman Publishers. 

  14. Nowotny, Helga, Peter Scott, and Michael Gibbons. 2001. Re-Thinking Science. Cambridge, Oxford, Malden: Polity Press.