PSD paper in the upcoming ACM AudioMostly’15 Conference

The preliminary findings of the project will be presented at  the ACM AudioMostly’15 Conference, more info at

C. Erkut, S. Serafin, M.H. Andersen, and J. Sårde, “Product Sound Design: Form, Function, and Experience”, Proc. Audio Mostly, Thessaloniki, Greece (October 2015), Accepted.

Product Sound Design in the 3rd Wave HCI

About 10 years ago Susanne Bødker discussed in a NordiCHI keynote about the waves of HCI [1]. Recently, she evaluated the last decade of the 3rd Wave HCI [2]. Shortly,

the first wave was cognitive science and human factors. It was model-driven and focused on the human being as a subject to be studied through rigid guidelines, formal methods, and systematic testing, …

In the second wave, the focus was on groups working with a collection of applications. Theory focused on work settings and interaction within well-established communities of practice. Situated action, distributed cognition, and activity theory were important sources of theoretical reflection, and concepts like context came into focus in the analysis and design of human-computer interaction. Proactive methods, such as a variety of participatory design workshops, prototyping, and contextual inquiries, were added to the toolbox.

In the third wave, the use contexts and application types broadened, and intermixed, relative to the second wave’s focus on work. Technology spread from the workplace to our homes and everyday lives and culture. Research in the third wave challenged the values related to technology in the second wave (e.g., efficiency) and embraced experience and meaning-making … Early on in the third wave, second-wave methods such as participatory design were questioned, due to the perception that they were dealing only with existing (work) practices and not with emergent use.

It is intriguing to look to the Jabra products we have worked with in the PSD project from this perspective:

  1. Jabra Pulse Wireless
  2. Jabra Move Wireless
  3. Jabra Solemate

One way of thinking the headphones Pulse and Move is at the boundary of the 2nd and 3rd Waves, with mobile applications, several device support, and work-leisure transition (especially if we also consider Jabra Evolve, Move’s bigger cousin). Solemate, on the other hand is a truly 3rd wave product, that values experience, participation, and sharing.

How to visualize the changes in PSD with respect to HCI waves? One idea is to place the products on the boundaries of the waves, besides the characteristics of the 3rd wave. This would be a straightforward extension of Mark Rettig’s “Interaction Design History in a Teeny Little Nutshell” (2004), one drawing of which is reproduced below.

Retting-IXD History


[1] Bødker, S. (2006). When second wave HCI meets third wave challenges (pp. 1–8). Keynote at the Nordic Conf. Human-Computer Interaction, Oslo, Norway.

[2] Bødker, S. (2015). Third-wave HCI, 10 years later—participation and sharing. Interactions, 22(5).

Program of the Interactive Product sound design Workshop at ICAD’15, Graz, July 6, 2015

Here is the tentative program of the workshop at the following link:

09:00-09:45 Introduction to product sound
09:45-10:30 Bioinformatics and human computer interface
10:30-11:00 Coffee break
11:00-11:45 Introduction to HW and SW and case studies
11:45-12:30 Brainstorming on problems and forming groups
12:30-13:30 Lunch break
13:30-15:00 Working in groups
15:00-15:30 Coffee break
15:30-16:00 Finalization of group work
16:00-17:00 Presentations and discussions

Workshopping Product Sound Design

The PSD project was finalized with a thematic workshop at a Lynden’s Hus on May 13, 2015. Eleven participants (two female) have attended to the workshop. Most participants were from academia, with varying experience with sound and music. In the introduction phase, the key disciplines of the PSD project (product development, sound, and design) were introduced. For instance, by following the famous definition of design by H. Simon as “changing existing situations to preferred ones”, the participants discussed how sonic interaction design together with sonic branding can advance the usability and user experience of everyday products. The iterative nature of these advancements [14] were also mentioned, together with the conceptual tools, e.g., Sonic Incident, Action-Sound and Vocal Sketching [6]. As a technical tool, the Gestural Sound Toolkit [6] was introduced.

The first part of the workshop was purposefully no-tech, and followed roughly the ideation phase described in [6], with activities related to Sonic Incident and Imagining sonic interaction. Both activities fitted nicely to the Simon’s design definition above [14]. We have used some examples from [6] to elaborate the tasks.

Sonic Incident Technique

The sonic incident method facilitates participants recalling situations related to sound and music, and highlighting the desired operation of an interactive system [6]. We have asked the workshop participants to remember an inci- dent in the last two days where the sound was memorable, and guided them to concentrate on felt emotions (frustrat- ing, surprising, etc.), and to sketch the existing situation on post-its together with the sound descriptions. In particular, the minimum requirements for sound descriptions were set to contain an action, medium, and a listener. The next activity was a pair work: one member of the pair was instructed to take a post-it and describe the incident without words, whereas the other was to guess the incident aloud by interpreting the sketch, sounds, movement; by focusing particularly on emotions. After five minutes, the pair swapped the tasks. Vocal sketching was the [7] preferred method of communication; in our case too it required the participants to describe sound with sound [6].

Imagining Sonic Interaction

In this part, we have asked the participants to imagine possible interactions with their sonic incidents. Starting with the metaphor ”Superpowers” of [6], we have encour- aged the participants to imagine themselves as all-powerful beings who could create and act upon sounds. The particular solo task (5 minutes) was as follows:

Imagine you have superpowers. Through the action with your body, you can manipulate the sounds/situations in your sonic incident. So, look at your sonic incident, and imagine what happens and how it happens. It does not have to be realistic (remember your superpowers).

Realizing Sonic Interaction

In this phase of the workshop, the participants were divided into groups of four. Each group was provided with the Gestural Sound Toolkit (GST) [6] running on a laptop, a Leap Motion Sensor, and access to the internet. The groups have first discussed their imagined sonic incidents and interactions, and choose the best one to prototype. Then they have launched the GST, loaded the related sounds from to form a sound bank, dragged and dropped the sound bank to one of the synthesizers, patched the sensor data to analysis, and finally to the synthesizers. Due to the limited time, simple mappings that do not involve machine learning of gestures were encouraged. In addition, the energetic analysis provided too low output amplitudes with the Leap Motion sensor. Two groups did not use it, and one group has scaled the output amplitudes. This glitch has now been corrected in the GST code repository.

Constraining Sonic Interaction

The last part of the workshop introduced further constraints based on the product family, brand, and the target group. This part has been carried out by the industrial partners; several product families were introduced together with the actual demands from the design and engineering teams. Example cases were illustrated and reflected upon. As a task, the participants were instructed to place the sensors on actual products, and consider all the constraints. The workshop has been wrapped up with presentations, critique, and open discussion on the process.


[6] B.Caramiaux,A.Altavilla,S.Pobiner, and A. Tanaka. Form Follows Sound: Designing Interactions from Sonic Memories. In Proc. Conf. Human Factors in Computing Systems (CHI), pages 3943–3952, Seoul, Republic of Korea, 2015.

[7] I. Ekman and M. Rinott. Using vocal sketching for designing sonic interactions. In Designing Interactive Systems, Aarhus, Denmark, 2010.

[15] C. Pacione. Evolution of the mind. interactions, 17(2):6–11, 2010.

Well done! Links and References

PSD Workshop organizers thank all participants for their great work. It was fun and creative event, was it not?

Here are some links and references as promised:

  • and
  • Erkut, C., 2011. Product Sound Design: Past, Present, Future. In Akustiikkapäivät 2011. Akustiikkapäivät 2011. Tampere, Finland, pp. 37–43.
  • Rocchesso, D., Serafin, S. & Rinott, M., 2013. Pedagogical approaches to sonic interaction design. In S. Serafin & K. Franinovic, eds. Sonic Interaction Design. MIT Press
  • Serafin, S., Franinovic, K., Hermann, T., Lemaitre, G., Rinott, M., & Rocchesso, D. (2011). Sonic Interaction Design. In T. Hermann, A. Hunt, & J. G. Neuhoff, The Sonification Handbook (pp. 87–110). Berlin, Germany: Logos Publishing House.
  • Caramiaux, B., Altavilla, A., Pobiner, S., & Tanaka, A. (2015). Form Follows Sound: Designing Interactions from Sonic Memories. Proc. CHI, Seoul, Korea. See also 

More information on the Product Sound Design Workshop at Lydens Hus 13.5 at 13:00

Product Sound Design Tools of the Trade

Tools of the trade, from a previous workshop.

Here is more information and tentative program about the workshop

Time Activity Leader Activities
13:00 Introduction Cumhur, Linn SA Holm Danish Sound, PSD, Gestural Sound Toolkit
13:15 SID WS Cumhur, Stefania SonicIncident, VocalSketch, ActionSound
14:00 CP#1 GST Working, ActionSound Ready
14:15 PSD WS Michael P#1, 1 minute student projects
15:00 CP#2 Cumhur Synthesis banks loaded from
15:15 Brand WS Jonniy Audiowise Sound ID intro, case studies, P#2
16:00 CP#3 All P#2 ideas, GST material ready
16:00 Presentations
16:15 PSD Final Presentation ALL, Whitepaper#1 Wrap up
16:30 END

Few seats are still available, please signup here.


1. Why are we doing this?
    Workshops are situated and active way of knowledge production (learning by doing). This shared knowledge between Industry and Academia should enable more enjoyable and usable products in the future.
2. Did the organizers made similar workshops before?
    Yes, since 2008 on, with durations ranging from an hour to several weeks. Check, e.g.,, here, and here,
3. OK, I am interested. What should I bring with me?
    Nothing, practically. If you want to test/reuse some of the workshop at home, here are some suggestions:
– (free/open source OSC/MIDI controller for connecting to the GST)
– for Creative-Commons licensed sounds and music

A new Sonic Interaction Design Workshopping paper at CHI

A collaboration between Goldsmith EEAI and IRCAM produced a series of workshop called Form Follows Sound. The outcomes of these workshops are outlined in

Caramiaux, B., Altavilla, A., Pobiner, S., & Tanaka, A. (2015). Form Follows Sound: Designing Interactions from Sonic Memories. Presented at the Conf. Human Factors in Computing Systems.

Watch Atau Tanaka introducing one of the workshops and the general topic of embodied sonic interaction at the video below

Product sound design project

The product sound design project, a collaboration between Aalborg University Copenhagen, Jabra and Audiowise, supported by the Danish Sound Technology Network is starting on February 2015

Screen Shot 2015-02-02 at 09.19.34

PSD project in a nutshell: Development and evaluation of interactive prototypes, with the product sound design as the central activity, complemented by sonic interaction design and sound branding