Property of Researcher Salvatore Andolina & Professor Steven Dow Carnegie Mellon University: HCI Development Lab

Showcased at the User Interface Software and Technology Conference 2013

Co-design efforts attempt to account for many diverse view-points. However, design teams lack support for meaningful real-time interaction with a large community of potential stakeholders. We present Crowdboard, a novel whiteboard system that enables many potential stakeholders to provide real-time input during early-stage design activities, such as concept mapping. Local design teams develop ideas on a standard whiteboard, which is augmented with annotations and comments from online participants. The system makes it possible for design teams to solicit real-time opinions and ideas from a community of people intrinsically motivated to shape the product/ service. To evaluate the system, we staged a design session with a local three-person team and eight remote crowd participants. The study demonstrates how a crowd can shape a design discussion as it unfolds and raises a number of challenges for how to better facilitate crowd and designer interactions.


Role: UX Designer, HCI Researcher, and Front-End Developer: Launching research and studies into crowd sourcing usability. Designer iterations of the web interface & physical interface. Coding the front-end product of the project. 


The Research

Study Participants

For each of our studies, we recruited teams of 2-3 design students (12 total) with a range of design experience via social media and the university’s behavioral research participant pool, as well as 90 online participants recruited from Mechanical Turk. Online participants were mainly Americans (65%) and Indians (31%), and ranged from 19 to 68 years of (average 33). The 59% were male and 39% female. They represented a wide range of occupations (e.g. firemen, librarians managers, etc.), but a majority were in the IT (24%) or Education (13%) fields.

Study Procedure

We conducted "long” brainstorming sessions of about 35 minutes and “short” sessions of 15 minutes. In each session, we proposed one of two design briefs. One asked the team to discuss and create a concept map for the idea of traveling to a foreign country, and the other asked them to brainstorm how to improve the user experience in a cultural heritage site, the Palatine Chapel in Palermo. After each session, we asked both teams and crowds to fill out a survey. We also interviewed design teams to get a better understanding of their thoughts about the experience.

The Crowd

Our results suggest that generic crowds can be successfully engaged in synchronous design discussions with local team members. As a quantitative measure of this improvement, we observed that Crowdboard helped teams generate more ideas on average. In our experiments, 35% of the total number of ideas generated in each session comes from crowd input on average, a rate of 0.56 ideas per minute.




Real-time large-scale input from a crowd

One of the goals of Crowdboard is to enable many remote participants to provide diverse input in real-time. The main module that makes this feature possible is a web-server. It is written in Node.js and it is responsible for storing the status of the system and keeping the clients synchronized. It has been shown that Node.js is particularly suitable for data-intensive real-time  applications that run across distributed devices. To enable the communication back and forth between the client and the server we used


Integration with Physical Design Studio

Another goal of Crowdboard is to allow the team to leverage the creative crowd input while working into a physical studio. To implement this feature we designed two modules: the WhiteboardCapture module and the TouchDetection module. Both run on the local laptop.



Interactions among the remote

Online participants interact with the team by using a web interface. The left panel contains the live video broadcast of the meeting from the web camera, a group chat window and the list of online participants. The right panel contains a synchronously updated view of the studio white- board. Online participants can create a new discussion in a particular (x,y) position on the screenshot with a simple double click. They can also expand and add to an existing dis- cussion. Under the whiteboard view, online participants can manipulate a timeline interface; the videocast, the whiteboard state, and the comment threads will update appropriately.