TY - Generic T1 - The Genie in the Bottle: Different Stakeholders, Different Interpretations of Machine Learning T2 - Hawai'i International Conference on System Science Y1 - 2020 A1 - Mahboobeh Harandi A1 - Kevin Crowston A1 - Corey Jackson A1 - Carsten Østerlund AB -

Machine learning (ML) constitute an algorithmic phenomenon with some distinctive characteristics (e.g., being trained, probabilistic). Our understanding of such systems is limited when it comes to how these unique characteristics play out in organizational settings and what challenges different groups of users will face in working with them. We explore how people developing or using an ML system come to understand its capabilities and challenges. We draw on the social construction of technology tradition to frame our analysis of interviews and discussion board posts involving designers and users of a ML-supported citizen-science crowdsourcing project named Gravity Spy. Our findings reveal some of the challenges facing different relevant social groups. We find that groups with less interaction with the technology have their understanding. We find that the type of understandings achieved by groups having less interaction with the technology is shaped by outside influences rather than the specifics of the system and its role in the project. Notable, some users mistake human input for ML input. This initial understanding of how different participants understand and engage with ML point to challenges that need to be overcome to help participants deal with the opaque position ML often hold in a work system.

JF - Hawai'i International Conference on System Science CY - Wailea, HI ER - TY - JOUR T1 - Teaching Citizen Scientists to Categorize Glitches using Machine-Learning-Guided Training JF - Computers in Human Behavior Y1 - 2020 A1 - Corey Jackson A1 - Carsten Østerlund A1 - Kevin Crowston A1 - Mahboobeh Harandi A1 - Sarah Allen A1 - Sara Bahaadini A1 - Scott Coughlin A1 - Vicky Kalogera A1 - Aggelos Katsaggelos A1 - Shane Larson A1 - Neda Rohani A1 - Joshua Smith A1 - Laura Trouille A1 - Michael Zevin AB -

Training users in online communities is important for making high performing contributors. However, several conundrums exists in choosing the most effective approaches to training users. For example, if it takes time to learn to do the task correctly, then the initial contributions may not be of high enough quality to be useful. We conducted an online field experiment where we recruited users (N = 386) in a web-based citizen-science project to evaluate the two training approaches. In one training regime, users received one-time training and were asked to learn and apply twenty classes to the data. In the other approach, users were gradually exposed to classes of data that were selected by trained machine learning algorithms as being members of particular classes. The results of our analysis revealed that the gradual training produced “high performing contributors”. In our comparison of the treatment and control groups we found users who experienced gradual training performed significantly better on the task (an average accuracy of 90% vs. 54%), contributed more work (an average of 228 vs. 121 classifications), and were retained in the project for a longer period of time (an average of 2.5 vs. 2 sessions). The results suggests online production communities seeking to train newcomers would benefit from training regimes that gradually introduce them to the work of the project using real tasks.

VL - 105 ER - TY - Generic T1 - The Hermeneutics of Trace Data: Building an Apparatus Y1 - 2016 A1 - Carsten Østerlund A1 - Kevin Crowston A1 - Corey Jackson AB - When people interact via information systems, the data is captured by the systems as a side effect of the interaction. These data are increasingly interesting and available for research. In a sense, these systems become a new kind of research apparatus, and like all advances in instrumentation, open up new areas of study with the potential for discovery. While at first glance, such “big data” analysis seems to be most suitable for a positivist quantitative research approach. However, a closer inspection reveals that interpretive research strategies may better support the challenges associated with digital trace data. By merging insights from hermeneutics and sociomateriality, we argue that trace data analysis entails the building of a research apparatus. Hermeneutic principles play a key role in the application of this apparatus and allow researchers to make sense of the often partial traces left by online participants. Drawing on longitudinal trace data from a study of citizen science practices the paper illustrates the value of merging insights from hermeneutics with sociomaterial insights. The approach allows researchers to account for not only the material dynamics of digital trace data but also the temporal dimension of online practices. JF - IFIP Working Group 8.2 Working Conference ER - TY - Generic T1 - Being Present in Online Communities: Learning in Citizen Science T2 - 7th International Conference on Communities and Technologies Y1 - 2015 A1 - Gabriel Mugar A1 - Carsten Østerlund A1 - Corey Jackson A1 - Kevin Crowston AB -

How online community members learn to become valuable contributors constitutes a long-standing concern of Community & Technology researchers. The literature tends to highlight participants’ access to practice, feedback from experienced members, and relationship building. However, not all crowdsourcing environments offer participants opportunities for access, feedback, and relationship building (e.g., Citizen Science). We study how volunteers learn to participate in a citizen science project, Planet Hunters, through participant observation, interviews, and trace ethnography. Drawing on Sørensen’s sociomaterial theories of presence, we extend the notion of situated learning to include several modes of learning. The empirical findings suggest that volunteers in citizen science engage more than one form of access to practice, feedback, and relationship building. Communal relations characterize only one form of learning. Equally important to their learning are authority–subject and agent-centered forms of access, feedback, and relationship building.

JF - 7th International Conference on Communities and Technologies CY - Limerick, Ireland ER -