Boundaries of data journalism in U.S. public radio newsrooms - Jastrzebski, Stan; Henderson, Keren; McKinnon-Crowley, Jocelyn; Crowston, Kevin (Journalism)

Taming artificial intelligence: A theory of control–accountability alignment among AI developers and users - Gudela Grote; Sharon Parker; Kevin Crowston (Academy of Management Review)
The growing agency of artificial intelligence (AI) systems, more specifically systems based on machine learning, has raised concern about the security, safety, and ethical risks of AI use. We argue that core to mitigating AI risks is proper alignment of control and accountability for the stakeholders involved in AI development and use. Control enables and accountability motivates stakeholders to achieve desired and avoid undesired outcomes using AI. However, AI systems’ capabilities for autonomous adaptivity reduce control even for the experts who create them. Moreover, increasing interdependencies between AI development and use render it difficult to unambiguously locate control and accountability. In this paper, we address these challenges for mitigating AI risks by postulating decentralized forms of stakeholder governance and integrative negotiations among stakeholders during the AI life cycle as conducive to aligning control and accountability for AI development and use. Further, we specify that extensive information sharing aided by perspective taking and a shared norm of accountability facilitate integrative negotiation strategies. We conclude by discussing the implications of our theory for management scholarship on the impact of AI and identify promising avenues for future research at micro, meso, and macro levels of analysis.
Keywords: accountability; artificial intelligence; control; risk; stakeholder negotiation

Deskilling and upskilling with AI systems - Kevin Crowston; Bolici, Francesco (Information Research an International Electronic Journal, 2025)
Introduction. Deskilling is a long-standing prediction of the use of information technology, raised anew by the increased capabilities of AI (AI) systems. A review of studies of AI applications suggests that deskilling (or levelling of ability) is a common outcome but systems can also require new skills, i.e., upskilling. Method. To identify which settings are more likely to yield deskilling vs. upskilling, we propose a model of a human interacting with an AI system for a task. The model highlights the possibility for a worker to develop and exhibit (or not) skills in prompting for, and evaluation and editing of system output, thus yielding upskilling or deskilling. Findings. We illustrate these model-predicted effects on work with examples of current studies of AI-based systems. Conclusions. We discuss organizational implications of systems that deskill or upskill workers and suggest future research directions.

The Role of Human Creativity in the Presence of AI Creativity Tools at Work: A Case Study on AI-Driven Content Transformation in Journalism - Sitong Wang; Jocelyn McKinnon-Crowley; Tao Long; Kian Loong Lua; Keren Henderson; Kevin Crowston; Jeffrey V. Nickerson; Mark Hansen; Lydia B. Chilton (2025)

The task matters: The effect of perceived similarity to AI on intention to use in different task types - Qingyu Liang; Kevin Crowston; Sangseok You (Hawai'i International Conference on System Science, 2025)
With the development of AI technologies, especially generative AI (GAI) like ChatGPT, GAI is increasingly assisting people in various tasks. However, people may have different requirements for GAI when using it for different kinds of tasks. For instance, when brainstorming new ideas, people may want GAI to propose different ideas that supplement theirs with different problem-solving perspectives, but for decision-making tasks, they may prefer GAI adopt a similar problem-solving process with people to make a similar or even the same decision as they would. We conducted an online experiment examining how perceived similarities between GAI and human task-solving influence people’s intention to use GAI, mediated by trust, for four task types (creativity, planning, intellective, and decision-making tasks). We demonstrate that the effect of similarity on trust (and so intent to use AI) depends on the type of task. This paper contributes to understanding the impact of task types on the relationship between perceived similarity and GAI adoption, with implications for future use of GAI in various task contexts.

Artificial Intelligence and the Future of Citizen Science - Fortson, Lucy; Crowston, Kevin; Kloetzer, Laure; Ponti, Marisa (Citizen Science: Theory and Practice, 2024)

Editorial: Sharing work with AI: introduction to the special issue on the futures of work in the age of intelligent machines - Crowston, Kevin; Erickson, Ingrid; Nickerson, Jeffrey (Information Technology & People, 2024)

Folksonomies in crowdsourcing platforms: Three tensions associated with the development of shared language in distributed groups - Bullard, Julia; Crowston, Kevin; Jackson, Corey B.; Smith, Alexander O.; Østerlund, Carsten (The European Conference on Computer-Supported Cooperative Work (ECSCW), 2024)
Members of cooperative groups can work together more effectively if they develop a shared language, but distributed groups face barriers to doing so. To better understand how shared language can emerge in and support the work of distributed groups, we review the literature on folksonomies (a kind of shared language) in crowdsourcing systems (one type of distributed work). The review highlights three tensions associated with the development of folksonomies in crowdsourcing. The first problem is who has the power to decide on adopted terminology. Second, different users of the language may have different needs. There might be tension if people tagging objects are not the same as those using these tags to search for content. Finally, projects need to decide when to intervene to maintain a balance between a stable ontology and the ability of the project to accommodate ongoing changes. We illustrate these considerations by comparing how they are handled in the story-sharing site Archive of Our Own, the citizen-science project GravitySpy, and the photo-sharing site Flickr.

Framing and feelings on social media: The futures of work and intelligent machines - Ayşe Öcal; Kevin Crowston (Information, Technology & People, 2024)
Purpose: This study explores interpretations and feelings about futures of work and intelligent machines expressed on social media. Design/methodology/approach: We investigate public interpretations, assumptions and expectations expressed in social media conversations through which users freely share their most recent ideas. In addition to frames, this study also coded the emotions and attitudes expressed in the text data. More specifically, a corpus consisting of 998 unique Reddit post titles and their corresponding 16,611 comments were analyzed by using computer-aided textual analysis comprising a BERTopic model, and two BERT text classification models, one for emotion and the other for sentiment analysis, supported by human judgment. Finally, relationships among frames and attitudes and frames and emotions were examined. Findings: Twelve clusters were found that related to futures of work with intelligent machines. Based on the prior literature, two frames were chosen from these clusters and analyzed in detail: (1) general impacts of intelligent machines on wealth and society and (2) replacement of tasks (augmentation and substitution). The general attitude observed in conversations was positive, moreover the most common emotion category was approval. Findings also showed there are relationships between frames and attitudes and frames and emotions. Originality: This work extends the prior literature on a topic relevant for academia and industry. Findings of this research can help realize potential needs and benefits from the public’s vantage point in the case of possible transformations in the future of work with intelligent machines. The findings may also help enlighten researchers to shape research directions about futures of work. Furthermore, firms, organizations or industries may also employ framing methods to receive customers’ or workers’ responses, or even to influence the responses. Aside from the empirical findings, another crucial implication of this work is application of theory of technological frames for systematizing the interpretations of how people conceptualize the future of work with the technology of intelligent machines. This study constitutes a bridge that connects fields of IS, computational science and empirical social research.

Gravity Spy: Lessons Learned and a Path Forward - Michael Zevin; Corey B. Jackson; Zoheyr Doctor; Yunan Wu; Carsten Østerlund; L. Clifton Johnson; Christopher P. L. Berry; Kevin Crowston; Scott B. Coughlin; Vicky Kalogera; Sharan Banagiri; Derek Davis; et al. (European Physical Journal Plus, 2024)
The Gravity Spy project aims to uncover the origins of glitches, transient bursts of noise that hamper analysis of gravitational-wave data. By using both the work of citizen-science volunteers and machine-learning algorithms, the Gravity Spy project enables reliable classification of glitches. Citizen science and machine learning are intrinsically coupled within the Gravity Spy framework, with machine-learning classifications providing a rapid first-pass classification of the dataset and enabling tiered volunteer training, and volunteer-based classifications verifying the machine classifications, bolstering the machine-learning training set and identifying new morphological classes of glitches. These classifications are now routinely used in studies characterizing the performance of the LIGO gravitational-wave detectors. Providing the volunteers with a training framework that teaches them to classify a wide range of glitches, as well as additional tools to aid their investigations of interesting glitches, empowers them to make discoveries of new classes of glitches. This demonstrates that, when giving suitable support, volunteers can go beyond simple classification tasks to identify new features in data at a level comparable to domain experts. The Gravity Spy project is now providing volunteers with more complicated data that includes auxiliary monitors of the detector to identify the root cause of glitches.

Implications for hybrid newswork from the activities of local US television journalists during COVID - Kevin Crowston; Keren Henderson; Kian Loong Lua; Raghav Raheja (Journalism Practice, 2024)
In response to the COVID-19 crisis, many local television (TV) newsrooms decided to have employees work from home (WFH) or from the field rather than from the newsroom, creating a kind of hybrid work characterized by flexible work location. From a review of research on telework and WFH, we identified possible impacts of WFH on work and on workers, with a particular focus on news work and news workers. Data on the impacts of hybrid work are drawn from interviews with local television news directors and journalists in the United States and observations of remote work. We found that through creative application of technology, WFH news workers could successfully create a newscast, albeit with some concerns about story quality. However, WFH did not seem to satisfy workers’ needs for socialization or learning individually or as a group. Lifted restrictions on gatherings are mitigating some of the experienced problems, but we expect to see continued challenges to news workers’ informal learning in hybrid work settings.

Making sense of AI systems development - Mateusz Dolata; Kevin Crowston (IEEE Transactions on Software Engineering, 2024)
We identify and describe episodes of sensemaking around challenges in modern Artificial-Intelligence (AI)-based systems development that emerged in projects carried out by IBM and client companies. All projects used IBM Watson as the development platform for building tailored AI-based solutions to support workers or customers of the client companies. Yet, many of the projects turned out to be significantly more challenging than IBM and its clients had expected. The analysis reveals that project members struggled to establish reliable meanings about the technology, the project, context, and data to act upon. The project members report multiple aspects of the projects that they were not expecting to need to make sense of yet were problematic. Many issues bear upon the current-generation AI’s inherent characteristics, such as dependency on large data sets and continuous improvement as more data becomes available. Those characteristics increase the complexity of the projects and call for balanced mindfulness to avoid unexpected problems.

Mutual learning in human-AI interaction - Østerlund, Carsten; Crowston, Kevin; Jackson, Casey B.; Takou-Ayaoh, Marlene; Wu, Yunan; Katsaggelos, Aggelos K. (Trust and Reliance in Evolving Human-AI Workflows (TREW) Workshop, ACM CHI Conference, 2024)
We explore the bi-directional relationship between human and machine learning in citizen science. Theoretically, the study draws on the Zone of Proximal Development concept, which allows us to describe the augmentation of human learning by AI, human augmentation of machine learning and how tasks can be designed to facilitate co-augmentation. Methodologically, the study utilizes a design-science approach to explore the design, deployment, and evaluations of the Gravity Spy citizen science project. The findings highlight the challenges and opportunities of co-augmentation, where both humans and machines contribute to each other’s learning and capabilities.  The research contributes to the existing literature by emphasizing the role of ZPD in citizen science projects, showcasing how the concept supports ongoing learning for volunteers and keeps machine learning aligned with evolving data.

ReelFramer: Human-AI Co-Creation for News-to-Video Translation - Wang, Sitong; Menon, Samia; Long, Tao; Henderson, Keren; Li, Dingzeyu; Crowston, Kevin; Hansen, Mark; Nickerson, Jeffrey V; Chilton, Lydia B (CHI '24: CHI Conference on Human Factors in Computing SystemsProceedings of the CHI Conference on Human Factors in Computing Systems, 2024)

Supporting and augmenting human and machine learning in citizen science: Lessons from Gravity Spy - Carsten Østerlund; Kevin Crowston; Corey B. Jackson; Yunan Wu; Alexander O. Smith; Aggelos K. Katsaggelos (Citizen Science: Theory and Practice, 2024)

AngleKindling: Supporting Journalistic Angle Ideation with Large Language Models - Petridis, Savvas; Diakopoulos, Nicholas; Crowston, Kevin; Hansen, Mark; Henderson, Keren; Jastrzebski, Stan; Nickerson, Jefrey V; Chilton, Lydia B (Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023)

Data quality up to the third observing run of Advanced LIGO: Gravity Spy glitch classifications - Glanzer, J; Banagiri, S; Coughlin, S B; Soni, S; Zevin, M; Berry, C P L; Patane, O; Bahaadini, S; Rohani, N; Crowston, K; Østerlund, C (Classical and Quantum Gravity, 2023)
Keywords: FOS: Physical sciences; General Relativity and Quantum Cosmology (gr-qc); Instrumentation and Methods for Astrophysics (astro-ph.IM)

A decade of surveys on attitudes to data sharing highlights three factors for achieving open science - Joshua Borycz; Alison Specht; Kevin Crowston (LSE Impact of Social Science, 2023)

Design principles for background knowledge to enhance learning in citizen science - Kevin Crowston; Corey Brian Jackson; Isabella Corieri; Carsten Østerlund (Information for a Better World: Normality, Virtuality, Physicality, Inclusivity: 18th International Conference, iConference, 2023)
Citizen scientists make valuable contributions to science but need to learn about the data they are working with to be able to perform more advanced tasks. We present a set of design principles for identifying the kinds of background knowledge that are important to support learning at different stages of engagement, drawn from a study of how free/libre open source software developers are guided to create and use documents. Specifically, we suggest that newcomers require help understanding the purpose, form and content of the documents they engage with, while more advanced developers add understanding of information provenance and the boundaries, relevant participants and work processes. We apply those principles in two separate but related studies. In study 1, we analyze the background knowledge presented to volunteers in the Gravity Spy citizen-science project, mapping the resources to the framework and identifying kinds of knowledge that were not initially provided. In study 2, we use the principles proactively to develop design suggestions for Gravity Spy 2.0, which will involve volunteers in analyzing more diverse sources of data. This new project extends the application of the principles by seeking to use them to support understanding of the relationships between documents, not just the documents individually. We conclude by discussing future work, including a planned evaluation of Gravity Spy 2.0 that will provide a further test of the design principles.

Perceived benefits of open data are improving but scientists still lack resources, skills, and rewards - Borycz, Joshua; Olendorf, Robert; Specht, Alison; Grant, Bruce; Crowston, Kevin; Tenopir, Carol; Allard, Suzie; Rice, Natalie M.; Hu, Rachael; Sandusky, Robert J. (Humanities and Social Sciences Communications, 2023)

Surveys Provide Insight Into Three Factors That Encourage Open Data and Science - Joshua Borycz; Alison Specht; Kevin Crowston (Social Science Space, 2023)

Video for ReelFramer: Co-creating News Reels on Social Media with Generative AI - Sitong Wang; Samia Menon; Tao Long; Keren Henderson; Dingzeyu Li; Kevin Crowston; Mark Hansen; Jeffrey V. Nickerson; Lydia B. Chilton (2023)

Artificial intelligence in information systems: State of the art and research roadmap - Pär J. Ågerfalk; Kieran Conboy; Kevin Crowston; Sirkka L. Jarvenpaa; Jenny Eriksson Lundström; Patrick Mikalef; Sudha Ram (Communications of the Association for Information Systems (CAIS), 2022)
Many would argue that artificial intelligence (AI) is not just a technology but represents a paradigmatic shift in the relationship between humans and machines. Much of the literature assumes that AI-powered practices are substantially different and profoundly changes organizational structures, communication, affordances, and ecosystems. However, research in AI is often fragmented and lacks clarity. While the Information Systems (IS) field can play a pivotal role in the emergence and use of AI, there is a need for a clear direction that specifies how IS can contribute and what are to be the key research themes and questions. This paper draws on a PDW at ICIS 2020 and the discussions that followed. It summarizes and synthesizes five decades of the impact of AI on organizational practices, providing views from various perspectives. It identifies weaknesses in the current AI literature as measured against conceptual clarity, theoretical glue, cumulative tradition, parsimony, and applicability. The paper concludes by identifying direct actions that the IS research community can undertake to address these issues. The final contribution is a next-step research agenda to guide AI research in the coming years.

Communicating with the masses from isolation: What happened when local television journalists worked from home - Keren Henderson; Raghav Raheja; Kevin Crowston (Hawai'i International Conference on System Sciences, 2022)
In response to the COVID-19 crises, many local TV newsrooms decided to have employees work from home (WFH) or the field rather than from the newsroom. From a review of research on telework, we identified possible impacts of WFH on worker effectiveness, conceptualized as including output, individual satisfaction and growth, and group well-being. From a case study of a local TV newsroom and interviews with news directors, we found that WFH was successful in creating a newscast, albeit with some concerns about story quality. However, WFH did not seem to satisfy workers individually or as a group. The current lifting of restrictions on gatherings might mitigate some of the experienced problems, but we expect to see challenges to news worker learning with continued WFH.

Interdisciplinary collaboration from diverse science teams can produce significant outcomes - Alison Specht; Kevin Crowston (PLoS ONE, 2022)
Scientific teams are increasingly diverse in discipline, international scope and demographics. Diversity has been found to be a driver of innovation but also can be a source of interpersonal friction. Drawing on a mixed-method study of 22 scientific working groups, this paper presents evidence that team diversity has a positive impact on scientific output (i.e., the number of journal papers and citations) through the mediation of the interdisciplinarity of the collaborative process, as evidenced by publishing in and citing more diverse sources. Ironically these factors also seem to be related to lower team member satisfaction and perceived effectiveness, countered by the gender balance of the team. Qualitative data suggests additional factors that facilitate collaboration, such as trust and leadership. Our findings have implications for team design and management, as team diversity seems beneficial, but the process of integration can be difficult and needs management to lead to a productive and innovative process.

Project archetypes: A blessing and a curse for AI development - Mateusz Dolata; Kevin Crowston; Gerhard Schwabe (International Conference on Information Systems (ICIS), 2022)
Software projects rely on what we propose to call project archetypes, i.e., pre-existing mental images of how projects work. They guide distribution of responsibilities, planning, or expectations. However, with the technological progress, project archetypes may become outdated, ineffective, or counterproductive by impeding more adequate approaches. Understanding archetypes of software development projects is core to leverage their potential. The development of applications using machine learning and artificial intelligence provides a context in which existing archetypes might outdate and need to be questioned, adapted, or replaced. We analyzed 36 interviews from 21 projects between IBM Watson and client companies and identified four project archetypes members initially used to understand the projects. We then derive a new project archetype, cognitive computing project, from the interviews. It can inform future development projects based on AI-development platforms. Project leaders should pro-actively manage project archetypes while researchers should investigate what guides initial understandings of software projects.

Assessment, Usability, and Sociocultural Impacts of DataONE: A Global Research Data Cyberinfrastructure Initiative - Robert J. Sandusky; Suzie Allard; Lynn Baird; Leah Cannon; Kevin Crowston; Amy Forrester; Bruce Grant; Rachael Hu; Robert Olendorf; Danielle Pollock; Alison Specht; Carol Tenopir; et al. (International Journal of Digital Curation, 2021)
DataONE, funded from 2009-2019 by the U.S. National Science Foundation, is an early example of a large-scale project that built both a cyberinfrastructure and culture of data discovery, sharing, and reuse. DataONE used a Working Group model, where a diverse group of participants collaborated on targeted research and development activities to achieve broader project goals. This article summarizes the work carried out by two of DataONE’s working groups: Usability & Assessment (2009-2019) and Sociocultural Issues (2009-2014). The activities of these working groups provide a unique longitudinal look at how scientists, librarians, and other key stakeholders engaged in convergence research to identify and analyze practices around research data management through the development of boundary objects, an iterative assessment program, and reflection. Members of the working groups disseminated their findings widely in papers, presentations, and datasets, reaching international audiences through publications in 25 different journals and presentations to over 5,000 people at interdisciplinary venues. The working groups helped inform the DataONE cyberinfrastructure and influenced the evolving data management landscape. By studying working groups over time, the paper also presents lessons learned about the working group model for global large-scale projects that bring together participants from multiple disciplines and communities in convergence research.

Discovering features in gravitational-wave data through detector characterization, citizen science and machine learning - S Soni; C P L Berry; S B Coughlin; M Harandi; C B Jackson; K Crowston; C Østerlund; O Patane; A K Katsaggelos; L Trouille; V-G Baranowski; W F Domainko; et al. (Classical and Quantum Gravity, 2021)

Evaluating MIDST, a system to support stigmergic team coordination - Kevin Crowston; Jeffery Saltz; Niraj Sitaula; Yatish Hegde (Proceedings of the ACM on Human-Computer Interaction, 2021)
Data science teams working on a shared analysis face coordination problems such as dividing up the work to be done, monitoring performance and integrating the pieces. Research on distributed software development teams has raised the potential of stigmergic coordination, that is, coordination through a shared work product in place of explicit communication. The MIDST system was developed to support stigmergic coordination by making individual contributions to a shared work product visible, legible and combinable. In this paper, we present initial studies of a total of 40 student teams (24 using MIDST) that shows that teams that used MIDST did experience the intended system affordances to support their work, did seem to coordinate at least in part stigmergically and performed better on an assigned project.
Keywords: stigmergic coordination; translucency; awareness; data-science teams

Examining Open Innovation in Science (OIS): What Open Innovation can and cannot offer the science of science - Susanne Beck; Marcel LaFlamme; Carsten Bergenholtz; Marcel Bogers; Tiare-Maria Brasseur; Marie-Louise Conradsen; Kevin Crowston; Diletta Di Marco; Agnes Effert; Despoina Filiou; Lars Frederiksen; Thomas Gillier; et al. (Innovation: Organization & Management, 2021)
Scholars across disciplines increasingly hear calls for more open and collaborative approaches to scientific research. The concept of Open Innovation in Science (OIS) provides a framework that integrates dispersed research efforts aiming to understand the antecedents, contingencies, and consequences of applying open and collaborative research practices. While the OIS framework has already been taken up by science of science scholars, its conceptual underpinnings require further specification. In this essay, we critically examine the OIS concept and bring to light two key aspects: 1) how OIS builds upon Open Innovation (OI) research by adopting its attention to boundary-crossing knowledge flows and by adapting other concepts developed and researched in OI to the science context as exemplified by two OIS cases in the area of research funding; 2) how OIS conceptualises knowledge flows across boundaries. While OI typically focuses on well-defined organizational boundaries, we argue that blurry and even invisible boundaries between communities of practice may more strongly constrain flows of knowledge related to openness and collaboration in science. Given the uptake of this concept, this essay brings needed clarity to the meaning of OIS, which has no particular normative orientation toward a close coupling between science and industry. We end by outlining the essay’s contributions to OI and the science of science, as well as to science practitioners.

Functional and Visionary Leadership in Self-Managing Virtual Teams - Eseryel, U. Yeliz; Kevin Crowston; Heckman, Robert (Group & Organization Management, 2021)
In this conceptual article, we present a theory of leadership in self-managing virtual teams. We describe leadership in this setting as a process that results in the creation, reinforcement and evolution of shared mental models and shared norms that influence team member behaviour towards the successful accomplishment of shared goals. We distinguish two types of leadership. We identify leadership that works within and reinforces existing models and norms to influence team contributions as “functional” leadership. We identify leadership that results in changes in models and norms as “visionary” leadership. We propose that successful self-managing virtual teams require both types of leadership and that they will exhibit a paradoxical combination of shared, distributed functional leadership complemented by strong, concentrated and centralized visionary leadership and that visionary leadership is enabled by functional leadership in the form of substantive team member contributions.
Keywords: functional leadership; Leadership; Self-managing teams; structuration theory; virtual teams; visionary leadership

Hybrid intelligence in business networks - Ebel, Philipp; Söllner, Matthias; Leimeister, Jan Marco; Crowston, Kevin; de Vreede, Gert-Jan (Electronic Markets, 2021)

Imagine All the People: Citizen Science, Artificial Intelligence, and Computational Research - Lea A. Shanley; Lucy Fortson; Tanya Berger-Wolf; Kevin Crowston; Pietro Michelucci (A Computing Community Consortium (CCC) Quadrennial Paper, 2021)
Machine learning, artificial intelligence, and deep learning have advanced significantly over the past decade. Nonetheless, humans possess unique abilities such as creativity, intuition, context and abstraction, analytic problem solving, and detecting unusual events. To successfully tackle pressing scientific and societal challenges, we need the complementary capabilities of both humans and machines. The Federal Government could accelerate its priorities on multiple fronts through judicious integration of citizen science and crowdsourcing with artificial intelligence (AI), Internet of Things (IoT), and cloud strategies.

The Impacts of Automation on Journalists - Ayse Dalgali; Kevin Crowston (2021)
We examined journalists’ tasks as described on the Occupational Information Network (O*NET) website. The tasks we analyzed specifically are those assigned to reporters and correspondents (27-3022.00 - Reporters and Correspondents). From the tasks, we developed a workflow diagram for journalist’s work. Reviewing the previous relevant literature, we made predictions to explore which journalism tasks are more automatable and which are less automatable, and how the trending advent of technologies such automatic content generation may change journalists’ roles. Highlighting research and practical implications, we conclude the paper with limitations and areas for further research.

Occasional Groups in Crowdsourcing Platforms - Mahboobeh Harandi (School of Information Studies, 2021)
Contributors to online crowdsourcing systems generally work independently on pieces of the product but in some cases, task interdependencies may require collaboration to develop a final product. These collaborations though take a distinctive form because of the nature of crowdsourced work. Collaboration may be implicit instead of explicit. Individuals engaged in a group conversation may not stay with the group for long, i.e., the group is an “occasional group.” Occasional group interactions are often not well supported by systems, as they are not designed for team work. This dissertation examines the characteristics and work of occasional groups in the Gravity Spy citizen science project. Occasional groups in this system form to reach agreement about the description of novel categories of data that volunteers identify in the system. The author first employed virtual ethnography over six months to investigate volunteers’ interactions and to identify features of the occasional groups in this setting. Most groups were transient, interacting only for a short time to develop one product, but a few worked together repeatedly. To describe the overall process of finding new categories brings individuals to work together, the author interviewed nine active volunteers about their work practices. Volunteers individually or collectively use tools such as hashtags, collections and a search tool to identify examples of a new category and to agree on a name and description. Finally, the author investigated the details of the processes of developing proposals for four new categories over three years. She employed virtual and trace ethnography to collect messages from several discussion threads and boards to identify the analytical moves made by occasional group members in developing a new category. Volunteers would speculate on a new pattern and its causes, discuss how different categories are related and split or merge descriptions. They employed techniques such as detailed descriptions of data to create common ground, @-mention of other volunteers to increase the visibility of their work to each other and use of the category proposal as a vehicle to coordinate their actions. Findings contribute to the group literature by recognizing that groups with no formal formation and work processes are capable of doing work that would not otherwise be possible. The results advance our understanding of group categorization literature by showing how the analytical moves are different when group members work occasionally. The thesis also provides some suggestions for better support of occasional groups in crowdsourcing platforms.

Participation in Community-Based Free/Libre Open Source Software Development Tasks: The Impact of Task Characteristics - Kangning Wei; Eseryel, U. Yeliz; Kevin Crowston (Internet Research, 2021)
Prior research on participation in FLOSS development has focused mainly on factors at the individual and/or project levels. In this research, we focus on task characteristics and explore their impacts on participation in FLOSS development tasks. Analyzing tasks from five projects in two categories, we find differences in participation related to different task triggers and task topics. Further, our results suggest the mediating role of number of participants in the relationship between task characteristics and the number of messages and the moderating role of project type in the relationships between task characteristics and the number of participants.

Algorithmic Journalism and Its Impacts on Work - Ayse Dalgali; Kevin Crowston (Computation + Journalism Symposium, 2020)
In the artificial intelligence era, algorithmic journalists can produce news reports in natural language from structured data thanks to natural language generation (NLG) algorithms. This paper presents several algorithmic content generation models and discusses the impacts of algorithmic journalism on work within a framework consisting of three levels: replacing tasks of journalists, increasing efficiency, and developing new capabilities within journalism. The findings indicate that algorithmic journalism technology may lead some changes in journalism by enabling individual users to produce their own stories. This paper may contribute to an understanding of how algorithmic news is created and how algorithmic journalism technology impacts work.

Building an apparatus: Refractive, reflective and diffractive readings of trace data - Carsten Østerlund; Kevin Crowston; Corey Brian Jackson (Journal of the Association for Information Systems, 2020)
We propose a set of methodological principles and strategies for the use of trace data, i.e., data capturing performances carried out on or via information systems, often at a fine level of detail. Trace data comes with a number of methodological and theoretical challenges associated with the inseparable nature of the social and material. Drawing on Haraway and Barad’s distinctions among refraction, reflection and diffraction, we compare three approaches to trace data analysis. We argue that a diffractive methodology allows us to explore how trace data are not given but created though construction of a research apparatus to study trace data. By focusing on the diffractive ways in which traces ripple through an apparatus, it is possible to explore some of the taken-for-granted, invisible dynamics of sociomateriality. Equally, important this approach allows us to describe what and when distinctions within entwined phenomena emerge in the research process. Empirically, we illustrate the guiding principles and strategies by analyzing trace data from Gravity Spy, a crowdsourced citizen science project on Zooniverse. We conclude by suggesting that a diffractive methodology may help us draw together quantitative and qualitative research practices in new and productive ways that also raises interesting design questions.

Decision-Making Processes in Community-based Free/Libre Open Source Software Development Teams with Internal Governance: An Extension to Decision-Making Theory - Eseryel, U. Yeliz; Kangning Wei; Kevin Crowston (Communications of the Association for Information Systems, 2020)
FLOSS teams are an extreme example of distributed teams, prominent in software development. At the core of distributed team success is team decision-making and execution. The lack formal organizational structures to guide practices and the reliance on asynchronous communication might be expected to make decision making problematic. While there is a paucity of research in how organizations make IS development decisions, the research in FLOSS decision making models is limited. Decision-making literature in FLOSS teams is limited to the investigation of the distribution of decision-making power. Therefore, it is not clear which decision-making theories fit FLOSS context best, or whether novel decision-making models are required. Despite these challenges many FLOSS teams are effective. We adopted a process-based perspective to analyze decision-making in five community-based FLOSS teams. We identified five different decision-making processes, indicating FLOSS teams use multiple processes when making decisions. Decision-making behaviors were stable across projects despite different type of knowledge required. We help fill in the literature gap due to the lack of investigations the extent to which FLOSS decision mechanisms can be explained using classical decision-making theories. Practically, community and company leaders should use these decision processes to infrastructure that fits best with the FLOSS decision-making processes.

Effects of stigmergic and explicit coordination on Wikipedia article quality - Kevin Crowston; Amira Rezgui (Hawai'i International Conference on System Science, 2020)
Prior research on Wikipedia has noted the importance of both explicit coordination of edits (i.e., through the article Talk page) and stigmergic coordination (i.e., through the article itself). Using a panel data set of article quality and edits for 23 articles over time, we examine the impact of different kinds of edits on article quality. We find that stigmergically-coordinated edits seem to have the biggest effect on quality, but that explicit coordination of major edits also predicts article quality. The findings have implications for both research on coordination inWikipedia and for supporting editors.

Factors Influencing Approval of Wikipedia Bots - Ayse Dalgali; Kevin Crowston (Hawai'i International Conference on System Science, 2020)
Before a Wikipedia bot is allowed to edit, the operator of the bot must get approval. The Bot Approvals Group (BAG), a committee of Wikipedia bot developers, users and editors, discusses each bot request to reach consensus regarding approval or denial. We examine factors related to approval of a bot by analyzing 100 bots’ project pages. The results suggest that usefulness, value-based decision making and the bot’s status (e.g., automatic or manual) are related to approval. This study may contribute to understanding decision making regarding the human-automation boundary and may lead to developing more efficient bots.

The Genie in the Bottle: Different Stakeholders, Different Interpretations of Machine Learning - Mahboobeh Harandi; Kevin Crowston; Corey Jackson; Carsten Østerlund (Hawai'i International Conference on System Science, 2020)
Machine learning (ML) constitute an algorithmic phenomenon with some distinctive characteristics (e.g., being trained, probabilistic). Our understanding of such systems is limited when it comes to how these unique characteristics play out in organizational settings and what challenges different groups of users will face in working with them. We explore how people developing or using an ML system come to understand its capabilities and challenges. We draw on the social construction of technology tradition to frame our analysis of interviews and discussion board posts involving designers and users of a ML-supported citizen-science crowdsourcing project named Gravity Spy. Our findings reveal some of the challenges facing different relevant social groups. We find that groups with less interaction with the technology have their understanding. We find that the type of understandings achieved by groups having less interaction with the technology is shaped by outside influences rather than the specifics of the system and its role in the project. Notable, some users mistake human input for ML input. This initial understanding of how different participants understand and engage with ML point to challenges that need to be overcome to help participants deal with the opaque position ML often hold in a work system.

GitLab: Work where you want, when you want - Choudhury, Prithwiraj; Crowston, Kevin; Dahlander, Linus; Minervini, Marco S.; Raghuram, Sumita (Journal of Organizational Design, 2020)
GitLab is a software company that works “all remote” at the scale of more than 1000 employees located in more than 60 countries. GitLab has no physical office and its employees can work from anywhere they choose. Any step of the organizational life of a GitLab employee (e.g., hiring, onboarding and firing) is performed remotely, except for a yearly companywide gathering. GitLab strongly relies on asynchronous coordination, allowing employees to work anytime they want. After highlighting some of the main practices implemented by GitLab to effectively work all remotely and asynchronously, I asked renowned organizational scientists their thoughts on this interesting case and to question the generalizability of the all remote asynchronous model. Understanding whether and under what conditions this model can succeed can be of guidance for organizational designers that are now considering different remote models in response of the COVID-19 shock and its aftermath.

Impacts of the Use of Machine Learning on Work Design - Kevin Crowston; Bolici, Francesco (8th International Conference on Human-Agent Interaction, 2020)
The increased pervasiveness of technological advancements in automation makes it urgent to address the question of how work is changing in response. Focusing on applications of machine learning (ML) to automate information tasks, we draw on a simple framework for identifying the impacts of an automated system on a task that suggests 3 patterns for the use of ML—decision support, blended decision making and complete automation. In this paper, we extend this framework by considering how automation of one task might have implications for interdependent tasks and how automation applies to coordination mechanisms.
Keywords: artificial intelligence; automation; Coordination; machine learning; work design

Knowledge Tracing to Model Learning in Online Citizen Science Projects - Kevin Crowston; Carsten Østerlund; Tae Kyoung Lee; Corey Brian Jackson; Mahboobeh Harandi; Sarah Allen; Sara Bahaadini; Scott Coughlin; Aggelos Katsaggelos; Shane Larson; Neda Rohani; Joshua Smith; et al. (IEEE Transactions on Learning Technologies, 2020)
We present the design of a citizen science system that uses machine learning to guide the presentation of image classification tasks to newcomers to help them more quickly learn how to do the task while still contributing to the work of the project. A Bayesian model for tracking volunteer learning for training with tasks with uncertain outcomes is presented and fit to data from 12,986 volunteer contributors. The model can be used both to estimate the ability of volunteers and to decide the classification of an image. A simulation of the model applied to volunteer promotion and image retirement suggests that the model requires fewer classifications than the current system.

Lessons for Supporting Data Science from the Everyday Automation Experience of Spell-Checkers - Kevin Crowston (Automation Experience across Domains (AutomationXP20), CHI'20 Workshop, 26 April 2020, Virtual, 2020)
We apply two theoretical frameworks to analyze spell-checkers as a form of automation and apply the lessons learned to analyze opportunities to support data science. The analysis distinguishes between automation of analysis to suggest actions  and automation of implementation of actions. Having the automation work in the same space as users (e.g., editing the same document) supports stigmergic coordination between the two, but attention is needed to ensure that the contributions can be combined and have a recognizable form that indicates their purpose.

MIDST: An enhanced development environment that improves the maintainability of a data science analysis - Jeffery Saltz; Kevin Crowston; Heckman, Robert; Yatish Hegde (International Journal of Information Systems and Project Management, 2020)
With the increasing ability to generate actionable insight from data, the field of data science has seen significant growth. As more teams develop data science solutions, the analytical code they develop will need to be enhanced in the future, by an existing or a new team member. Thus, the importance of being able to easily maintain and enhance the code required for an analysis will increase. However, to date, there has been minimal research on the maintainability of an analysis done by a data science team. To help address this gap, data science maintainability was explored by (1) creating a data science maintainability model, (2) creating a new tool, called MIDST (Modular Interactive Data Science Tool), that aims to improve data science maintainability, and then (3) conducting a mixed method experiment to evaluate MIDST. The new tool aims to improve the ability of a team member to update and rerun an existing data science analysis by providing a visual data flow view of the analysis within an integrated code and computational environment. Via an analysis of the quantitative and qualitative survey results, the experiment found that MIDST does help improve the maintainability of an analysis. Thus, this research demonstrates the importance of enhanced tools to help improve the maintainability of data science projects.

Shifting forms of Engagement: Volunteer Learning in Online Citizen Science - Corey Brian Jackson; Carsten Østerlund; Mahboobeh Harandi; Kevin Crowston; Laura Trouille (Proceedings of the ACM on Human-Computer Interaction, 2020)
Open collaboration platforms involve people in many tasks, from editing articles to analyzing datasets. To facilitate mastery of these practices, communities offer a number of learning resources, ranging from project-defined FAQs to individually-oriented search tools and communal discussion boards. However, it is not clear which project resources best support participant learning, overall and at different stages of engagement with the project. We draw on Sørensen’s framework of forms of presence to distinguish three forms of engagement with learning resources: authoritative, agent-centered and communal. We analyzed trace data from the GravitySpy citizen-science project using a mixed-effects logistic regression with volunteer performance as an outcome variable. The findings suggest that engagement with authoritative resources (e.g., those constructed by project organizers) facilitates performance initially. However, as tasks become more difficult, volunteers seek and benefit from engagement with their own agent-centered resources and community generated resources. These findings suggest a broader scope for the design of learning resources for online communities.

Teaching Citizen Scientists to Categorize Glitches using Machine-Learning-Guided Training - Corey Jackson; Carsten Østerlund; Kevin Crowston; Mahboobeh Harandi; Sarah Allen; Sara Bahaadini; Scott Coughlin; Vicky Kalogera; Aggelos Katsaggelos; Shane Larson; Neda Rohani; Joshua Smith; et al. (Computers in Human Behavior, 2020)
Training users in online communities is important for making high performing contributors. However, several conundrums exists in choosing the most effective approaches to training users. For example, if it takes time to learn to do the task correctly, then the initial contributions may not be of high enough quality to be useful. We conducted an online field experiment where we recruited users (N = 386) in a web-based citizen-science project to evaluate the two training approaches. In one training regime, users received one-time training and were asked to learn and apply twenty classes to the data. In the other approach, users were gradually exposed to classes of data that were selected by trained machine learning algorithms as being members of particular classes. The results of our analysis revealed that the gradual training produced “high performing contributors”. In our comparison of the treatment and control groups we found users who experienced gradual training performed significantly better on the task (an average accuracy of 90% vs. 54%), contributed more work (an average of 228 vs. 121 classifications), and were retained in the project for a longer period of time (an average of 2.5 vs. 2 sessions). The results suggests online production communities seeking to train newcomers would benefit from training regimes that gradually introduce them to the work of the project using real tasks.

Classifying the unknown: Discovering novel gravitational-wave detector glitches using similarity learning - Scott Coughlin; Sara Bahaadini; Neda Rohani; Michael Zevin; Patane, Oli; Mahboobeh Harandi; Corey Brian Jackson; Noroozi, V.; Sarah Allen; Areeda, J.; Coughlin, M.; Ruiz, P.; et al. (Physical Review D, 2019)
The observation of gravitational waves from compact binary coalescences by LIGO and Virgo has begun a new era in astronomy. A critical challenge in making detections is determining whether loud transient features in the data are caused by gravitational waves or by instrumental or environmental sources. The citizen-science project Gravity Spy has been demonstrated as an efficient infrastructure for classifying known types of noise transients (glitches) through a combination of data analysis performed by both citizen volunteers and machine learning. We present the next iteration of this project, using similarity indices to empower citizen scientists to create large data sets of unknown transients, which can then be used to facilitate supervised machine-learning characterization. This new evolution aims to alleviate a persistent challenge that plagues both citizen-science and instrumental detector work: the ability to build large samples of relatively rare events. Using two families of transient noise that appeared unexpectedly during LIGO's second observing run, we demonstrate the impact that the similarity indices could have had on finding these new glitch types in the Gravity Spy program.

Comprehensive collaboration plans: Practical considerations spanning across individual collaborators to institutional supports - Hall, Kara L.; Vogel, Amanda L.; Crowston, Kevin (Strategies for Team Science Success, 2019)
This chapter provides a framework for integrating and applying the principles and strategies for effective team science that are described in this volume. The framework, called Collaboration Planning, aims to guide a deliberative approach to assess and plan for ten key influences on both scientific and collaborative success. These influences range from the initial scientific rationale for a team science approach to the collaboration readiness of participating individuals and institutions to team communication and coordination mechanisms to quality improvement for team functioning. The Collaboration Planning framework guides current or future collaborators through dialogue and planning around each influence. It draws their attention to key issues for consideration related to each influence, and facilitates discussion of how to leverage facilitating factors and plan for, or mitigate, challenges. Decisions are captured in a resulting written document called the Collaboration Plan. The Collaboration Plan summarizes the various ways the team plans to build the foundation for, and support, effective collaboration across the lifespan of the team science initiative. Collaboration Plans can be used in multiple ways. The Plans' core function is as a roadmap to facilitate effective team formation and functioning. The Plan also can be used for benchmarking or guiding quality improvement-oriented evaluation. Collaboration Plans also can be used to communicate a team’s likelihood of collaborative success, goals, and needs to a wide variety of audiences, including funders, current and future team members, stakeholders in the team’s success, and organizational leaders. In addition, they can be used as models to guide future teams in laying the foundation for success.

Coordinating Advanced Crowd Work: Extending Citizen Science - Kevin Crowston; Mitchell, Erica Michelle; Carsten Østerlund (Citizen Science: Theory and Practice, 2019)
Crowdsourcing work with high levels of coupling between tasks poses challenges for coordination. This paper presents a study of two online citizen science projects that involved volunteers in such tasks: not just analyzing bulk data but also interpreting data and writing a paper for publication in one project and identifying new classes of data in the other. However, extending the reach of citizen science adds tasks with more dependencies, which calls for more elaborate coordination mechanisms but the relationship between the project and volunteers limits how work can be coordinated. Contrariwise, a mismatch between dependencies and available coordination mechanisms can be expected to lead to performance problems. The results of the study offer recommendations for design of citizen science projects for advanced tasks.

Coordination in OSS 2.0: ANT Approach - Sangseok You; Kevin Crowston; Jeffery Saltz; Yatish Hegde (Proceedings of the 52nd Hawai'i International Conference on System Sciences (HICSS-52), 2019)
Open source software projects are increasingly driven by a combination of independent and professional developers, the former volunteers and the later hired by a company to contribute to the project to support commercial product development. This mix of developers has been referred to as OSS 2.0. However, we do not fully understand the multi-layered coordination spanning individuals, teams, and organizations. Using Actor-Network Theory (ANT), we describe how coordination and power dynamics unfold among developers and how different tools and artifacts both display activities and mediate coordination efforts. Internal communication within an organization was reported to cause broken links in the community, duplication of work, and political tensions. ANT shows how tools and code can exercise agency and alter a software development process as an equivalently active actor of the scene. We discuss the theoretical and practical implications of the changing nature of open source software development.
Keywords: actor-network theory; free/libre open source; Stigmergy

Documentation and Access to Knowledge in Online Communities: Know Your Audience and Write Appropriately? - Carsten Østerlund; Kevin Crowston (Journal of the American Society for Information Science and Technology, 2019)
Virtual collaborations bring together people who must work together despite having varied access to and understanding of the work at hand. In many cases, the collaboration is technology supported, meaning that the work is done through shared documents of various kinds. We develop a framework articulating the characteristics of documents supporting collaborators with asymmetric access to knowledge versus those with symmetric access to knowledge. Drawing on theories about document genre, boundary objects, and provenance, we hypothesize that documents supporting asymmetric collaborators are likely to articulate or prescribe their own (1) purpose, (2) context of use, (3) content and form, and (4) provenance in greater detail than documents supporting symmetric collaborators. We explore these hypotheses through content analysis of documents and instructions from a variety of free/libre open source projects (FLOSS). We present findings consistent with the hypotheses developed as well as results extending beyond our theory-derived assumptions. As participants gradually gain access to knowledge, the study suggests, prescriptions about the content of documents become less important compared to prescriptions about the context, provenance, and process of work. The study suggests new directions for research on communications in virtual collaborations, as well as advice for those supporting such collaborations.

Helping data science students develop task modularity - Jeffery Saltz; Heckman, Robert; Kevin Crowston; Sangseok You; Yatish Hegde (Proceedings of the 52nd Hawai'i International Conference on System Sciences (HICSS-52), 2019)
This paper explores the skills needed to be a data scientist. Specifically, we report on a mixed method study of a project-based data science class, where we evaluated student effectiveness with respect to dividing a project into appropriately sized modular tasks, which we termed task modularity. Our results suggest that while data science students can appreciate the value of task modularity, they struggle to achieve effective task modularity. As a first step, based our study, we identified six task decomposition best practices. However, these best practices do not fully address this gap of how to enable data science students to effectively use task modularity. We note that while computer science/information system programs typically teach modularity (e.g., the decomposition process and abstraction), and there remains a need identify a corresponding model to that used for computer science / information system students, to teach modularity to data science students.
Keywords: data science; modularity; Stigmergy

The impact of initial group characteristics on quality in online communities of creation - Amira Rezgui; Kevin Crowston; Nicolas Jullien (2019)
Open projects aimed at creating new knowledge (also known as online communities of creation) are increasingly central in the production of new and innovative knowledge. In our research study, we are interested in the impact of initial group characteristics on the quality of the output. We studied in particular Wikipedia in different three languages: Arabic, Romanian and Thai. Our results confirm the importance of the initial project group. We found a positive impact of a large initial group formed by members with an intermediate level of diversity in the focus of their editing but having an equal and lower level of longevity.

Impacts of machine learning on work - Kevin Crowston; Bolici, Francesco (Proceedings of the 52nd Hawai'i International Conference on System Sciences (HICSS-52), 2019)
The increased pervasiveness of technological advancements in automation makes it urgent to address the question of how work is changing in response. Focusing on applications of machine learning (ML) that automate information tasks, we present a simple framework for identifying the impacts of an automated system on a task. From an analysis of popular press articles about ML, we develop 3 patterns for the use of ML--decision support, blended decision making and complete automation--with implications for the kinds of tasks and systems. We further consider how automation of one task might have implications for other interdependent tasks. Our main conclusion is that designers have a range of options for systems and that automation of tasks is not the same as automation of work.
Keywords: artificial intelligence; automation; machine learning; work design

Linguistic adoption in online citizen science: A structurational perspective - Corey Brian Jackson; Carsten Østerlund; Mahboobeh Harandi; Dhruv Kharwar; Kevin Crowston (International Conference on Information Systems, 2019)
For peer-production projects to be successful, members must develop a specific and common language that enables them to cooperate. We address the question of what factors affect the development of shared language in open peer production communities? Answering this question is important because we want the communities to be productive even when self-managed, which requires understanding how shared language emerges. We examine this question using a structurational lens in the setting of a citizen science project. Examining the use of words in the Gravity Spy citizen science project, we find that many words are reused and that most novel words that are introduced are not picked up, showing reproduction of structure. However, some novel words are used by others, showing an evolution of the structure. Participants with roles closer to the science are more likely to have their words reused, showing the mutually reinforcing nature of structures of signification, legitimation and domination.

Sharing open deep learning models - Ayse Dalgali; Kevin Crowston (Proceedings of the 52nd Hawai'i International Conference on System Sciences (HICSS-52), 2019)
We examine how and why trained deep learning (DL) models are shared, and by whom, and why some developers share their models while others do not. Prior research has examined sharing of data and software code, but DL models are a hybrid of the two. The results from a Qualtrics survey administered to GitHub users and academics who publish on DL show that a diverse population shares DL models, from students to computer/data scientists. We find that motivations for sharing include: increasing citation rates; contributing to the collaboration of developing new DL models; encouraging to reuse; establishing a good reputation; receiving feedback to improve the model; and personal enjoyment. Reasons for not sharing include: lack of time; thinking that their models would not be interesting for others; and not having permission for sharing. The study contributes to our understanding of motivations for participating in a novel form of peer-production.

﻿﻿Socio-technical affordances for stigmergic coordination implemented in MIDST, a tool for data-science teams - Kevin Crowston; Jeffery Saltz; Amira Rezgui; Yatish Hegde; Sangseok You (Proceedings of the ACM, 2019)
We present a conceptual framework for socio-technical affordances for stigmergic coordination, that is, coordination supported by a shared work product. Based on research on free/libre open source software development, we theorize that stigmergic coordination depends on three sets of socio-technical affordances: the visibility and combinability of the work, along with defined genres of work contributions. As a demonstration of the utility of the developed framework, we use it as the basis for the design and implementation of a system, MIDST, that supports these affordances and that we thus expect to support stigmergic coordination. We describe an initial assessment of the impact of the tool on the work of project teams of three to six data-science students that suggests that the tool was useful but also in need of further development. We conclude with plans for future research and an assessment of theory-driven system design.
Keywords: stigmergic coordination; translucency; awareness; data-science teams

Appealing to different motivations in a message to recruit citizen scientists: results of a field experiment - Lee, Tae Kyoung; Kevin Crowston; Mahboobeh Harandi; Carsten Østerlund; Grant Miller (Journal of Science Communication, 2018)
This study examines the relative efficacy of citizen science recruitment messages appealing to four motivations that were derived from previous research on motives for participation in citizen-science projects. We report on an experiment (N=36,513) that compared the response to email messages designed to appeal to these four motives for participation. We found that the messages appealing to the possibility of contributing to science and learning about science attracted more attention than did one about helping scientists but that one about helping scientists generated more initial contributions. Overall, the message about contributing to science resulted in the largest volume of contributions and joining a community, the lowest. The results should be informative to those managing citizen-science projects.
Keywords: Citizen Science

Coordinating advanced crowd work: Extending citizen science - Kevin Crowston; Mitchell, Erica Michelle; Carsten Østerlund (Hawai'i International Conference on System Sciences, 2018)
Crowdsourcing work with high levels of coupling between tasks poses challenges for coordination. This paper presents a study of an online citizen science project that involved volunteers in such tasks: not just analyzing bulk data but also interpreting data and writing a paper for publication. However, extending the reach of citizen science adds tasks with more dependencies, which calls for more elaborate coordination mechanisms but the relationship between the project and volunteers limits how work can be coordinated. Contrariwise, a mismatch between dependencies and available coordination mechanisms can be expected to lead to performance problems. The results of the study offer recommendations for design of crowdsourcing of more complex tasks.

Did they login? Patterns of anonymous contributions to online communities - Corey Brian Jackson; Kevin Crowston; Carsten Østerlund (Proceedings of the ACM on Human-Computer Interaction, 2018)
Researchers studying user behaviors in online communities often conduct analyses of events collected in system logs, e.g., a system’s record of a comment post or of a contribution. However, analysis of user behaviors is more difficult if users make contributions without being logged-in (i.e., anonymously). Since a user’s account will not be associated with contributions that user makes anonymously, conclusions about user behaviors that look only at attributed actions might not account for a user’s full experience. To understand the impacts of anonymous contributions on research, we conducted an analysis of system logs containing anonymous activities in two online citizen science projects. By linking anonymous events with user IDs we found that (1) many users contribute anonymously, though with varied patterns of contribution; and (2) including anonymous activities alter conclusions made about users’ experience with the project. These results suggest that researchers of human behaviors in online communities should consider the possible impacts of anonymous interaction on their ability to draw conclusions about user behaviors in these settings.

Folksonomies to support coordination and coordination of folksonomies - Corey Brian Jackson; Kevin Crowston; Carsten Østerlund; Mahboobeh Harandi (Computer Supported Cooperative Work, 2018)
Members of highly-distributed groups in online production communities face challenges in achieving coordinated action. Existing CSCW research highlights the importance of shared language and artifacts when coordinating actions in such settings. To better understand how such shared language and artifacts are, not only a guide for, but also a result of collaborative work we examine the development of folksonomies (i.e., volunteer-generated classification schemes) to support coordinated action. Drawing on structuration theory, we conceptualize a folksonomy as an interpretive schema forming a structure of signification. Our study is set in the context of an online citizen-science project, Gravity Spy, in which volunteers label "glitches" (noise events recorded by a scientific instrument) to identify and name novel classes of glitches. Through a multi-method study combining virtual and trace ethnography, we analyze folksonomies and the work of labelling as mutually constitutive, giving folksonomies a dual role: an emergent folksonomy supports the volunteers in labelling images at the same time that the individual work of labelling images supports the development of a folksonomy. However, our analysis suggests that the lack of supporting norms and authoritative resources (structures of legitimation and domination) undermines the power of the folksonomy and so the ability of volunteers to coordinate their decisions about naming novel glitch classes. These results have implications design. If we hope to support the development of emergent folksonomies online production communities need to facilitate 1) tag gardening, a process of consolidating overlapping terms of artifacts; 2) demarcate a clear home for discourses around folksonomy disagreements; 3) highlight clearly when decisions have been reached; and 4) inform others about those decisions.

Stages of motivation for contributing user-generated content: A theory and empirical test - Kevin Crowston; Fagnot, Isabelle (International Journal of Human-Computer Studies, 2018)
User-generated content (UGC) projects involve large numbers of mostly unpaid contributors collaborating to create content. Motivation for such contributions has been an active area of research. In prior research, motivation for contribution to UGC has been considered a single, static and individual phenomenon. In this paper, we argue that it is instead three separate but interrelated phenomena. Using the theory of helping behaviour as a framework and integrating social movement theory, we propose a stage theory that distinguishes three separate sets (initial, sustained and meta) of motivations for participation in UGC. We test this theory using a data set from a Wikimedia Editor Survey (Wikimedia Foundation, 2011). The results suggest several opportunities for further refinement of the theory but provide support for the main hypothesis, that different stages of contribution have distinct motives. The theory has implications for both researchers and practitioners who manage UGC projects.

Stigmergic coordination in Wikipedia - Amira Rezgui; Kevin Crowston (OpenSym, 2018)
We examine a novel approach to coordination, namely stigmergic coordination, that is, coordination mediated by changes to a shared work product. Stigmergy stands in contrast to the two coordination approaches identified in the existing literature on coordination, explicit coordination, based on direct communication through discussion page or user talk pages, and implicit coordination, based on unspoken expectations and shared mental models of the task to be accomplished. We look for evidence of stigmergic coordination in the context of Wikipedia, as one of the most successful experiments in online collaborative knowledge building. Using a novel approach to identifying edits to the same part of a Wikipedia article, we show that a majority of edits to two example articles are not associated with discussion on the article Talk page, suggesting the possibility of stigmergic coordination. However, some amount of discussion does seem to be related to article quality, suggesting the limits to this approach to coordination.
Keywords: Coordination; distributed groups; on-line epistemic community; Stigmergy; Wikipedia

Attitudes and norms affecting scientists’ data reuse - Renata G. Curty; Kevin Crowston; Alison Specht; Bruce Grant; Elizabeth D. Walton (PLoS ONE, 2017)
The value of sharing data comes partly from the data’s being reused by other scientists, but questions regarding attitudes and norms that predict scientists’ data reuse remain open. We test the relationship between scientists’ beliefs and attitudes towards data reuse and their self-reported data reuse behaviour using responses to selected questions from a worldwide survey developed and administered by the DataONE Usability and Assessment Working Group. The data suggest first that data sharing and data reuse are largely separate phenomena. Second, the perceived efficacy of data reuse for answering research questions was found to be one of the strongest predictors of reuse behaviour. On the other hand, expressed lack of trust in reused data and perceived norms against data reuse did not seem to deter respondents from reuse. Finally, reported use of models and remote-sensed data was associated with more reuse. The results suggest that data reuse would be encouraged by demonstrations of the value and addressing norms about this practice.

Blending machine and human learning processes - Kevin Crowston; Carsten Østerlund; Lee, Tae Kyoung (Hawai'i International Conference on System Sciences, 2017)
Citizen science projects rely on contributions from volunteers to achieve their scientific goals and so face a dilemma: providing volunteers with explicit training might increase the quality of contributions, but at the cost of losing the work done by newcomers during the training period, which for many is the only work they will contribute to the project. Based on research in cognitive science on how humans learn to classify images, we have designed an approach to use machine learning to guide the presentation of tasks to newcomers that help them more quickly learn how to do the image classification task while still contributing to the work of the project. A Bayesian model for tracking this learning is presented.

Comparing data science project management methodologies via a controlled experiment - Jeffery Saltz; Shamshurin, Ivan; Kevin Crowston (Hawai'i International Conference on System Sciences (HICSS-50), 2017)

Core-periphery communication and the success of free/libre open source software projects - Kevin Crowston; Shamshurin, Ivan (Journal of Internet Services and Applications, 2017)
We examine the relationship between communications by core and peripheral members and Free/Libre Open Source Software project success. The study uses data from 74 projects in the Apache Software Foundation Incubator. We conceptualize project success in terms of success building a community, as assessed by graduation from the Incubator. We compare successful and unsuccessful projects on volume of communication and on use of inclusive pronouns as an indication of efforts to create intimacy among team members. An innovation of the paper is that use of inclusive pronouns is measured using natural language processing techniques. We also compare the volume and content of communication produced by core (committer) and peripheral members and by those peripheral members who are later elected to be core members. We find that volume of communication is related to project success but use of inclusive pronouns does not distinguish successful projects. Core members exhibit more contribution and use of inclusive pronouns than peripheral members.
Keywords: Apache Software Foundation; communication; core and periphery; free/libre open source software (FLOSS); inclusive pronouns; natural language processing; project success

Gamers, citizen scientists, and data: Exploring participant contributions in two games with a purpose - Nathan Prestopnik; Kevin Crowston; Wang, Jun (Computers in Human Behavior, 2017)
Two key problems for crowd-sourcing systems are motivating contributions from participants and ensuring the quality of these contributions. Games have been suggested as a motivational approach to encourage contribution, but attracting participation through game play rather than intrinsic interest raises concerns about the quality of the contributions provided. These concerns are particularly important in the context of citizen science projects, when the contributions are data to be used for scientific research. To assess the validity of concerns about the effects of gaming on data quality, we compare the quality of data obtained from two citizen science games, one a “gamified” version of a species classification task and one a fantasy game that used the classification task only as a way to advance in the game play. Surprisingly, though we did observe cheating in the fantasy game, data quality (i.e., classification accuracy) from participants in the two games was not significantly different. As well, data from short-time contributors was also at a usable level of accuracy. Finally, learning did not seem to affect data quality in our context. These findings suggest that various approaches to gamification can be useful for motivating contributions to citizen science projects.

Gravity Spy: Humans, machines and the future of citizen science - Kevin Crowston; The Gravity Spy Team (ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW 2017), 2017)
Gravity Spy is a citizen science project that draws on the contributions of both humans and machines to achieve its scientific goals. The system supports the Laser Interferometer Gravitational Observatory (LIGO) by classifying “glitches” that interfere with observations. The system makes three advances on the current state of the art: explicit training for new volunteers, synergy between machine and human classification and support for discovery of new classes of glitch. As well, it provides a platform for human-centred computing research on motivation, learning and collaboration. The system has been launched and is currently in operation.

Gravity Spy: Integrating Advanced LIGO Detector Characterization, Machine Learning, and Citizen Science - Michael Zevin; Scott Coughlin; Sara Bahaadini; Emre Besler; Neda Rohani; Sarah Allen; Miriam Cabero; Kevin Crowston; Aggelos Katsaggelos; Shane Larson; Tae Kyoung Lee; Chris Lintott; et al. (Classical and Quantum Gravity, 2017)

Lessons Learned from a Decade of FLOSS Data Collection - Crowston, Kevin; Squire, Megan (Big Data Factories, 2017)

Levels of trace data for social and behavioural science research - Kevin Crowston (Big Data Factories: Collaborative Approaches, 2017)
The explosion of data available from online systems such as social media is creating a wealth of trace data, that is, data that record evidence of human activity. The volume of data available offers great potential to advance social and behavioural science research. However, the data are of a very different kind than more conventional social and behavioural science data, posing challenges to use. This paper adopts a data framework from Earth Observation science and applies it to trace data to identify possible issues in analyzing trace data. Application of the framework also reveals issues for sharing and reusing data.

A pragmatic approach to managing enterprise IT infrastructures in the era of consumerization and individualization of IT - Mohammad Hossein Jarrahi; Kevin Crowston; Bondar, Kateryna; Bernhard Katzy (International Journal of Information Management, 2017)
Historically, organizations owned and controlled the information technologies (IT) their employees used: telephone, inter-office memos, mainframes and timesharing systems. Today, employees often want to use their own IT: not only personal smart phones and tablets, but also Twitter and Google Docs. This new trend can diversify and extend enterprise IT infrastructure, but leaves organizations struggling with technology uses that they cannot control. With the emergence of new technological paradigms in consumer markets and organizations, the management of IT infrastructure requires a more pragmatic and holistic approach that goes beyond simple technological considerations. In this paper, we present a three-part framework—technology, people and practice—that helps managers understand and mitigate these tensions. Drawing on two empirical studies of European executives and consultants form multiple management consulting firms, the paper further outlines changes taking place along the three aspects of the framework. It concludes by discussing three distinct approaches to the management of organizational IT infrastructure (passive, reactive, and pragmatic), and by offering greater insight regarding a pragmatic approach.

Pursuing best performance in research data management by using the Capability Maturity Model and rubrics - Jian Qin; Kevin Crowston; Arden Kirkland (Journal of eScience Librarianship, 2017)
OBJECTIVE: To support assessment and improvement of research data management (RDM) practices to increase the reliability of RDM, this paper describes the development of a capability maturity model (CMM) for RDM. Improved RDM is now a critical need, but low awareness of—or indeed lack of—data management is still common among research projects. METHODS: A CMM includes four key elements: key practices, key process areas, maturity levels and and generic processes. These elements were determined for RDM by a review and synthesis of the published literature on and best practices for RDM. RESULTS: The RDM CMM includes five chapters describing five key process areas for research data management: 1) data management in general; 2) data acquisition, processing and quality assurance; 3) data description and representation; 4) data dissemination; and 5) repository services and preservation. In each chapter, key data management practices are organized into four groups according to the CMM’s generic processes: commitment to perform, ability to perform, tasks performed and process assessment (combining the original measurement and verification). For each of practice, the document provides a rubric to help projects or organizations assess their level of maturity in RDM. CONCLUSIONS: By helping organizations identify areas of strength and weakness, the RDM CMM provides guidance on where effort is needed to improve the practice of RDM.

Recruiting messages matter: Message strategies to attract citizen scientists - Lee, Tae Kyoung; Kevin Crowston; Carsten Østerlund; Grant Miller (ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW 2017), 2017)
Although participation of citizen scientists is critical for a success of citizen science projects (a distinctive form of crowdsourcing), little attention has been paid to what types of messages can effectively recruit citizen scientists. Derived from previous studies on citizen scientists’ motivations, we created and sent participants one of four recruiting messages for a new project, Gravity Spy, appealing to different motivations (i.e., learning about science, social proof, contribution to science, and altruism). Counter to earlier studies on motivation, our results showed that messages appealing to learning, contribution and social proof were more effective than a message appealing to altruism. We discuss the inconsistency between the present and prior study results and plans for future work.

Roles and politeness behavior in community-based Free/Libre Open Source Software development - Kangning Wei; Kevin Crowston; Eseryel, U. Yeliz; Heckman, Robert (Information and Management, 2017)
Community-based Free/Libre Open Source Software (FLOSS) development relies on contributions from both core and peripheral members. Prior research on core-periphery has focused on software coding-related behaviors. We study how core-periphery roles are related to social-relational behavior in terms of politeness behavior. Data from two FLOSS projects suggest that both core and peripheral members use more positive politeness strategies than negative strategies. Further, core and peripheral members use different strategies to protect positive face in positive politeness, which we term respect and intimacy respectively. Our results contribute to FLOSS research and politeness theory.
Keywords: Core-periphery structure; Open source software development; Politeness behavior

Work features to support stigmergic coordination in distributed teams - Kevin Crowston; James Howison; Bolici, Francesco; Carsten Østerlund (Academy of Management Annual Meeting, 2017)
When work products are shared via a computer system, members of distributed teams can see the work products produced by remote colleagues as easily as those from local colleagues. Drawing on coordination theory and work in computer-supported cooperative work (CSCW), we theorize that these work products can provide information to support team coordination, that is, that work can be coordinated through the outcome of the work itself, a mode of coordination analogous to the biological process of stigmergy. Based on studies of documents and work, we postulate three features of work products that enable them to support team coordination, namely having a clear genre, being visible and mobile, and being combinable. These claims are illustrated with examples drawn from free/libre open source software development teams. We conclude by discussing how the proposed theory might be empirically tested.
Keywords: Coordination; Stigmergy

Alignment in an inter-organisational network: the case of ARC transistance - Bernhard Katzy; Sung, Gordon; Kevin Crowston (European Journal of Information Systems, 2016)
We consider the processes of achieving alignment in coordinated inter-organizational networks through a case study of a system development project in ARC Transistance, a network of European automobile clubs that cooperate to provide pan-European service. The theoretical contribution of the paper is, first, an extended strategic alignment model for inter-organizational networks that distinguishes between integration of IS with business strategy and infrastructure, and what we label `accordance' between the strategies and infrastructures of the network and the member firms. Second, we propose that for a network organization, network and member strategies might be complementary as well as tightly coupled. We similarly argue that IS architectures for networks should strive for being `business strategy-neutral' to more easily accommodate the diversity of members. Finally, we discuss how the process of developing a network information system can be a driver towards network alignment, but how the lack of effective governance structures makes alignment harder to achieve.

Core-Periphery Communication and the Success of Free/Libre Open Source Software Projects - Kevin Crowston; Shamshurin, Ivan (IFIP International Conference on Open Source Systems, 2016)
We examine the relationship between communications by core and peripheral members and Free/Libre Open Source Software project success. The study uses data from 74 projects in the Apache Software Foundation Incubator. We conceptualize project success in terms of success building a community, as assessed by graduation from the Incubator. We compare successful and unsuccessful projects on volume of communication by core (committer) and peripheral community members and on use of inclusive pronouns as an indication of efforts to create intimacy among team members. An innovation of the paper is that use of inclusive pronouns is measured using natural language processing techniques. We find that core and peripheral members differ in their volume of contribution and in their use of inclusive pronouns, and that volume of communication is related to project success.

Encouraging Work in Citizen Science: Experiments in Goal Setting and Anchoring - Corey Brian Jackson; Kevin Crowston; Gabriel Mugar; Carsten Østerlund (ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW), 2016)
This paper describes the results of an online field experiment where we designed and analyzed the effects of a goal-setting tracker in an online citizen science project - Floating Forest. The design of our tracker was influenced by psychology theories of anchoring and goal-setting. Our results of our experiment revealed: (1) setting goals increases annotations in a session; (2) numeric anchors influence goals; and (3) participants in the treatment who saw a prompt but did not set a goal, contributed more annotations than the participants in the control group. Our research shows how goal-setting and anchoring combine to increase work in online communities.

“Guess what! You’re the first to see this event”: Increasing Contribution to Online Production Communities - Corey Brian Jackson; Kevin Crowston; Gabriel Mugar; Carsten Østerlund (ACM Group, 2016)
In this paper, we describe the results of an online field experiment examining the impacts of messaging about task novelty on the volume of volunteers’ contributions to an online citizen science project. Encouraging volunteers to provide a little more content as they work is an attractive strategy to increase the community’s output. Prior research found that an important motivation for participation in online citizen science is the wonder of being the first person to observe a particular image. To appeal to this motivation, a pop-up message was added to an online citizen science project that alerted volunteers when they were the first to annotate a particular image. Our analysis reveals that new volunteers who saw these messages increased the volume of annotations they contributed. The results of our study suggest an additional strategy to increase the amount of work volunteers contribute to online communities and citizen science projects specifically.

The Hermeneutics of Trace Data: Building an Apparatus - Carsten Østerlund; Kevin Crowston; Corey Jackson (IFIP Working Group 8.2 Working Conference, 2016)
When people interact via information systems, the data is captured by the systems as a side effect of the interaction. These data are increasingly interesting and available for research. In a sense, these systems become a new kind of research apparatus, and like all advances in instrumentation, open up new areas of study with the potential for discovery. While at first glance, such “big data” analysis seems to be most suitable for a positivist quantitative research approach. However, a closer inspection reveals that interpretive research strategies may better support the challenges associated with digital trace data. By merging insights from hermeneutics and sociomateriality, we argue that trace data analysis entails the building of a research apparatus. Hermeneutic principles play a key role in the application of this apparatus and allow researchers to make sense of the often partial traces left by online participants. Drawing on longitudinal trace data from a study of citizen science practices the paper illustrates the value of merging insights from hermeneutics with sociomaterial insights. The approach allows researchers to account for not only the material dynamics of digital trace data but also the temporal dimension of online practices.

Inter-team coordination in large-scale agile development: A test of organizational discontinuity theory - Kevin Crowston; Chudoba, Katherine M.; Mary Beth Watson-Manheim; Pouya Rahmati (The Large-Scale Agile Development Workshop, XP Conference, 2016)
We draw on Organizational Discontinuity Theory (ODT) to identify factors that increase communication and coordination problems between teams working on large software development projects. ODT posits that faced with a disruption in the expected flow of communication, called a discontinuity, individuals must make sense of the disruption to address the problem. They may be motivated to pay more attention to the situation and consider alternative actions to deal with the discontinuity, leading to the emergence of continuities, which are new behaviors, group practices and expectations. Continuities reduce or eliminate the attention and effort required to understand and manage the situation associated with problematic discontinuities. We propose a mixed-method study based on this model to examine the effects of discontinuities and the development of continuities on inter-team coordination in large-scale agile software development.

Manifesto on Engineering Academic Software (Dagstuhl Perspectives Workshop 16252) - Alice Allen; Cecilia Aragon; Christoph Becker; Jeffrey Carver; Andrei Chiş; Benoit Combemale; Mike Croucher; Kevin Crowston; Daniel Garijo; Ashish Gehani; Carole Goble; Robert Haines; et al. (Dagstuhl Manifestos, 2016)
Software is often a critical component of scientific research. It can be a component of the academic research methods used to produce research results, or it may itself be an academic research result. Software, however, has rarely been considered to be a citable artifact in its own right. With the advent of open-source software, artifact evaluation committees of conferences, and journals that include source code and running systems as part of the published artifacts, we foresee that software will increasingly be recognized as part of the academic process. The quality and sustainability of this software must be accounted for, both a priori and a posteriori. The Dagstuhl Perspectives Workshop on “Engineering Academic Software” has examined the strengths, weaknesses, risks, and opportunities of academic software engineering. A key outcome of the workshop is this Dagstuhl Manifesto, serving as a roadmap towards future professional software engineering for software-based research instruments and other software produced and used in an academic context. The manifesto is expressed in terms of a series of actionable “pledges” that users and developers of academic research software can take as concrete steps towards improving the environment in which that software is produced.

Stigmergic coordination in FLOSS development teams: Integrating explicit and implicit mechanisms - Bolici, Francesco; James Howison; Kevin Crowston (Cognitive Systems Research, 2016)
Keywords: Coordination; Stigmergy

Which Way Did They Go? Newcomer Movement through the Zooniverse - Corey Brian Jackson; Carsten Østerlund; Maidel, Veronica; Kevin Crowston; Gabriel Mugar (19th ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW), 2016)
Research on newcomer roles in peer production sites (e.g., Wikipedia) is characterized by a broad and relatively well-articulated set of functionally and culturally recognizable roles. But not all communities come with well-defined roles that newcomers can aspire to occupy. The present study explores activity clusters newcomers create when faced with few recognizable roles to fill and limited access to other participants’ work that serves as an exemplar. Drawing on a mixed method research design, we present findings from an analysis of 1,687 newcomers’ sessions in a citizen science project. Combining session- and individual-level analysis produced three findings (1) newcomers activities manifest a diverse range of session types; (2) Newcomers toggle between light work sessions and more involved types of production or community engagement; (3) an interesting relationship between high-level contributors who do a lot of work but little talk and a small group that does a lot of talk but less work. The former group draws heavily on posts contributed by the latter group. Identifying shifts and regularities in contribution facilitate improved mechanisms for engaging participants and the design of online citizen science communities.

Assessing IS research impact - Niederman, Fred; Kevin Crowston; Koch, Hope; Krcmar, Helmut; Powell, Philip; E. Burton Swanson (Communications of the Association for Information Systems, 2015)
Based on the International Conference on Information Systems’ (ICIS) 2013 senior scholars’ forum, this paper shares insights on IS research impact assessment. We define research impact as conducting research that makes a difference to individuals, businesses, industries and societies. While assessment groups like AACSB (the Association to Advance Collegiate Schools of Business) want scholars to make an impact, sometimes they operationalize impact in ways that may encourage scholars to pursue research goals tangential to making a difference. The purpose of this paper is to stimulate thinking in the IS community on creating research assessment techniques that encourage our scholars to make a difference.

Being Present in Online Communities: Learning in Citizen Science - Gabriel Mugar; Carsten Østerlund; Corey Jackson; Kevin Crowston (7th International Conference on Communities and Technologies, 2015)
How online community members learn to become valuable contributors constitutes a long-standing concern of Community & Technology researchers. The literature tends to highlight participants’ access to practice, feedback from experienced members, and relationship building. However, not all crowdsourcing environments offer participants opportunities for access, feedback, and relationship building (e.g., Citizen Science). We study how volunteers learn to participate in a citizen science project, Planet Hunters, through participant observation, interviews, and trace ethnography. Drawing on Sørensen’s sociomaterial theories of presence, we extend the notion of situated learning to include several modes of learning. The empirical findings suggest that volunteers in citizen science engage more than one form of access to practice, feedback, and relationship building. Communal relations characterize only one form of learning. Equally important to their learning are authority–subject and agent-centered forms of access, feedback, and relationship building.

Motivations for sustained participation in crowdsourcing: The role of talk in a citizen science case study - Corey Brian Jackson; Carsten Østerlund; Gabriel Mugar; Kevin Crowston; Katie DeVries Hassman (Proceedings of the Forty-eighth Hawai'i International Conference on System Sciences (HICSS-48), 2015)
The paper explores the motivations of volunteers in a large crowd sourcing project and contributes to our understanding of the motivational factors that lead to deeper engagement beyond initial participation. Drawing on the theory of legitimate peripheral participation (LPP) and the literature on motivation in crowd sourcing, we analyze interview and trace data from a large citizen science project. The analyses identify ways in which the technical features of the projects may serve as motivational factors leading participants towards sustained participation. The results suggest volunteers first engage in activities to support knowledge acquisition and later share knowledge with other volunteers and finally increase participation in Talk through a punctuated process of role discovery.

Open Source Technology Development - Kevin Crowston (Handbook of Science and Technology Convergence, 2015)

Perceived Discontinuities and Continuities in Transdisciplinary Scientific Working Groups - Kevin Crowston; Alison Specht; Carol Hoover; Chudoba, Katherine M.; Mary Beth Watson-Manheim (Science of the Total Environment, 2015)

“Personas” to Support Development of Cyberinfrastructure for Scientific Data Sharing - Kevin Crowston (Journal of e-Science Librarianship, 2015)
Objective: To ensure that cyberinfrastructure for sharing scientific data is useful, system developers need to understand what scientists and other intended users do with data as well as the attitudes and beliefs that shape that use. This paper introduces personas—detailed descriptions of an “archetypical user of a system”—as an approach for capturing and sharing knowledge about potential system users. Setting: Personas were developed to support development of the ‘DataONE’ (Data Observation Network for Earth) project, which has developed and deployed a sustainable long-term data preservation and access network to ensure the preservation and access to multi-scale, multi-discipline, and multi-national environmental and biological science data (https://www.dataone.org/what-dataone) (Michener et al. 2012). Methods: Personas for DataONE were developed based on data from surveys and interviews done by members of DataONE working groups along with sources such as usage scenarios for DataONE and the Data Conservancy project and the Purdue Data Curation Profiles (Witt et al. 2009). Results: A total of 11 personas were developed: five for various kinds of research scientists (e.g., at different career stages and using different types of data); a science data librarian; and five for secondary roles. Conclusion: Personas were found to be useful for helping developers and other project members to understand users and their needs. The developed DataONE personas may be useful for others trying to develop systems or programs for scientists involved in data sharing.

The Rise and Fall of an Online Project. Is Bureaucracy Killing Efficiency in Open Knowledge Production? - Nicolas Jullien; Kevin Crowston; Felipe Ortega (OpenSym, 2015)

Social Networks and the Success of Market Intermediaries: Evidence from the US Residential Real Estate Industry - Kevin Crowston; Sawyer, Steve; Rolf Wigand (The Information Society, 2015)
We examine the roles of market intermediaries in brokering connections among otherwise disconnected social networks. Market intermediaries are usually thought of as simply bringing together buyers and sellers with whom they have weak ties. However, intermediaries may also connect principals with other professionals who can provide assistance with the transaction. Because they are involved in repeated transactions, market intermediaries generally have strong ties with such professionals. We address the question of which set of relations—weak ties to buyers and sellers or strong ties to other professionals—are more important to the success of market intermediaries, using data from the US residential real estate industry—a setting in which transactions are complex and market intermediaries are common. From a national survey of 525 realtors, we find that strong tie relations are more important than weak ties as predictor of the market intermediary’s income, counter to the general wisdom about real estate in particular and market intermediaries more generally. This finding suggests that market intermediaries may become more successful by developing strong relations with other related professionals. The strong-tie arrangements among professional market intermediaries may behave like ‘quasi-firms’ that help buyers and sellers navigate complex market transactions.

Surveying the citizen science landscape - Wiggins, Andrea; Kevin Crowston (First Monday, 2015)
Citizen science has seen enormous growth in recent years, in part due to the influence of the Internet, and a corresponding growth in interest. However, the few stand-out examples that have received attention from media and researchers are not representative of the diversity of the field as a whole, and therefore may not be the best models for those seeking to study or start a citizen science project. In this work, we present the results of a survey of citizen science project leaders, identifying sub-groups of project types according to a variety of features related to project design and management, including funding sources, goals, participant activities, data quality processes, and social interaction. These combined features highlight the diversity of citizen science, providing an overview of the breadth of the phenomenon and laying a foundation for comparison between citizen science projects and to other online communities.

A Capability Maturity Model for Research Data Management - Jian Qin; Kevin Crowston; Arden Kirkland (Syracuse University School of Information Studies, 2014)
Objective: To support the assessment and improvement of research data management (RDM) practices to increase its reliability, this paper describes the development of a capability maturity model (CMM) for RDM. Improved RDM is now a critical need, but low awareness of – or lack of – data management is still common among research projects. Methods: A CMM includes four key elements: key practices, key process areas, maturity levels, and generic processes. These elements were determined for RDM by a review and synthesis of the published literature on and best practices for RDM. Results: The RDM CMM includes five chapters describing five key process areas for research data management: 1) data management in general; 2) data acquisition, processing, and quality assurance; 3) data description and representation; 4) data dissemination; and 5) repository services and preservation. In each chapter, key data management practices are organized into four groups according to the CMM’s generic processes: commitment to perform, ability to perform, tasks performed, and process assessment (combining the original measurement and verification). For each area of practice, the document provides a rubric to help projects or organizations assess their level of maturity in RDM. Conclusions: By helping organizations identify areas of strength and weakness, the RDM CMM provides guidance on where effort is needed to improve the practice of RDM.

Collaboration through superposition: How the IT artifact as an object of collaboration affords technical interdependence without organizational interdependence - James Howison; Kevin Crowston (MIS Quarterly, 2014)
This paper develops a theory of collaboration through superposition: the process of depositing separate layers on top of each other over time. The theory is developed in a study of development of community-based Free and Open Source Software (FLOSS), through a research arc of discovery (participant observation), replication (two archival case studies) and formalization (a model of developer choices). The theory explains two key findings: 1) the overwhelming majority of work is accomplished with only a single programmer working on a task and 2) when tasks appear too large for an individual they are more likely to be deferred until they are easier, rather than being undertaken through structured teamwork. It is theorized that this way of organizing is key to successful open collaboration where the IT artifact is the object of collaboration, because it allows the co-production of technically interdependent artifacts through motivationally interdependent work. The affordances of software as an object of collaboration are used as a framework to analyze efforts to learn from FLOSS in other domains of work and in the IS function of for-profit organizations.

Design of an Active Learning System with Human Correction for Content Analysis - Jasy Liew Suet Yan; McCracken, Nancy; Kevin Crowston (Workshop on Interactive Language Learning, Visualization, and Interfaces, 52nd Annual Meeting of the Association for Computational Linguistics, 2014)
Our research investigation focuses on the role of humans in supplying corrected examples in active learning cycles, an important aspect of deploying active learning in practice. In this paper, we discuss sampling strategies and sampling sizes in setting up an active learning system for human experiments in the task of content analysis, which involves labeling concepts in large volumes of text. The cost of conducting comprehensive human subject studies to experimentally determine the effects of sampling sizes and sampling sizes is high. To reduce those costs, we first applied an active learning simulation approach to test the effect of different sampling strategies and sampling sizes on machine learning (ML) performance in order to select a smaller set of parameters to be evaluated in human subject studies.

Digital assemblages: Evidence and theorizing from the computerization of the U.S. residential real estate industry - Sawyer, Steve; Kevin Crowston; Rolf Wigand (New Technology, Work and Employment, 2014)
The contribution of this paper is to theorize on the roles information and communication technologies (ICT) play in reshaping work arrangements and specifically to advance the concept of a digital assemblage as a lens for this analysis. We pursue an alternative conceptualization of the role of ICT: computerization. The fundamental premise of computerization is that actors are embedded in transactions and that ICT are taken up and used to support this embedding rather than for purposes of strict economic rationality. This work draws on data from a study of the U.S. residential real estate industry, which serves here as a “living laboratory” for studying information-intensive industries.

Editorial: The role of information systems in enabling open innovation - Eoin Whelan; Kieran Conboy; Kevin Crowston; Lorraine Morgan; Matti Rossi (Journal of the Association for Information Systems, 2014)

Exploring data quality in games with a purpose - Nathan Prestopnik; Kevin Crowston; Wang, Jun (iConference, 2014)
A key problem for crowd-sourcing systems is motivating contributions from participants and ensuring the quality of these contributions. Games have been suggested as a motivational approach to encourage contribution, but attracting participation through game play rather than scientific interest raises concerns about the quality of the data provided, which is particularly important when the data are to be used for scientific research. To assess whether these concerns are justified, we compare the quality of data obtained from two citizen science games, one a “gamified” version of a species classification task and one a fantasy game that used the classification task only as a way to advance in the game play. Surprisingly, though we did observe cheating in the fantasy game, data quality (i.e., classification accuracy) from participants in the two games was not significantly different. As well, the quality of data from short-time contributors was at a usable level of accuracy. These findings suggest that various approaches to gamification can be useful for motivating contributions to citizen science projects.

Group maintenance in technology-supported distributed teams - Kangning Wei; Kevin Crowston; Li, Na; Heckman, Robert (Information & Management, 2014)
In this paper we investigate group maintenance behavior in community-based Free/Libre Open Source Software (FLOSS) development teams. Adopting a sociolinguistic perspective, we conceptualize group maintenance behavior as interpersonal communication tactics—specifically, social presence and politeness tactics—that help maintain relationships among group members. Developer email messages were collected from two FLOSS projects with different development status and content-analyzed to identify frequently-used group maintenance tactics. We then compared the two projects on the group maintenance tactics used, finding differences that reflect changes in the project work practices. Our work contributes theoretically to FLOSS research and has practical implications for FLOSS practitioners.

How to Write a Collaboration Plan - Kara L. Hall; Kevin Crowston; Amanda L. Vogel (Team Science Toolkit, 2014)
Collaboration plans are written documents that investigators may use as a “roadmap” for future collaborations. Funding agencies may ask investigators to submit Collaboration Plans as part of their funding applications, analogous to submitting research plans. Submitted collaboration plans can then be used by reviewers to help assess the capacity of a proposed team to collaboratively execute its scientific objectives. Collaboration plans address a range of issues relevant to laying the foundation for the collaboration, implementing and managing the collaboration, and engaging in quality improvement activities specific to collaborative interactions. These plans identify existing supports and challenges relevant to the collaboration, and describe a program of action that will be implemented to help support smooth collaboration. This working document, called “How to Write a Collaboration Plan” is a product of a federal subcommittee on Collaboration and Team Science. The document provides guidance for writing a collaboration plan. It identifies ten key aspects of collaboration planning, and highlights specific issues for investigators to consider related to each of the ten aspects of planning. Collaboration planning may benefit any scientific endeavor that includes two or more investigators working together. Though as a proposed scientific collaboration grows in scope and size, such plans become increasingly important. More information on the origins of this document: The White House Office of National Science and Technology Policy’s (OSTP) NITRD Program (Networking and Information Technology Research and Development Program) provides a forum where many federal agencies come together to coordinate their networking and information technology (IT) research and development (R&D) efforts. (More information at: https://www.nitrd.gov.) Team Science is of particular interest, given the prevalence of virtual collaboration in IT R&D. In response, the NITRD Coordination Group on Social, Economic, and Workforce Implications of IT and IT Workforce Development (NITRD-SEW), developed a subcommittee on Collaboration and Team Science. The subcommittee includes members from the National Institutes of Health (NIH), National Science Foundation (NSF), Department of Justice (DOJ), NASA, and other federal agencies. In 2014, the subcommittee hosted a series of topical meetings on enhancing support for collaboration in science, which resulted in this document, “How to Write a Collaboration Plan”, authored by subcommittee co-chairs Dr. Kara Hall (NIH) and Dr. Kevin Crowston (NSF), along with subcommittee member Dr. Amanda Vogel (Leidos Biomed).

Optimizing Features in Active Machine Learning for Complex Qualitative Content Analysis - Jasy Liew Suet Yan; McCracken, Nancy; Shichun Zhou; Kevin Crowston (Workshop on Language Technologies and Computational Social Science, 52nd Annual Meeting of the Association for Computational Linguistics, 2014)
We propose a semi-automatic approach for content analysis that leverages machine learning (ML) being initially trained on a small set of hand-coded data to perform a first pass in coding, and then have human annotators correct machine annotations in order to produce more examples to retrain the existing model incrementally for better performance. In this “active learning” approach, it is equally important to optimize the creation of the initial ML model given less training data so that the model is able to capture most if not all positive examples, and filter out as many negative examples as possible for human annotators to correct. This paper reports our attempt to optimize the initial ML model through feature exploration in a complex content analysis project that uses a multidimensional coding scheme, and contains codes with sparse positive examples. While different codes respond optimally to different combinations of features, we show that it is possible to create an optimal initial ML model using only a single combination of features for codes with at least 100 positive examples in the gold standard corpus.

Planet Hunters and Seafloor Explorers: Legitimate Peripheral Participation Through Practice Proxies in Online Citizen Science - Gabriel Mugar; Carsten Østerlund; Katie DeVries Hassman; Kevin Crowston; Corey Brian Jackson (17th ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW 2014), 2014)
Making the traces of user participation in primary activities visible in online crowdsourced initiatives has been shown to help new users understand the norms of participation but participants do not always have access to others’ work. Through a combination of virtual and trace ethnography we explore how new users in two online citizen science projects engage other traces of activity as a way of compensating. Merging the theory of legitimate peripheral participation with Erickson and Kellogg’s theory of social translucence we introduce the concept of practice proxies; traces of user activities in online environment that act as resources to orient newcomers towards the norms of practice. Our findings suggest that newcomers seek out practice proxies in the social features of the projects that highlight contextualized and specific characteristics of primary work practice.

Semi-Automatic Content Analysis of Qualitative Data - Jasy Liew Suet Yan; McCracken, Nancy; Kevin Crowston (iConference, 2014)

Boundary-Spanning Documents in Online FLOSS Communities: Does One Size Fit All? - Carsten Østerlund; Kevin Crowston (Forty-sixth Hawai'i International Conference on System Sciences (HICSS-46), 2013)

Forgotten island: A story-driven citizen science adventure - Nathan Prestopnik; Souid, Dania (CHI '13 Extended Abstracts on Human Factors in Computing Systems, 2013)
Forgotten Island, a citizen science video game, is part of an NSF-funded design science research project, Citizen Sort. It is a mechanism to help life scientists classify photographs of living things and a research tool to help HCI and information science scholars explore storytelling, engagement, and the quality of citizenproduced data in the context of citizen science.

Motivation and data quality in a citizen science game: A design science evaluation - Kevin Crowston; Nathan Prestopnik (Forty-sixth Hawai'i International Conference on System Sciences (HICSS-46), 2013)

Open Source Software Adoption: A Technological Innovation Perspective - Kevin Crowston; François Deltour; Nicolas Jullien (Association Information et Management, 2013)
This research-in-progress aims to indentify the salient factors explaining adoption of open source software (OSS), as a technological innovation. The theoretical background of the paper is based on the technological innovation literature. We choose to focus on the open ERP case, as it is considered as a promising innovation for firms – especially medium firms - but open ERP also faces numerous challenges. The paper provides a framework and a method for investigation that has to be implemented.

Sustainability of Open Collaborative Communities: Analyzing Recruitment Efficiency - Kevin Crowston; Nicolas Jullien; Felipe Ortega (Technology Innovation Management Review, 2013)
Extensive research has been conducted over the past years to improve our understanding of sustainability conditions for large-scale collaborative projects, especially from an economic and governance perspective. However, the influence of recruitment and retention of participants in these projects has received comparatively less attention from researchers. Nevertheless, these concerns are significant for practitioners, especially regarding the apparently decreasing ability of the main open online projects to attract and retain new contributors. A possible explanation for this decrease is that those projects have simply reached a mature state of development. Marwell and Oliver (1993) and Oliver, Marwell, and Teixeira (1985) note that, at the initial stage in collective projects, participants are few and efforts are costly; in the diffusion phase, the number of participants grows, as their efforts are rewarding; and in the mature phase, some inefficiency may appear as the number of contributors is greater than required for the work. In this article, we examine this possibility. We use original data from 36 Wikipedias in different languages to compare their efficiency in recruiting participants. We chose Wikipedia because the different language projects are at different states of development, but are quite comparable on the other aspects, providing a test of the impact of development on efficiency. Results confirm that most of the largest Wikipedias seem to be characterized by a reduced return to scale. As a result, we can draw interesting conclusions that can be useful for practitioners, facilitators, and managers of collaborative projects in order to identify key factors potentially influencing the adequate development of their communities over the medium-to-long term.

Is Wikipedia Inefficient? Modelling Effort and Participation in Wikipedia - Kevin Crowston; Nicolas Jullien; Felipe Ortega (Forty-sixth Hawai'i International Conference on System Sciences (HICSS-46), 2013)

Amazon Mechanical Turk: A research tool for organizations and information systems scholars - Kevin Crowston (IFIP Working Group 8.2 Conference: Shaping the Future of ICT Research: Methods and Approaches, 2012)

Citizen science system assemblages: Understanding the technologies that support crowdsourced science - Nathan Prestopnik; Kevin Crowston (iConference 2012, 2012)
We explore the nature of technologies to support citizen science, a method of inquiry that leverages the power of crowds to collect and analyze scientific data. We evaluate these technologies as system assemblages, collections of interrelated functionalities that support specific activities in pursuit of overall project goals. The notion of system assemblages helps us to explain how different citizen science platforms may be comprised of widely varying functionalities, yet still support relatively similar goals. Related concepts of build vs. buy and web satisfiers vs. web motivators are used to explore how different citizen science functionalities may lead to successful project outcomes. Four detailed case studies of current citizen science projects encompassing a cross-section of varying project sizes, resource levels, technologies, and approaches to inquiry help us to answer the following research questions: 1) What do typical system assemblages for citizen science look like? 2) What factors influence the composition of a system assemblage for citizen science? 3) What effect does the assemblage composition have on scientific goals, participant support, motivation, and satisfaction? and 4) What are the design implications for the system assemblage perspective on citizen science technologies?

Free/Libre Open Source Software Development: What we know and what we do not know - Kevin Crowston; Kangning Wei; James Howison; Wiggins, Andrea (ACM Computing Surveys, 2012)
We review the empirical research on Free/Libre and Open Source Software (FLOSS) development and assess the state of the literature. We develop a framework for organizing the literature based on the input-mediator-output- input (IMOI) model from the small groups literature. We present a quantitative summary of articles selected for the review and then discuss findings of this literature categorized into issues pertaining to inputs (e.g., member characteristics, technology use and project characteristics), processes (software development and social processes), emergent states (e.g., trust and task related states) and outputs (e.g. team performance, FLOSS implementation and project evolution). Based on this review, we suggest topics for future research, as well as identifying methodological and theoretical issues for future inquiry in this area, including issues relating to sampling and the need for more longitudinal studies.

The future of citizen science: emerging technologies and shifting paradigms - Newman, Greg; Wiggins, Andrea; Crall, Alycia; Graham, Eric; Newman, Sarah; Kevin Crowston (Frontiers in Ecology and the Environment, 2012)
Citizen science creates a nexus between science and education that, when coupled with emerging technologies, expands the frontiers of ecological research and public engagement. Using representative technologies and other examples, we examine the future of citizen science in terms of its research processes, program and participant cultures, and scientific communities. Future citizen-science projects will likely be influenced by sociocultural issues related to new technologies and will continue to face practical programmatic challenges. We foresee networked, open science and the use of online computer/video gaming as important tools to engage non-traditional audiences, and offer recommendations to help prepare project managers for impending challenges. A more formalized citizen-science enterprise, complete with networked organizations, associations, journals, and cyberinfrastructure, will advance scientific research, including ecology, and further public education.

Goals and tasks: Two typologies of citizen science projects - Wiggins, Andrea; Kevin Crowston (Forty-fifth Hawai’i International Conference on System Sciences (HICSS-45), 2012)
Citizen science is a form of research collaboration involving members of the public in scientific research projects to address real-world problems. Often organized as a virtual collaboration, these projects are a type of open movement, with collective goals addressed through open participation in research tasks. We conducted a survey of citizen science projects to elicit multiple aspects of project design and operation. We then clustered projects based on the tasks performed by participants and on the project’s stated goals. The clustering results group projects that show similarities along other dimensions, suggesting useful divisions of the projects.

Innovation in academic-industry partnerships: Measuring the challenges to effective performance - Mary Beth Watson-Manheim; Chudoba, Katherine M.; Kevin Crowston (Academy of Management Conference, Technology and Innovation Management Division, 2012)

Knowledge Portals: Components, Functionalities, and Deployment Challenges - Kevin Crowston; Claudia Löbbecke (International Conference on Information Systems, 2012)

Perceived discontinuities and constructed continuities in virtual work - Mary Beth Watson-Manheim; Chudoba, Katherine M.; Kevin Crowston (Information Systems Journal, 2012)

Poster: Socially intelligent computing for coding of qualitative data - Kevin Crowston; McCracken, Nancy (Syracuse University School of Information Studies, 2012)

Poster: Socially intelligent computing to support citizen science - Kevin Crowston (Syracuse University School of Information Studies, 2012)

Purposeful gaming & socio-computational systems: A citizen science design case - Nathan Prestopnik; Kevin Crowston (Group '12 Conference, 2012)

Team dynamics in long-standing technology-supported virtual teams - Misiolek, Nora; Kevin Crowston; Joshua Seymour (Academy of Management Annual Meeting, Organizational Behaviour Division, 2012)

Using natural language processing for qualitative data analysis - Kevin Crowston; Allen, Eileen E.; Heckman, Robert (International Journal of Social Research Methodology, 2012)
Social researchers often apply qualitative research methods to study groups and their communications artefacts. The use of computer-mediated communications has dramatically increased the volume of text available, but coding such text requires considerable manual effort. We discuss how systems that process text in human languages (i.e., natural language processing, NLP) might partially automate content analysis by extracting theoretical evidence. We present a case study of the use of NLP for qualitative analysis in which the NLP rules showed good performance on a number of codes. With the current level of performance, use of an NLP system could reduce the amount of text to be examined by a human coder by an order of magnitude or more, potentially increasing the speed of coding by a comparable degree. The paper is significant as it is one of the first to demonstrate the use of high-level NLP techniques for qualitative data analysis.

Boundary-spanning documents in online communities (Research-in-Progress) - Carsten Østerlund; Kevin Crowston (International Conference on Information Systems (ICIS), 2011)
Online communities bring together people with varied access to and understanding of the work at hand, who must collaborate through documents of various kinds. We develop a framework articulating the characteristics of documents supporting collaborators with asymmetric access to knowledge versus those with symmetric knowledge. Drawing on theories about document genre, boundary objects and provenance, we hypothesize that documents supporting asymmetric groups are likely to articulate or prescribe their own 1) purpose, 2) context of use and 3) content and form and 4) provenance in greater detail than documents used by people with symmetric access to knowledge. We are testing these hypotheses through content analysis of documents and instructions from a variety of free/libre open source projects. We present preliminary findings consistent with the hypotheses developed. The completed study will suggest new directions for research on communications in online communities, as well as advice for those supporting such communities.
Keywords: boundary objects; Documents; Genre; Online communities; provenance

A capability maturity model for scientific data management: Evidence from the literature - Kevin Crowston; Jian Qin (American Society for Information Science and Technology Annual Meeting, 2011)

Citizen science system assemblages: Toward greater understanding of technologies to support crowdsourced science - Nathan Prestopnik; Kevin Crowston (Syracuse University School of Information Studies, 2011)
We explore the nature of technologies to support citizen science, a method of inquiry that leverages the power of crowds to collect and analyze scientific data. We evaluate these technologies as system assemblages, collections of interrelated functionalities that support specific activities in pursuit of overall project goals. The notion of system assemblages helps us to explain how different citizen science platforms may be comprised of widely varying functionalities, yet still support relatively similar goals. Related concepts of build vs. buy, support for science vs. support for participants, and web satisfiers vs. web motivators are used to explore how different citizen science functionalities may lead to successful project outcomes. Four detailed case studies of current citizen science projects encompassing a cross-section of varying project sizes, resource levels, technologies, and approaches to inquiry help us to answer the following research questions: 1) What factors influence the composition of a system assemblage for citizen science? 2) What do typical system assemblages for citizen science look like? 3) What effect does the assemblage composition have on scientific goals, participant support, motivation, and satisfaction? and 4) What are the design implications for the system assemblage perspective on citizen science technologies?

Citizen Science System Assemblages: Toward Greater Understanding of Technologies to Support Crowdsourced Science - Nathan Prestopnik; Kevin Crowston (2011)

Describing public participation in scientific research - Wiggins, Andrea; Kevin Crowston (Syracuse University School of Information Studies, 2011)
We report the results of a descriptive survey of citizen science projects, a form of scientific collaboration engaging members of the public with professional researchers. This phenomenon has seen explosive growth in recent years and is garnering interest from a broadening variety of research domains. However, the lack of adequate description of this diverse population hinders useful research. To address this gap, we conducted a survey of citizen science projects. We present a description of the phenomenon to establish a basis for sampling and evaluation of research on citizen science, including details on project resources, participation, technologies, goals, and outcomes. We then reflect on several points of potential development, including technologies to support participation, potential for expanding engagement, and data policies. The diverse organizational and functional arrangements in citizen science projects suggest a variety of areas for future research.

Design for Citizen Science Workshop Report - Wiggins, Andrea (Syracuse University School of Information Studies, 2011)

Exploring Collective Intelligence Games With Design Science: A Citizen Science Design Case - Nathan Prestopnik; Kevin Crowston (2011)
Citizen science is a form of collective intelligence where members of the public are recruited to contribute to scientific investigations. Citizen science projects often use web-based systems to support collaborative scientific activities, but finding ways to attract participants and confirm the veracity of the data produced by non-scientists are key research questions. We describe a series of web-based tools and games currently under development to support taxonomic classification of organisms in photographs collected by citizen science projects. In the design science tradition, the systems are purpose-built to test hypotheses about participant motivation and techniques for ensuring data quality. Findings from preliminary evaluation and the design process itself are discussed.

From Conservation to Crowdsourcing: A Typology of Citizen Science - Wiggins, Andrea; Kevin Crowston (Proceedings of the Forty-fourth Hawai'i International Conference on System Sciences (HICSS-44), 2011)
Citizen science is a form of research collaboration involving members of the public in scientific research projects to address real-world problems. Often organized as a virtual collaboration, these projects are a type of open movement, with collective goals addressed through open participation in research tasks. Existing typologies of citizen science projects focus primarily on the structure of participation, paying little attention to the organizational and macrostructural properties that are important to designing and managing effective projects and technologies. By examining a variety of project characteristics, we identified five types—Action, Conservation, Investigation, Virtual, and Education—that differ in primary project goals and the importance of physical environment to participation.

Gaming for (citizen) science: Exploring motivation and data quality in the context of crowdsourced science through the design and evaluation of a social-computational system - Nathan Prestopnik; Kevin Crowston (“Computing for Citizen Science” workshop at the IEEE eScience Conference, 2011)
In this paper, an ongoing design research project is described. Citizen Sort, currently under development, is a web-based social-computational system designed to support a citizen science task, the taxonomic classification of various insect, animal, and plant species. In addition to supporting this natural science objective, the Citizen Sort platform will also support information science research goals on the nature of motivation for social-computation and citizen science. In particular, this research program addresses the use of games to motivate participation in social-computational citizen science, and explores the effects of system design on motivation and data quality. A design science approach, where IT artifacts are developed to solve problems and answer research questions is described. Research questions, progress on Citizen Sort planning and implementation, and key challenges are discussed.
Keywords: Citizen Science; data quality; Design; Design Science; Games; Gaming; Motivation; Participation; Social Computational Systems

Integrated Customer-Focused Knowledge Portals: Design Challenges and Empirical Approaches - Claudia Löbbecke; Kevin Crowston; Nicolas Friederici (Academy of Management Conference, OCIS Division, 2011)
Knowledge Portals (KPs) are highly integrative Knowledge Management Systems (KMS) that promise to synthesize widely dispersed knowledge and to interconnect individuals by functioning as a 'one-stop knowledge shop'. Yet, in practice, KPs face major challenges, which are for the most part due to the intricacies of knowledge exchange being subjected to multi-faceted individual and social factors. At the same time, growing anecdotal evidence from case studies indicates KPs' enormous potential. This paper makes an effort to more distinctly conceptualize KPs and emphasize a KP's role to unify networking and repository KMS features. The paper develops three major challenges to successful KP deployment, namely (1) knowledge integration, (2) sufficient participation, and (3) favorable organizational culture and validates these as applicable to KP through a review of 42 empirical papers. The paper concludes with suggestions towards a set of design principles for KP.

Lessons from volunteering and free/libre open source software development for the future of work - Kevin Crowston (IFIP Working Group 8.2 Working Conference: Researching The Future, 2011)

Mechanisms for Data Quality and Validation in Citizen Science - Wiggins, Andrea; Newman, Greg; Stevenson, Robert D.; Kevin Crowston ("Computing for Citizen Science" workshop at the IEEE eScience Conference, 2011)
Data quality is a primary concern for researchers employing public participation in scientific research, or “citizen science,” to accomplish data collection and analysis tasks. This mode of scientific collaboration relies on contributions from a large, often unknown population of volunteers with widely variable expertise. In this paper, we review the commonly employed mechanisms for ensuring data quality. We also discuss results of a survey of citizen science projects that reports on the use of some of these mechanisms, noting that it is most common for projects to employ multiple mechanisms to ensure data quality and appropriate levels of validation.
Keywords: Citizen Science; data quality; data validation

Participation in ICT-Enabled Meetings - Chudoba, Katherine M.; Mary Beth Watson-Manheim; Kevin Crowston; Lee, Chei Sian (Journal of Organizational and End User Computing, 2011)
Meetings are a common occurrence in contemporary organizations, and almost everyone shares an understanding of what a meeting is and what participation in a meeting looks like. Yet our exploratory study at Intel, an innovative global technology company, suggests that meetings are evolving beyond this familiar perspective as the pervasive use of information and communication technologies (ICTs) changes work practices associated with meetings. Drawing on data gathered from interviews prompted by entries in the employees' electronic calendar system, we examine the multiple ways in which meetings build and reflect work in the organization and derive propositions to guide future research. Specifically, we identified four aspects of meetings that reflect work in the 21st century: meetings are integral to work in team-centered organizations, tension between group and personal objectives, discontinuities, and ICT support for fragmented work environment.
Keywords: Computer-Mediated Communication; Information Technology

Technology adoption and use: Theory review for studying scientists’ continued use of cyber-infrastructure - Kim, Youngseek; Kevin Crowston (American Society for Information Science and Technology Annual Meeting, 2011)
In this paper, we seek to identify factors that might increase the likelihood of adoption and continued use of cyber-infrastructure by scientists. To do so, we review the main research on Information and Communications Technology (ICT) adoption and use by addressing research problems, theories and models used, findings, and limitations. We focus particularly on the individual user perspective. We categorize previous studies into two groups: Adoption research and post-adoption (continued use) research. In addition, we review studies specifically regarding cyber-infrastructure adoption and use by scientists and other special user groups. We identify the limitations of previous theories, models and research findings appearing in the literature related to our current interest in scientists’ adoption and continued use of cyber-infrastructure. We synthesize the previous theories and models used for ICT adoption and use, and then we develop a theoretical framework for studying scientists’ adoption and use of cyber-infrastructure. We also proposed a research design based on the research model developed. Implications for researchers and practitioners are provided.

Validity issues in the use of social network analysis with digital trace data - James Howison; Kevin Crowston; Wiggins, Andrea (Journal of the Association for Information Systems, 2011)
There is an exciting natural match between social network analysis methods and the growth of data sources produced by social interactions via information technologies, from online communities to corporate information systems. Information Systems researchers have not been slow to embrace this combination of method and data. Such systems increasingly provide "digital trace data" that provide new research opportunities. Yet digital trace data are substantively different from the survey and interview data for which network analysis measures and interpretations were originally developed. This paper examines ten validity issues associated with the combination of data digital trace data and social network analysis methods, with examples from the IS literature, to provide recommendations for improving the validity of research using this combination.

What characterizes documents that bridge boundaries compared to documents that do not? An exploratory study of documentation in FLOSS teams - Carsten Østerlund; Kevin Crowston (Hawai'i International Conference on System Science, 2011)

Work as coordination and coordination as work: A process perspective on FLOSS development projects - Kevin Crowston; Carsten Østerlund; James Howison; Bolici, Francesco (Third International Symposium on Process Organization Studies, 2011)

Analyzing Leadership Dynamics in Distributed Group Communication - Kevin Crowston; Wiggins, Andrea; James Howison (Proceedings of the Forty-third Hawaiʼi International Conference on System Sciences (HICSS-43), 2010)
We apply social network analysis (SNA) to examine the dynamics of leadership in distributed groups, specifically Free/Libre Open Source Software development projects, and its relation to group performance. Based on prior work on leadership in distributed groups, we identify leaders with those who make the highest level of contribution to the group and assess the degree of leadership by measuring centralization of communications. We compare the dynamics of leadership in two FLOSS projects, one more and one less effective. We find that in both projects, centralization was higher in developer-oriented communications venues than in user-oriented venues, suggesting higher degrees of leadership in developer venues. However, we do not find a consistent relation between centralization and effectiveness. We suggest that SNA can instead be useful for identifying interesting periods in the history of the project, e.g., periods where the leadership of the project is in transition.
Keywords: FLOSS; Leadership

A Capability Maturity Model for Scientific Data Management - Kevin Crowston; Jian Qin (American Society for Information Science and Technology (ASIST) Annual Conference, 2010)
In this paper, we propose a capability maturity model (CMM) for scientific data management (SDM) practices, with the goal of supporting assessment and improvement of these practices. The CMM describes key process areas and practices necessary for effective SDM. The CMM further characterizes organizations by the level of maturity of these processes, meaning the organizational capability to reliably perform the processes. We suggest that this framework will be useful to organizations in evaluating and planning improvements to their SDM practices.
Keywords: Data Management; eScience

Designing scholarly communications - Kevin Crowston (IFIP Working Group 8.2/8.6 Joint Working Conference, 2010)
This paper presents suggestions for the design of a scholarly communications system for the IFIP Working Group 8.2 (WG8.2) community. Learned societies such as IFIP have long been important in the system of scholarly communications. With the affordances of the Internet, WG8.2 can play a larger role in promoting scholarly communications to achieve multiple goals: dissemination and archiving of quality research, but also supporting the development of scholars and the research community. A particular goal of this paper is to take a design perspective to suggest new systems to fit the emerging system and to assess the role that WG8.2 might play in deploying them.
Keywords: eScience; Scholarly Communications

Developing a Conceptual Model of Virtual Organizations for Citizen Science - Wiggins, Andrea; Kevin Crowston (International Journal of Organizational Design and Engineering, 2010)
This paper develops an organization design-oriented conceptual model of scientific knowledge production through citizen science virtual organizations. Citizen science is a form of organization design for collaborative scientific research involving scientists and volunteers, for which Internet-based modes of participation enable massive virtual collaboration by thousands of members of the public. The conceptual model provides an example of a theory development process and discusses its application to an exploratory study. The paper contributes a multi-level process model for organizing investigation into the impact of design on this form of scientific knowledge production.

Distributed Scientific Collaboration: Research Opportunities in Citizen Science - Wiggins, Andrea; Kevin Crowston (The Changing Dynamics of Scientific Collaboration, CSCW 2010 workshop, 2010)
This paper introduces a conceptual framework for research on citizen science, a form of collaboration involving scientists and volunteers in scientific research. Designing CSCW systems to support this type of scientific collaboration requires understanding the effects of organizational and work design on the scientific outcomes of citizen science projects. Initial directions for future research are identified, with the goal of developing a foundation for research on and development of cyberinfrastructure and collaborative technologies for supporting citizen science.

The impact of national culture on knowledge sharing in global virtual collaboration - Kangning Wei; Kevin Crowston (Proceedings of International Conference on Information Systems (ICIS), 2010)
Keywords: Knowledge Sharing; National Culture; Virtuality

Internet Genres - Kevin Crowston (Encyclopedia of Library and Information Sciences, 2010)
Rhetoricians since Aristotle have attempted to classify communications or documents into categories or “genres” with similar form, topic or purpose. This article surveys research on genre as it relates to Internet documents. The article briefly presents the concept of genre in general, and then reviews the evolution and emergence of genres on the Internet. It concludes with an examination of the possible use of genre for improving information access on the Internet, with specific discussion of the issues in developing taxonomies of genre and automatically recognizing document genre.

Leadership in self-managing virtual teams - Kevin Crowston; Heckman, Robert; Misiolek, Nora (Syracuse University School of Information Studies, 2010)
In this paper, we present a theory of leadership in self-managing virtual teams. We are particularly interested in self-managing virtual teams because self-management seems to be a common phenomenon in teams that interact primarily through information technology (so-called virtual teams). Building on leadership theory and structuration theory, the theory describes leadership as a process that results in the reinforcement, creation and evolution of ongoing structures and distinguishes between two types of leadership. We identify first-order leadership as leadership that works within and reinforces existing structures to elicit and guide group contributions. We define second-order leadership as behavior that effects changes in the structure that guides group action. We argue that second-order leadership is enabled by first-order leadership, is therefore action embedded, and is grounded in processes that define the social identity of the team. We propose that effective self-managing virtual teams will exhibit a paradoxical combination of shared, distributed first-order leadership complemented by strong, concentrated, and centralized second-order leadership. We conclude by presenting a set of research questions and suggestions for future research.

Machine Learning and Rule-Based Automated Coding of Qualitative Data - Kevin Crowston; Xiaozhong Liu; Allen, Eileen E.; Heckman, Robert (American Society for Information Science and Technology (ASIST) Annual Conference, 2010)
Researchers often employ qualitative research approaches but large volumes of textual data pose considerable challenges to manual coding. In this research, we explore how to implement fully or semi-automatic coding on textual data (specifically, electronic messages) by leveraging Natural Language Processing (NLP). In particular, we compare the performance of human-developed NLP rules to those inferred by machine learning algorithms. The experimental results suggest that NLP with machine learning can be an effective way to assist researchers in coding qualitative data.
Keywords: FLOSS; NLP

Problems in the use-centered development of a taxonomy of web genres - Kevin Crowston; Kwaśnik, Barbara H.; Rubleske, Joe (Genres on the Web: Computational Models and Empirical Studies, 2010)

Reclassifying Success and Tragedy in FLOSS Projects - Wiggins, Andrea; Kevin Crowston (Proceedings of the International Conference on Open Source Systems (OSS), 2010)
This paper presents the results of a replication of English & Schweik’s 2007 paper classifying FLOSS projects according to their stage of growth and indicators of success. We recreated the analysis using a comparable data set from 2006, with one additional point in time. We also expanded upon the original results by applying different criteria for evaluating the rate of new software releases for sustainability of project activity. We discuss the points of convergence and divergence from the original work from these extensions of the classification, and their implications for studying FLOSS development using archival data. The paper contributes new analysis of operationalizing success in FLOSS projects, with discussion of implications of the findings.
The attached zip file (OSS2010AnalysisScripts.zip) contains scripts for replicating the research. The data files for this analysis can be retrieved from: http://flossdb.syr.edu/classifier_summary_data.sql.gz (10 MB) http://flossdb.syr.edu/classifier_results.sql.gz (6 MB)

SoCS: Socially intelligent computing to support citizen science - Kevin Crowston (Proposal submitted to the NSF SOCS program, 2010)

The under-appreciated role of stigmergic coordination in software development - Bolici, Francesco; James Howison; Kevin Crowston (2010)
Coordination in software development teams has been a topic of perennial interest in empirical software engineering research. The vast majority of this literature has drawn on a conceptual separation between work and coordination mechanisms, separate from the work itself, which enable groups to achieve coordination. Traditional recommendations and software methods focused on planning: using analysis to predict and manage dependencies. Empirical research has demonstrated the limits of this approach, showing that many important dependencies are emergent and pointing to the persistent importance of explicit discussion to managing these dependencies as they arise. Drawing on work in Computer-Supported Collaborative Work and building from an analogy to collaboration amongst insects (stigmergy), we argue that the work product itself plays an under-appreciated role in helping software developers manage dependencies as they arise. This short paper presents the conceptual argument with empirical illustrations and explains why this mechanism would have significant implications for Software Engineering coordination research. We discuss issues in marshaling clear positive evidence, arguing that these issues are responsible, in part, for the under-consideration of this mechanism in software engineering and outlining research strategies which may overcome these issues.
Keywords: Coordination; FLOSS; Stigmergy

Coordination without discussion? Socio-technical congruence and Stigmergy in Free and Open Source Software projects - Bolici, Francesco; James Howison; Kevin Crowston (2nd International Workshop on Socio-Technical Congruence, ICSE, 2009)
Keywords: Coordination; FLOSS

Designing Virtual Organizations for Citizen Science - Wiggins, Andrea; Kevin Crowston (IFIP Working Group 8.2 OASIS workshop 2009, 2009)

Group Maintenance Behaviours of Core and Peripheral Members of Free/Libre Open Source Software Teams - Scialdone, Michael J.; Heckman, Robert; Kevin Crowston (Proceedings of The Fifth International Conference on Open Source Systems (OSS), 2009)
Group Maintenance is pro-social, discretionary, and relation-building behavior that occurs between members of groups in order to maintain reciprocal trust and cooperation. This paper considers how Free/libre Open Source Software (FLOSS) teams demonstrate such behaviors within the context of e-mail, as this is the primary medium through which such teams communicate. We compare group maintenance behaviors between both core and peripheral members of these groups, as well as behaviors between a group that remains producing software today and one which has since dissolved. Our findings indicate that negative politeness tactics (those which show respect for the autonomy of others) may be the most instrumental group maintenance behaviors that contribute to a FLOSS group‘s ability to survive and continue software production.
Keywords: FLOSS; Group Maintenance

Heartbeat: Measuring Active User Base and Potential User Interest in FLOSS Projects - Wiggins, Andrea; James Howison; Kevin Crowston (Proceedings of The Fifth International Conference on Open Source Systems (OSS), 2009)
This paper presents a novel method and algorithm to measure the size of an open source project’s user base and the level of potential user interest that it generates. Previously unavailable download data at a daily resolution confirms hypothesized patterns related to release cycles. In short, regular users rapidly download the software after a new release giving a way to measure the active user base. In contrast, potential new users download the application independently of the release cycle, and the daily download figures tend to plateau at this rate when a release has not been made for some time. An algorithm for estimating these measures from download time series is demonstrated and the measures are examined over time in two open source projects.

ICIS 2008 Panel Report: Open Access Publishing to Nurture the Sprouts of Knowledge and the Future of Information Systems Research - Avital, Michel; Björk, Bo-Christer; Boland, Richard J; Kevin Crowston; Lyytinen, Kalle; Majchrzak, Ann (Communications of the Association for Information Systems, 2009)
The advent of the Internet and the subsequent adoption of Open Access schemata are changing the nature of the scholarly discourse. In response, we seek to stimulate a debate about the role and desired forms of Open Access publishing in the context of the Information System (IS) discipline. We explore the potential contribution of an Open Access perspective on publishing IS-related research and also discuss the roles of traditional journals and their prospects in the contexts of our observations. In particular, we focus on the new possibilities of publishing work-in-progress and its potential benefit for knowledge dissemination including the prospects of turning today's limited scholarly exchange into mass collaboration. We illustrate our vision with a working prototype of an Open Access disciplinary repository entitled Sprouts (http://sprouts.aisnet.org). Our aim is to inspire new thinking about the role of Open Access publishing, the potential of its application to disciplinary repositories of emergent work, its anticipated repercussions on our work practices, and its long-term implication for the impact of IS scholarship and the well-being of our community at large. We call for participation and further action in realizing a global repository of IS research in progress. This paper builds on a panel on Open Access that was presented at the 2008 International Conference on Information Systems (ICIS), held in Paris, France, in December 2008.
Keywords: Information System

Asynchronous Decision-Making in Distributed Teams (Poster) - Li, Qing; Heckman, Robert; Allen, Eileen E.; Kevin Crowston; Eseryel, U. Yeliz; James Howison; Wiggins, Andrea (Conference on Computer-Supported Cooperative Work, 2008)
Keywords: Decision-Making; FLOSS

Bug Fixing Practices within Free/Libre Open Source Software Development Teams - Kevin Crowston; Scozzi, Barbara (Journal of Database Management, 2008)
Free/libre open source software (FLOSS, e.g., Linux or Apache) is primarily developed by distributed teams. Developers contribute from around the world and coordinate their activity almost exclusively by means of email and bulletin boards, yet some how profit from the advantages and evade the challenges of distributed software development. In this article we investigate the structure and the coordination practices adopted by development teams during the bug-fixing process, which is considered one of main areas of FLOSS project success. In particular, based on a codification of the messages recorded in the bug tracking system of four projects, we identify the accomplished tasks, the adopted coordination mechanisms, and the role undertaken by both the FLOSS development team and the FLOSS community. We conclude with suggestions for further research.

The bug fixing process in proprietary and free/libre open source software: A coordination theory analysis - Kevin Crowston (Business Process Transformation, 2008)
To support business process transformation, we must first be able to represent business processes in a way that allows us to compare and contrast them or to design new ones. In this paper, I use coordination theory to analyze the bug fixing processes in the proprietary operating system development group of a large mini-computer manufacturer and for the Free/Libre Open Source Software Linux operating system kernel. Three approaches to identifying dependencies and coordination mechanisms are presented. Mechanisms analyzed include those for task assignment, resource sharing and managing dependencies between modules of source code. The proprietary development organization assigned problem reports to engineers based on the module that appeared to be in error, since engineers only worked on particular modules. Alternative task assignment mechanisms include assignment to engineers based on workload or voluntary assignment, as in Linux. In the proprietary process, modules of source code were not shared, but rather “owned” by one engineer, thus reducing the need for coordination. In Linux, where multiple developers can work on the same modules, alternative resource sharing mechanisms have been developed to manage source code. Finally, the proprietary developers managed dependencies between modules informally, relying on their personal knowledge of which other engineers used their code. The Linux process allows developers to change code in multiple modules, but emphasizes modularity to reduce the need to do so.
Keywords: Coordination; FLOSS

Competency rallying for technical innovation: The case of the Virtuelle Fabrik - Bernhard Katzy; Kevin Crowston (Technovation, 2008)
Technology improves at an ever-increasing rate, but the speed at which firms can adapt their strategies and competencies to develop technological innovations and exploit market opportunities remains limited. While networks provide an option to increase agility through collaborative access to relevant external competencies, we know little about systematically managing such networks. This paper identifies a collaborative network process that we label competency rallying. We describe the set of inter-organizational routines involved in competency rallying in a case study of the interactions among the partners of the Virtuelle Fabrik, a case of an organized regional network in the manufacturing industry in Switzerland. We describe competency rallying as the (1) identification and development of competencies, (2) identification and facing of market opportunities, (3) marshalling of competencies, and (4) short-term cooperative effort for technological innovation and commercialization. The paper contributes a model that furthers the understanding of the organizational character of networks based on specific, learned network capabilities and which allows prediction of the likelihood of success of practical collaboration projects in networked organizations.
Keywords: Virtual Organization

Decision Making Paths in Self-Organizing Technology-Mediated Distributed Teams - Li, Qing; Heckman, Robert; Kevin Crowston; James Howison; Allen, Eileen E.; Eseryel, U. Yeliz (Proceedings of the International Conference on Open Source Systems, 2008)
This paper investigates decision making in self-organizing technology-mediated distributed teams. This context provides an opportunity to examine how the use of technological support to span temporal and organizational discontinuities affects decision-making processes. 258 software-modification decision episodes were collected from the public emailing lists of six Free/Libre Open Source Software (FLOSS) projects over a span of five years. Six decision-making paths were identified as 1) short-cut decision-making path; 2) implicit-development decision-making path; 3) implicit-evaluation decision-making path; 4) normative decision-making path; 5) dynamic decision-making path; and 6) interrupted/delayed decision-making path. We suggest that the nature of the tasks and the affordances of the technology used reduce the need for explicit coordination, resulting in a broader range of possible decision processes than are observed in face-to-face groups.

Depicting What Really Matters: Using Episodes to Study Latent Phenomenon - Annabi, Hala; Kevin Crowston; Heckman, Robert (Proceedings of the International Conference on Information Systems (ICIS), 2008)
Research on processes and practices around information systems is often best conducted in naturalistic setting. To conduct valid and reliable research in such settings, researchers must find ways to reliably bound the phenomenon in which they are interested. In this paper we propose that researchers use episodes—events or processes occurring over a specified period of time—to isolate that which interests them from the vast set of related human behavior. The paper discusses the nature of episodes in the literature and suggests particular research settings in which episodes can be useful. The paper describes a three stage methodology to identify episodes for systematic data collection and analysis. The paper presents an example study using episodes to study group learning process in distributed groups.
Keywords: Learning; Process

eResearch workflows for studying free and open source software development - James Howison; Wiggins, Andrea; Kevin Crowston (Proceedings of the IFIP 2.13 Working Conference on Open Source Software (OSS), 2008)
This paper proposes a demonstration of eResearch workflow tools as a model for the research community studying free and open source software and its development. For purposes of background and justification, the paper first introduces eResearch as increasingly practiced in fields such as astrophysics and biology, then contrasts the practice of research on free and open source software. After outlining the suitable public data sources the paper introduces a class of tools known as scientific workflow frameworks, specifically focusing on one---Taverna---and introducing its features. To further explain the tool a complete workflow used for original research on FLOSS is described and the agenda for the live demonstration is outlined.
Slides from the presentation at the Oxford eResearch Conference of lessons learned in replicating research in eResearch workflows.
Keywords: eResearch; FLOSS; Workflow

eSocialScience for Free/Libre Open Source Software researchers - Kevin Crowston; James Howison; Wiggins, Andrea (Proceedings of the 4th International Conference on e-Social Science, 2008)
This abstract presents a case study of the potential application of eScience tools and practices for the social science research community studying Free/Libre Open Source Software (FLOSS) development practices. We first describe the practice of research on FLOSS to motivate the need for eScience. After outlining suitable public data sources, we describe our initial efforts to introduce eScience tools for FLOSS research, potential obstacles and how the use of such tools might affect the practice of research in this field.
Keywords: eScience; FLOSS

Group Maintenance in Technology-Supported Distributed Teams - Scialdone, Michael J.; Li, Na; James Howison; Heckman, Robert; Kevin Crowston (Proceedings of the Best Paper Proceedings, Academy of Management Annual Meeting, 2008)
Are geographically-distributed teams which exhibit high levels of group maintenance between members successful? We answer this through content analysis of emails from two Free/Libre Open Source Software (FLOSS) teams. Our results illustrate that the groups utilize low levels of organizational citizenship behaviors and high levels of positive politeness actions.
Keywords: FLOSS; Group Maintenance

The motivational arc of massive virtual collaboration - Kevin Crowston; Fagnot, Isabelle (Proceedings of the IFIP WG 9.5 Working Conference on Virtuality and Society: Massive Virtual Communities, 2008)
Massive virtual collaborations (MVC) involve large numbers of mostly unpaid contributors collectively creating new content. Wikipedia is the most dramatic example of MVC; smaller-scale examples include blogs and discussion groups and free/libre open source software (FLOSS) projects. In this paper, we propose a model of motivations for contribution to MVC that integrates various theoretical perspectives to extend prior work. Specifically, we distinguish three different levels of contribution to projects (initial, sustained and meta) and capture the dynamic and recursive effects of contributions on emergent individual and project states.
Please refer to the more recent version of the paper.

Opportunities for eScience research on Free/Libre Open Source Software - Kevin Crowston; James Howison; Wiggins, Andrea (Proceedings of the Oxford e-Research 08 Conference, 2008)

Replication of FLOSS Research as eResearch - Wiggins, Andrea; James Howison; Kevin Crowston (Proceedings of the Oxford e-Research Conference 08, 2008)
We are working to introduce the ideas of eResearch to a multi-disciplinary research domain: those researchers examining Free/Libre and Open Source Software (FLOSS) and its development (Howison, Wiggins, & Crowston, 2008). The first phase of this work focused on building a repository for data on FLOSS teams, FLOSSmole (Howison, Conklin, & Crowston, 2006), and collaborating with other nascent data repositories in the field. Recently we have begun a second phase, which is to introduce another established principle of eResearch, that of broader collaboration through shared workflows accessing these data repositories. To provide an example of the potential value of this principle, we are replicating seminal FLOSS papers using eResearch approaches. This paper describes research outcomes and lessons learned from translating published literature into eResearch workflows.

Shared mental models among open source software developers - Scozzi, Barbara; Kevin Crowston; Eseryel, U. Yeliz; Li, Qing (Proceedings of the 41st Hawai'i International Conference on System Sciences (HICSS), 2008)
Shared understandings are important for software development as they guide to effective individual contributions to, and coordination of, the software development process. In this paper, we present the results of a preliminary analysis on shared mental models within Free/Libre Open Source Software (FLOSS) development teams. Based on structuration theory and by adopting cognitive mapping and process analysis, we represented and com-pared the mental models of some developers of the Lucene Java project. Our analysis suggests that there is a high-level of sharing among core developers but the shar-ing is not complete, with some differences related to ten-ure in the project.

Social dynamics of FLOSS team communication across channels - Wiggins, Andrea; James Howison; Kevin Crowston (Proceedings of the IFIP 2.13 Working Conference on Open Source Software (OSS), 2008)
This paper extends prior investigation into the social dynamics of free and open source (FLOSS) teams by examining the methodological questions arising from research using social network analysis on open source projects. We evaluate the validity of data sampling by examining dynamics of communication centralization, which vary across multiple communication channels. We also introduce a method for intensity-based smoothing in dynamic social network analysis.
Keywords: FLOSS

Towards a data and workflow collaboratory for research on Free and Open Source Software and its development (Poster) - James Howison; Squire, Megan; Kevin Crowston (iConference, 2008)
Keywords: eScience; FLOSS; Workflow

Building a collaboratory for research on open source software development (Poster) - James Howison; Squire, Megan; Kevin Crowston (eSocial Science Conference, 2007)
Keywords: eScience; FLOSS

Building a Corpus of Genre-Tagged Web Pages for an Information-Access Experiment - Rubleske, Joe; Kevin Crowston; Kwaśnik, Barbara H.; Chun, You-Lee (Colloquium on Web Genres, Corpus Linguistics, 2007)

Competency rallying processes in virtual organizations - Bernhard Katzy; Kevin Crowston (Proceedings of the IFIP Working Group 8.2/9.5 Working Conference on Virtuality and Virtualization, 2007)
Firms face an environment changing at an increasingly rapid pace. Unfortunately, the speed at which organizations can adapt their strategies and competencies to exploit such opportunities remains limited. In this paper we weave together an external perspective on market-facing with an internal perspective on competency development and marshalling to describe the organizational activities necessary for firms to cooperate within a virtual organization. We argue that firms can address their individual limitations through a systematic process that we call “competence rallying,” with which they can access market opportunities and additional needed competencies. Specifically, we present a local process theory of how one network of firms reliably engineers and delivers manufacturing projects using an inter-organizational process that works to meet short-term market opportunities. Our theory is grounded in the experiences of the Virtuelle Fabrik project, an organized network for regional cooperation in the manufacturing industry around the Bodensee in Europe. The success of manufacturing projects in a virtual organization is predicated on specific organizational activities in four phases of the competence rallying process: 1) identification and development of competencies, 2) identification and facing of market opportunities, 3) marshalling of competencies, and 4) a short-term cooperative effort. The Virtuelle Fabrik project was started by the Institute for Technology Management, University of St. Gallen. Financial support was provided by The Swiss Commission for Scientific Research (KwF) and the Virtuelle Fabrik partner companies. The Virtuelle Fabrik project team included Günter Schuh, Bernhard Katzy, Kai Millarg, Thomas Zehnder, Stefan Eisen, and Åsa Göransson, as well as managers from the partner firms. The authors thank all of the participants in the Virtuelle Fabrik for their contributions and in particular, for the many discussions that led to the concepts discussed in this paper. The authors take sole responsibility for the work presented here. This paper has benefited greatly from discussion with Paul van Fenema, Steven Sawyer, Ping Zhang, Robert Heckman, Barbara Kwasnik, and Gisela von Dran.
Keywords: Virtual Organization

Distance Matters, Except When It Doesn't: Discontinuities in Virtual Work - Mary Beth Watson-Manheim; Chudoba, Katherine M.; Kevin Crowston (Academy of Management Conference, 2007)
Virtual work has become an increasingly common phenomenon in today's organizations. Substantial and continuing changes in organizational processes and IT infrastructure have increased the pace and intensity of working across traditionally impermeable boundaries, enabling diverse forms of collaboration. However, our understanding of the consequences and implications of virtual work still lags and research results have been contradictory. We suggest that some of these inconsistencies have been because the boundaries that characterize virtual work-time, space, culture, organization, and so forth-are objective demarcations that are not uniformly problematic. It is only when those working in virtual settings perceive a boundary to be a discontinuity that it hinders work processes. We develop a model of virtual work that differentiates between boundaries and discontinuities, which helps account for contradictory findings. By examining the process of virtual work in more detail, we can uncover issues that are the underlying cause of problems, rather than deal with the more obvious symptoms that can mask underlying problem. Our model has implications both for research and for those working in virtual environments.
Keywords: Discontinuity; Virtuality

Emergent decision-making practices in Free/Libre Open Source Software FLOSS development teams - Heckman, Robert; Kevin Crowston; Eseryel, U. Yeliz; James Howison; Allen, Eileen E.; Li, Qing (Proceedings of the IFIP 3rd International Conference on Open Source Software, 2007)
We seek to identify work practices that make Free/Libre Open Source Software (FLOSS) development teams effective. Particularly important to team effectiveness is decision making. In this paper, we report on an inductive qualitative analysis of 360 decision episodes of six FLOSS development teams. Our analysis revealed diversity in decision-making practices that seem to be related to differences in overall team characteristics and effectiveness.

Emergent leadership in self-organizing virtual teams (Poster) - Heckman, Robert; Kevin Crowston; Misiolek, Nora; Eseryel, U. Yeliz (Proceedings of the International Conference on Information Systems (ICIS), 2007)
Keywords: FLOSS; Leadership

Investigating the Dynamics of FLOSS Development Teams (Poster) - Li, Na; Li, Qing; Kangning Wei; Heckman, Robert; Eseryel, U. Yeliz; Liddy, Elizabeth D.; James Howison; Kevin Crowston; Allen, Eileen E.; Scialdone, Michael J.; Inoue, Keisuke; Harwell, Sarah; et al. (2007)
SD 2007 poster - Full Adobe PDF 2007 HSD PI's conference poster reporting on the grant project work to date in a full Adobe PDF file. HSD 2007 poster - Small PDF HSD 2007 conference grant progress reporting poster in a smaller PDF file.

A proposed data and analysis archive for research on Free and Open Source Software and its development (Poster) - James Howison; Squire, Megan; Kevin Crowston (2007)
Keywords: FLOSS

A proposed data and analysis archive for research on Free and Open Source Software and its development (Poster) - James Howison; Squire, Megan; Kevin Crowston (eSocial Science Conference, 2007)
Keywords: FLOSS

The role of face-to-face meetings in technology-supported self-organizing distributed teams - Kevin Crowston; James Howison; Masango, Chengetai; Eseryel, U. Yeliz (IEEE Transactions on Professional Communications, 2007)
We examine the role of face-to-face meetings in the context of technology-supported self-organizing distributed or virtual teams, specifically Free/Libre Open Source Software (FLOSS) development teams. Based on a qualitative inductive analysis of data from interviews and observations at FLOSS conferences, we identify a variety of settings in which developers meet face-to-face, activities performed in these settings and benefits obtained. Contrary to the conventional wisdom, FLOSS developers generally do not meet face-to-face until the project is well under way. An additional benefit of face-to-face meetings is time away from a regular job and speed of interaction for certain kinds of tasks.
Keywords: FLOSS

Self-organization of teams in free/libre open source software development - Kevin Crowston; Kangning Wei; Li, Qing; Eseryel, U. Yeliz; James Howison (Information and Software Technology Journal: Special issue on Understanding the Social Side of Software Engineering, Qualitative Software Engineering Research, 2007)
This paper provides empirical evidence about how free/libre open source software development teams self-organize their work, specifically, how tasks are assigned to project team members. Following a case study methodology, we examined developer interaction data from three active and successful FLOSS projects using qualitative research methods, specifically inductive content analysis, to identify the task-assignment mechanisms used by the participants. We found that ‘self-assignment’ was the most common mechanism across three FLOSS projects. This mechanism is consistent with expectations for distributed and largely volunteer teams. We conclude by discussing whether these emergent practices can be usefully transferred to mainstream practice and indicating directions for future research.

A structurational perspective on leadership in virtual teams - Heckman, Robert; Kevin Crowston; Misiolek, Nora (Proceedings of the IFIP Working Group 8.2/9.5 Working Conference on Virtuality and Virtualization, 2007)
Building on behavioural leadership theory and structuration theory, we present a two-order theory of leadership. It describes four classes of first-order leadership behaviours (task coordination, substantive task contribution, group maintenance and boundary spanning) and defines second-order leadership as behaviour that influences changes in the structure that guides group action. We argue that second-order leadership is enabled by first-order leadership and is therefore action embedded and grounded in processes that define the social identity of the group. We propose that effective virtual teams will exhibit a paradoxical combination of shared, distributed first-order leadership complemented by strong, concentrated, and centralized second-order leadership. We conclude by suggesting future research that might be conducted to test and further elaborate our theory.
Keywords: FLOSS; Leadership

Assessing the health of open source communities - Kevin Crowston; James Howison (IEEE Computer, 2006)
An invited column that discusses what a healthy FLOSS community looks like, and how one ought to go about assessing it (particularly in the context of software specification).

Audio of ICIS 2006 Presentation - Kevin Crowston; Eseryel, U. Yeliz; Li, Qing (2006)
Audio of Kevin, Yeliz and Qing presenting our paper at ICIS 2006. The audio is ok, but next time I'm definitely going to put the recorder nearer the speakers!
Keywords: FLOSS

Challenges in creating a taxonomy for genres of digital documents - Kwaśnik, Barbara H.; Chun, You-Lee; Kevin Crowston; D'Ignazio, J.; Rubleske, Joe (Proceedings of the ISKO Conference, 2006)

Coordination theory: A ten-year retrospective - Kevin Crowston; Rubleske, Joe; James Howison (Human-Computer Interaction in Management Information Systems, 2006)
Since the initial publication in 1994, Coordination Theory has been referenced in nearly 300 journal articles, book chapters, conference papers and theses. Coordination Theory provides an approach to a core problem in HCI: analyzing group work to suggest alternative approaches involving computer support. Coordination Theory suggests identifying the dependencies between the tasks the different group members are carrying out and the coordination mechanisms the group use to coordinate their work and then considering alternative mechanisms. This chapter will analyze the contribution of this body of research to determine how Coordination Theory has been used for user task analysis and modelling for HCI. Issues that will be addressed include: 1)how the theory has been applied; 2) factors that led to the success of the theory; and 3)identification of areas needing further research.
Keywords: Coordination

Core and periphery in Free/Libre and Open Source software team communications - Kevin Crowston; Kangning Wei; Li, Qing; James Howison (Proceedings of the 39th Hawai'i International Conference on System Sciences (HICSS-39), 2006)
The concept of the core group of developers is important and often discussed in empirical studies of FLOSS projects. This paper examines the question, “how does one empirically distinguish the core?” Being able to identify the core members of a FLOSS development project is important because many of the processes necessary for successful projects likely involve core members differently than peripheral members, so analyses that mix the two groups will likely yield invalid results. We compare 3 analysis approaches to identify the core: the named list of developers, a Bradford’s law analysis that takes as the core the most frequent contributors and a social network analysis of the interaction pattern that identifies the core in a core-and-periphery structure. We apply these measures to the interactions around bug fixing for 116 SourceForge projects. The 3 techniques identify different individuals as core members; examination of which individuals are identified leads to suggestions for refining the measures. All 3 measures though suggest that the core of FLOSS projects is a small fraction of the total number of contributors.
Keywords: FLOSS

Customer satisfaction with electronic service encounters - Massad, Nelson; Heckman, Robert; Kevin Crowston (International Journal of Electronic Commerce, 2006)
Customer relationship management is an integral component of business strategy for on-line service providers. This paper investigates the aspects of on-line transactions in electronic retailing that are most likely to satisfy or dissatisfy customers, thereby increasing or decreasing the likelihood of building and maintaining relationships with them. For this study, 513 respondents reported behaviors, perceptions, beliefs, events, features, characteristics, attributes, and situations that expressed their satisfaction or dissatisfaction with electronic service encounters. Content analysis of these encounters yielded three meta-categories, six categories, and 33 subcategories of customer satisfaction or dissatisfaction with on-line service providers. The findings suggested that three major categories are robust even in the electronic context of the Internet. The antecedents identified were relevant both to product-related services (e.g., books, apparel) and to pure services (e.g., on-line banking, on-line stock trading). The study found that the characteristics and behaviors of customer-contact employees play an important role in on-line service encounters. It also revealed a percentage decrease in satisfactory incidents, a percentage increase in unsatisfactory incidents, and a percentage increase in unsatisfactory incidents involving employee characteristics and behaviors as service encounters move from a bricks-and-mortar environment to an electronic context. This suggests that customer-contact employees may not be well equipped to deal with on-line customers.

Emergent decision-making practices in technology-supported self-organizing distributed teams - Heckman, Robert; Kevin Crowston; Li, Qing; Allen, Eileen E.; Eseryel, U. Yeliz; James Howison; Kangning Wei (Proceedings of the International Conference on Information Systems (ICIS), 2006)
We seek to identify work practices that make technology-supported self-organizing distributed (or virtual) teams (TSSODT for short) effective in producing outputs satisfactory to their sponsors, meeting the needs of their members and continuing to function. A particularly important practice for team effectiveness is decision making: are the right decisions made at the right time to get the work done in a way that satisfies team sponsors, keeps contributors happy and engaged, and enables continued team success? In this research-in-progress paper, we report on an inductive qualitative analysis of 120 decision episodes taken by 2 Free/libre Open Source Software development teams. Our analysis revealed differences in decision-making practices that seem to be related to differences in overall team effectiveness.
Keywords: Decision-Making; FLOSS

FLOSSmole: A collaborative repository for FLOSS research data and analyses - James Howison; Conklin, Megan S.; Kevin Crowston (International Journal of Information Technology and Web Engineering, 2006)
This paper introduces and expands on previous work on a collaborative project, called FLOSSmole (formerly OSSmole), designed to gather, share and store comparable data and analyses of free and open source software development for academic research. The project draws on the ongoing collection and analysis efforts of many research groups, reducing duplication, and promoting compatibility both across sources of FLOSS data and across research groups and analyses. The paper outlines current difficulties with the current typical quantitative FLOSS research process and uses these to develop requirements and presents the design of the system.
Keywords: FLOSSmole

From Individual Contribution to Group learning: the Early Years of Apache Web Server - Annabi, Hala; Kevin Crowston; Heckman, Robert (Proceedings of the IFIP 2nd International Conference on Open Source Software, 2006)
Open Source Software (OSS) groups experience many benefits and challenges with respect to the core group’s effectiveness. In order to capitalize on the benefits and minimize the challenges, OSS groups must learn not only on the individual level, but also on the group level. OSS groups learn by integrating individual contributions into the group’s product and processes. This paper reports on the characteristics of the learning process in OSS groups. The study utilized an embedded single case study design that observed and analyzed group learning processes in the Apache Web server OSS project. The study used learning opportunity episodes (LOE) as the embedded unit of analysis and developed and utilized three content analytic schemes to describe the characteristics of the learning process and the factors affecting this process.
Keywords: FLOSS; Learning

Hierarchy and centralization in Free and Open Source Software team communications - Kevin Crowston; James Howison (Knowledge, Technology & Policy, 2006)
Free/Libre Open Source Software (FLOSS) development teams provide an interesting and convenient setting for studying distributed work. We begin by answering perhaps the most basic question: what is the social structure of these teams? Based on a social network analysis of interactions represented in 62,110 bug reports from 122 large and active projects, we find that some OSS teams are highly centralized, but contrary to expectation, others are not. Projects are mostly quite hierarchical on four measures of hierarchy, consistent with past research but contrary to the popular image of these projects. Furthermore, we find that the level of centralization is negatively correlated with project size, suggesting that larger projects become more modular. The paper makes a further methodological contribution by identifying appropriate analysis approaches for interaction data. We conclude by sketching directions for future research.
Keywords: FLOSS

Information systems success in Free and Open Source Software development: Theory and measures - Kevin Crowston; James Howison; Annabi, Hala (Software Process–Improvement and Practice, 2006)
Information systems success is one of the most widely used dependent variables in information systems (IS) research, but research on Free/Libre and Open Source software (FLOSS) often fails to appropriately conceptualize this important concept. In this paper, we reconsider what success means within a FLOSS context. We first review existing models of IS success and success variables used in FLOSS research and assess them for their usefulness, practicality and fit to the FLOSS context. Then, drawing on a theoretical model of group effectiveness in the FLOSS development process, as well as an online discussion group with developers, we present additional concepts that are central to an appropriate understanding of success for FLOSS. In order to examine the practicality and validity of this conceptual scheme, the second half of our paper presents an empirical study that demonstrates its operationalization of the chosen measures and assesses their internal validity. We use data from SourceForge to measure the project’s effectiveness in team building, the speed of the project at responding to bug reports and the project’s popularity. We conclude by discussing the implications of this study for our proposed extension of IS success in the context of FLOSS development and highlight future directions for research.
Keywords: FLOSS; Success

Investigating the Dynamics of FLOSS Development Teams (Poster) - Li, Qing; Kangning Wei; Heckman, Robert; Eseryel, U. Yeliz; Liddy, Elizabeth D.; James Howison; Kevin Crowston; Allen, Eileen E.; Inoue, Keisuke; Harwell, Sarah; Rowe, Steven; McCracken, Nancy (2006)
Poster describing the current state of the project for the HSD Principal Investigators' conference, 14-15 September 2006, Washington DC.

Language and power in self-organizing distributed teams - Li, Qing; Kevin Crowston; Heckman, Robert; James Howison (OCIS Division, Academy of Management Conference, 2006)
In this paper, a comparative case study is conducted to explore the way power is expressed and exercised through language use in distributed or virtual teams. Our research questions are “how is power expressed in online interactions in self-organizing distributed teams, in a context without formal authority or hierarchy?” and “What effects do expressions of power have on team outcomes?” To fully understand the role of power in self-organizing teams, we apply an input-process-output model on two open source projects-one successful and the other less successful. Two set of codes (source of power and power mechanism) are drawn from the data, and different power patterns interestingly show up between them. The findings lead us to speculate that strong, centralized leadership, the assertive exercise of power, and direct language may contribute to effectiveness in FLOSS teams. And the relevant conclusions and suggestions are provided for further research.
Keywords: FLOSS

The role of mental models in FLOSS development work practices - Kevin Crowston; Scozzi, Barbara (Proceedings of the IFIP 2nd International Conference on Open Source Software, 2006)
Shared understandings are important for software development as they guide to effective individual contributions to, and coordination of, the software development process. In this paper, we present the theoretical background and research design for a proposed study on shared mental models within Free/Libre Open Source Software (FLOSS) development teams. In particular, we plan to perform case studies on several projects and to use cognitive maps analysis to represent and compare the mental models of the involved members so as to gauge the degree of common knowledge and the development of a collective mind as well as to better understand the reasons that underlie team members actions and the way common mental models, if any, arise.
Keywords: FLOSS; Mental Model

Social dynamics of free and open source team communications - James Howison; Inoue, Keisuke; Kevin Crowston (Proceedings of the IFIP Second International Conference on Open Source Systems, 2006)
This paper furthers inquiry into the social structure of free and open source software (FLOSS) teams by undertaking social network analysis across time. Contrary to expectations, we confirmed earlier findings of a wide distribution of centralizations even when examining the networks over time. The paper also provides empirical evidence that while change at the center of FLOSS projects is relatively uncommon, participation across the project communities is highly skewed, with many participants appearing for only one period. Surprisingly, large project teams are not more likely to undergo change at their centers.
Keywords: FLOSS

Can Genre Metadata Improve Information Retrieval? - Rubleske, Joe; Kevin Crowston; Kwaśnik, Barbara H. (Connections: The 10th Annual Great Lakes Information Science Conference, 2005)

Collaboration Using OSSmole: A repository of FLOSS data and analyses - Conklin, Megan S.; James Howison; Kevin Crowston (Symposium on Mining Software Repositories, 2005)
This paper introduces a collaborative project OSSmole designed to collect, share, and store comparable data and analyses of free, libre and open source software (FLOSS) development for research purposes. The project is designed to be a clearinghouse for data from the ongoing collection and analysis efforts of many disparate research groups. A collaborative data repository will reduce duplication and promote compatibility both across sources of FLOSS data and across research groups and analyses. The primary objective of OSSmole is to mine FLOSS source code repositories and provide the resulting data and summary analyses as open source products. However, the OSSmole data model additionally supports donated raw and summary data from a variety of open source researchers and other software repositories. The paper first outlines current difficulties with the typical quantitative FLOSS research process and uses these to develop requirements for such a collaborative data repository. Finally, the design of the OSSmole system is presented, as well as examples of current research and analyses using OSSmole.

Coordination of Free/Libre Open Source Software development - Kevin Crowston; Kangning Wei; Li, Qing; Eseryel, U. Yeliz; James Howison (Proceedings of the International Conference on Information Systems (ICIS), 2005)
The apparent success of free/libre open source software (FLOSS) development projects such as Linux, Apache, and many others has raised the question, what lessons from FLOSS development can be transferred to mainstream software development? In this paper, we use coordination theory to analyze coordination mechanisms in FLOSS development and compare our analysis with existing literature on coordination in proprietary software development. We examined developer interaction data from three active and successful FLOSS projects and used content analysis to identify the coordination mechanisms used by the participants. We found that there were similarities between the FLOSS groups and the reported practices of the proprietary project in the coordination mechanisms used to manage task-task dependencies. However, we found clear differences in the coordination mechanisms used to manage task-actor dependencies. While published descriptions of proprietary software development involved an elaborate system to locate the developer who owned the relevant piece of code, we found that “self-assignment” was the most common mechanism across three FLOSS projects. This coordination mechanism is consistent with expectations for distributed and largely volunteer teams. We conclude by discussing whether these emergent practices can be usefully transferred to mainstream practice and indicating directions for future research.
Keywords: Coordination; FLOSS

Effective work practices for FLOSS development: A model and propositions - Kevin Crowston; Annabi, Hala; James Howison; Masango, Chengetai (Proceedings of the 38th Hawai'i International Conference on System Sciences (HICSS), 2005)
We review the literature on Free/Libre Open Source Software (FLOSS) development and on software development, distributed work and teams more generally to develop a theoretical model to explain the performance of FLOSS teams. The proposed model is based on Hackman’s [1] model of effectiveness of work teams, with coordination theory [2] and collective mind [3] to extend Hackman’s model by elaborating team practices relevant to effectiveness in software development. We propose a set of propositions to guide further research.
"Effective work practices for Software Engineering: Free/Libre Open Source Software Development". Presentation at the Workshop on Interdisciplinary Software Engineering 2004, SIGSOFT 2004/FSE-12 Conference, Newport Beach, CA, 5 November. Powerpoint file.

An exploratory study of factors related to effectiveness of Free/Libre Open Source Software teams - Kevin Crowston; Eseryel, U. Yeliz (2005)
Talk at the Padua Open Source Software Symposium, 14-15 April 2005.
Keywords: FLOSS

Face-to-face interactions in self-organizing distributed teams - Kevin Crowston; James Howison; Masango, Chengetai; Eseryel, U. Yeliz (Academy of Management Conference, 2005)
We explore the role of face-to-face meetings in the life of distributed teams using data from Free/Libre Open Source Software (FLOSS) development teams. Such distributed teams are part of many organizations’ new vision of management in the 21st century. Practitioner research has suggested the need for face-to-face meetings when a team is formed, but few studies have considered the role of face-to-face meetings during a team’s life. Based on a qualitative inductive analysis of data from interviews and observations at FLOSS conferences, we identify a variety of settings in which FLOSS developers meet face-to-face, activities performed in these settings and benefits obtained. Contrary to prior research, we find that FLOSS developers generally do not meet until the project is well under way. We also find that an additional benefit of face-to-face meetings is time away from a regular job. We conclude by noting limitations in our data collection due to a focus on core developers in large projects and with directions for further research.
Keywords: FLOSS

Future research on FLOSS development - Kevin Crowston (First Monday, 2005)

Introduction to the special issue: Genres of digital documents - Kwaśnik, Barbara H.; Kevin Crowston (Information, Technology & People, 2005)
Purpose – To introduce the special issue on “Genres of digital documents.” While there are many definitions of genre, most include consideration of the intended communicative purpose, form and sometimes expected content of a document. Most also include the notion of social acceptance, that a document is of a particular genre to the extent that it is recognized as such within a given discourse community. Design/methodology/approach – The article reviews the notion of document genre and its applicability to studies of digital documents and introduces the four articles in the special issue. Findings – Genre can be studied based on intrinsic genre attributes or on the extrinsic function that genre fulfills in human activities. Studies on intrinsic attributes include classifications of genres as clusters of attributes, though these classifications can be problematic because documents can be used in flexible ways. Also, new information technologies have enabled the appearance of novel genres. Studies on extrinsic function include ways to use genre for education or information accesses, as well as the use of genre as a lens for understanding communications in organizations. The four articles in the special issue illustrate these approaches. Originality/value – The paper provides a framework that organizes the range of research about genres of digital documents that should be helpful to those reading this research or planning their own studies.

Meet Me in Cyberspace: Meetings in the Distributed Work Environment - Chudoba, Katherine M.; Mary Beth Watson-Manheim; Lee, Chei Sian; Kevin Crowston (Academy of Management Conference, OCIS Division, 2005)
Meetings have long been a critical activity in contemporary work life. At least since Mintzberg's classic study of managerial behavior, researchers have documented and practitioners have bemoaned the amount of time spent in meetings. Despite these problems, meetings are becoming even more common in organizations. Teams are now found throughout the organization, from the manufacturing floor to senior management. Organizational work increasingly occurs in teams, with participation in meetings becoming a core work activity. How are these meetings conducted? What are the implications of the increased use of ICT in these meetings? As more and more members of organizational teams are distributed and must make extensive use of ICT in order to work together, these questions become even more important. Using the device of genre system, an interrelated set of socially constructed communicative actions, we examine meetings from the perspective of employees at a company in the technology industry. We began with the questions: Are meetings in the 21st century different from Mintzberg's conceptualization? If so, how? Drawing on data gathered from interviews that used entries in the employees' electronic calendar system, we found that employees are attending a large number of meetings (20% reported more than 25 meetings in a week) and spending significant time in meetings (27% reported more than 30 hours in a week in meetings). The majority of meetings included non-collocated participants and extensive use of ICT. We explore the implications of these and other findings for collaboration and ICT support.
Keywords: Computer-Mediated Communication; Virtuality

Methods for modeling and supporting innovation processes in SMEs - Scozzi, Barbara; Garavelli, C.; Kevin Crowston (European Journal of Innovation Management, 2005)
Purpose – Sets out to investigate business modeling techniques (BMTs) which can be used to support and improve innovation processes within small and medium-sized enterprises (SMEs). Design/methodology/approach – Based on a literature review, different analysis perspectives on innovation processes are identified and discussed, and some firm needs and problems are pointed out. The importance of BMTs to firms is further tested by an empirical study whose initial results are reported. Finally, by matching problems and techniques characterized by the same ontology, the BMTs most suitable to address SME needs are identified and their role within the innovation process discussed. Findings – The main result of the paper is the identification of the problems facing SMEs in innovation processes and the possible support offered by BMTs. Though methods and models alone do not assure the success in the innovation development process (IDP), they are enabling factors and can support the creation of strategies, reasoning, insights and communication. Originality/value – The adoption of such BMTs, facilitating the codification of the characteristics of the IDP, might be particularly useful in those environments where, due to the lack of specialized resources, it is difficult to structure all of the information related to the innovation process and to exploit the related benefits and opportunities.
Keywords: Coordination; Organizational Modelling

Redefining access: Uses and roles of information and communication technologies in the US residential real estate industry from 1995 to 2005 - Sawyer, Steve; Rolf Wigand; Kevin Crowston (Journal of Information Technology, 2005)
We discuss three industry-level changes in the US residential real estate industry due, in part, to the take up and uses of information and communication technologies (ICT): (1) changes in the processes of transacting residential real estate, (2) changing roles for information, and (3) changing nature of intermediation, with the real estate transaction as more complex than the seller–agent–buyer simplification would suggest. We speculate that these changes are currently indeterminate due to ongoing confusion among the impacts of first and second-level effects, the roles of ICT in redefining access to data, and the importance of localized, social structures of real estate markets. To develop these findings, we take an institutional perspective and draw on multiple data collection methods. This provides us a means to highlight the value of an institutional perspective for studying industrial-level change.
Keywords: Computer-Mediated Communication; Real Estate

SE, IS & (FL)OSS @ EASE - Kevin Crowston (2005)
A keynote address at the Conference on Empirical Assessment of Software Engineering at Keele University, 11-13 April 2005.
Keywords: FLOSS

The social structure of Free and Open Source Software development - Kevin Crowston; James Howison (First Monday, 2005)
Metaphors, such as the Cathedral and Bazaar, used to describe the organization of FLOSS projects typically place them in sharp contrast to proprietary development by emphasizing FLOSS’s distinctive social and communications structures. But what do we really know about the communication patterns of FLOSS projects? How generalizable are the projects that have been studied? Is there consistency across FLOSS projects? Questioning the assumption of distinctiveness is important because practitioner–advocates from within the FLOSS community rely on features of social structure to describe and account for some of the advantages of FLOSS production. To address this question, we examined 120 project teams from SourceForge, representing a wide range of FLOSS project types, for their communications centralization as revealed in the interactions in the bug tracking system. We found that FLOSS development teams vary widely in their communications centralization, from projects completely centered on one developer to projects that are highly decentralized and exhibit a distributed pattern of conversation between developers and active users. We suggest, therefore, that it is wrong to assume that FLOSS projects are distinguished by a particular social structure merely because they are FLOSS. Our findings suggest that FLOSS projects might have to work hard to achieve the expected development advantages which have been assumed to flow from "going open." In addition, the variation in communications structure across projects means that communications centralization is useful for comparisons between FLOSS teams. We found that larger FLOSS teams tend to have more decentralized communication patterns, a finding that suggests interesting avenues for further research examining, for example, the relationship between communications structure and code modularity.
First Monday, Special Issue #2: Open Source — 3 October 2005 The social structure of free and open source software development (originally published in Volume 10, Number 2, February 2005)
Keywords: FLOSS

A structurational perspective on leadership in Free/Libre Open Source Software teams - Kevin Crowston; Heckman, Robert; Annabi, Hala; Masango, Chengetai (Proceedings of the 1st Conference on Open Source Systems (OSS), 2005)
In this conceptual paper, we present a structuration-based theory of leadership behaviours in self-organizing virtual distributed teams such as Free/Libre Open Source Software development teams. Such teams are often composed of members of relatively equal status or who are so disparate in background that formal organizational status seems irrelevant, reducing the usual leadership cues provided by organizational status and title. Building on a functional view of leadership and structuration theory, we suggest that leaders are individuals who develop team structures that then guide the actions of team members. Specifically, we examine structures of signification in the form of shared mental models, structures of domination in the form of role structures and structures of legitimation in form of rules and norms. The main contribution of our paper is the integration of various social theories to describe emergent leadership behaviours in distributed teams. We develop a set of propositions and illustrate with examples taken from Free/Libre Open Source Software development teams. We conclude by suggesting future research that might be conducted to test and further elaborate our theory.
Keywords: FLOSS; Leadership

Coordination practices for bug fixing within FLOSS development teams - Kevin Crowston; Scozzi, Barbara (1st International Workshop on Computer Supported Activity Coordination, 6th International Conference on Enterprise Information Systems, 2004)
Free/Libre Open Source Software (FLOSS) is primarily developed by distributed teams. Developers contribute from around the world and coordinate their activity almost exclusively by means of email and bulletin boards. FLOSS development teams some how profit from the advantages and evade the challenges of distributed software development. Despite the relevance of the FLOSS both for research and practice, few studies have investigated the work practices adopted by these development teams. In this paper we investigate the structure and the coordination practices adopted by development teams during the bug-fixing process, which is considered one of main areas of FLOSS project success. In particular, based on a codification of the messages recorded in the bug tracking system of four projects, we identify the accomplished tasks, the adopted coordination mechanisms, and the role undertaken by both the FLOSS development team and the FLOSS community. We conclude with suggestions for further research.

Effective work practices for software engineering - Kevin Crowston; Annabi, Hala; James Howison; Masango, Chengetai (Proceedings of the 2004 ACM workshop on Interdisciplinary software engineering research (WISER '04), 2004)
We review the literature on Free/Libre Open Source Software (FLOSS) development and on software development, distributed work and teams more generally to develop a theoretical model to explain the performance of FLOSS teams. The proposed model is based on Hackman’s [34] model of effectiveness of work teams, with coordination theory [52] and collective mind [79] to extend Hackman’s model by elaborating team practices relevant to effectiveness in software development. We propose a set of propositions to guide further research.
Keywords: FLOSS

A framework for creating a facetted classification for genres: Addressing issues of multidimensionality - Kwaśnik, Barbara H.; Kevin Crowston (Proceedings of the 37th Hawai'i International Conference on System Sciences (HICSS), 2004)
People recognize and use document genres as a way of identifying useful information and of participating in mutually understood communicative acts. Crowston and Kwasnik [1] discuss the possibility of improving information access in large digital collections through the identification and use of document genre metadata. They draw on the definition of genre proposed by Orlikowski and Yates [3], who describe genre as “a distinctive type of communicative action, characterized by a socially recognized communicative purpose and common aspects of form” (p. 543). Scholars in fields such as rhetoric and library science have attempted to describe and systematize the notion of genre, and have offered many different definitions of genre. We like Orlikowski and Yates’s definition because it takes into account all three aspects of genre that we recognize as fundamental: content, form, and purpose. A document’s genre is a subtle and complex concept in which the content and form of a document are fused with its purpose or function. As such, a document’s genre cannot be separated from the context in which it is used; the same document may be construed as being of a different genre depending on how it is invoked in a given situation. Starting from the document, a letter may be a personal communication, or a piece of evidence in a court of law, or an agreement, or even a work of art. Starting from the situation, we note that differences in an information situation are often reflected in the kind of document that is considered helpful (e.g., a problem set vs. a lesson plan vs. a tutorial about mathematics, for instance). Thus, we see genre as a multidimensional phenomenon, which takes into account not only the attributes of the document itself, but also of its role in human endeavor. In this paper, we discuss some considerations in developing a facetted classification for genres to address the problem of multi-dimensionality.

Information Systems in Organizations and Society: Speculating on the Next 25 Years of Research - Sawyer, Steve; Kevin Crowston (Proceedings of the IFIP WG 8.2 Conference on “Relevant Theory and Informed Practice: Looking Forward from a 20-year Perspective on IS Research, 2004)
The community of scholars focused on information systems in organizations and society (the IFIP 8.2 community) has grown in number, voice, and influence over the last 25 years. What will this community contribute during the next 25 years? We speculate on two possible areas: more articulate conceptualizations of information systems and more detailed socio-technical theories of their effects. For both of these possibilities, we project forward from the historical trajectory of the IFIP 8.2 community’s involvement. Like all speculative scholarship, our argumentation is more about imagining possible directions than arguing the superiority of one particular view relative to all others. This considered speculation is directed at both stirring the community’s collective mind and advancing the value of this community’s work to interested others.
Keywords: Information System; Organization

Information technology and the transformation of industries: Three research perspectives - Kevin Crowston; Myers, Michael D. (Journal of Strategic Information Systems, 2004)
It is often claimed that information technology has the potential to transform entire industries. However, we find that very little IS research has been conducted at the industry level. Moreover, the small amount of research that has been conducted on IT and industries has been based largely on just one perspective of industries. Given the scale and potential impact of the changes that are happening at an industry level, we believe a concerted effort is needed to study this phenomenon. We propose a research agenda for studying IT and industries. We suggest three research perspectives for studying IT and industries: an economic perspective, an institutional perspective, and a socio-cultural perspective. Just as IS research that addresses these aspects at the organizational level has grown in recent years and contributed to our understanding of IS, so in this paper we argue that a similar broadening, as well as more studies, are needed at the industry level of analysis. Weprovide an example from the real estate industry to illustrate the usefulness of the three research perspectives.
Keywords: Information Technology; Real Estate

The paradox of discontinuities and continuities: Toward a more comprehensive view of virtuality - Mary Beth Watson-Manheim; Chudoba, Katherine M.; Kevin Crowston (Academy of Management Conference, 2004)
Understanding the fundamental dynamics in virtual work environments is a challenge for organizational researchers. We propose that virtuality is, paradoxically, generally composed of factors that simultaneously simplify and complicate the work environment. We use the metaphorical construction of continuities and discontinuities to explore this phenomenon, and show that considering boundaries as creating discontinuities in work environments enhances our understanding in two ways. First, the language of discontinuities allows us to employ the device of paradox to explore the underlying dynamics of virtuality. This device makes it easier to examine the more complex reality of virtual work. Second, consideration of discontinuities draws attention to possible problems encountered in virtual work environments and ways that individuals and teams may compensate for the tension and differences implicit in discontinuities; in other words, paying attention to the seemingly logical antithesis of discontinuities, or continuities. By taking a process perspective, one can focus on the behavioral component of work, which in turn, has a subjective component. An example from a field study of a global virtual team is examined to illustrate the usefulness of the discontinuities/continuities framework.
Keywords: Discontinuity; Virtuality

The perils and pitfalls of mining SourceForge - James Howison; Kevin Crowston (Workshop on Mining Software Repositories, 26th International Conference on Software Engineering, 2004)
SourceForge provides abundant accessible data from Open Source Software development projects, making it an attractive data source for software engineering research. However it is not without theoretical peril and practical pitfalls. In this paper, we outline practical lessons gained from our spidering, parsing and analysis of SourceForge data. SourceForge can be practically difficult: projects are defunct, data from earlier systems has been dumped in and crucial data is hosted outside SourceForge, dirtying the retrieved data. These practical issues play directly into analysis: decisions made in screening projects can reduce the range of variables, skewing data and biasing correlations. SourceForge is theoretically perilous: because it provides easily accessible data items for each project, tempting researchers to fit their theories to these limited data. Worse, few are plausible dependent variables. Studies are thus likely to test the same hypotheses even if they start from different theoretical bases. To avoid these problems, analyses of SourceForge projects should go beyond project level variables and carefully consider which variables are used for screening projects and which for testing hypotheses.

A structurational model of leadership in virtual distributed groups - Kevin Crowston; Annabi, Hala; Heckman, Robert (2004)
"A structurational model of the dynamics of Free/Libre Open Source Software development teams". Presentation at the IFIP WG 8.2 OASIS Workshop 2004, Washington, DC, 12 December 2004. (Powerpoint file).
Keywords: FLOSS

Towards a portfolio of FLOSS project success measures - Kevin Crowston; Annabi, Hala; James Howison; Masango, Chengetai (Workshop on Open Source Software Engineering, 26th International Conference on Software Engineering, 2004)
Project success is one of the most widely used dependent variables in information systems research. However, conventional measures of project success are difficult to apply to Free/Libre Open Source Software projects. In this paper, we present an analysis of four measures of success applied to SourceForge projects: number of members of the extended development community, project activity, bug fixing time and number of downloads. We argue that these four measures provide different insights into the collaboration and control mechanisms of the projects.

Will real estate agents survive? The transformation of the real estate industry by information technology - Myers, Michael D.; Kevin Crowston (University of Auckland Business Review, 2004)
Keywords: Real Estate

Can document-genre metadata improve information access to large digital collections? - Kevin Crowston; Kwaśnik, Barbara H. (Library Trends, 2003)
We discuss the issues of resolving the information-retrieval problem in large digital collections through the identification and use of document genres. Explicit identification of genre seems particularly important for such collections because any search usually retrieves documents with a diversity of genres that are undifferentiated by obvious clues as to their identity. Also, because most genres are characterized by both form and purpose, identifying the genre of a document provides information as to the document’s purpose and its fit to the user’s situation, which can be otherwise difficult to assess. We begin by outlining the possible role of genre identification in the information-retrieval process. Our assumption is that genre identification would enhance searching, first because we know that topic alone is not enough to define an information problem and, second, because search results containing genre information would be more easily understandable. Next, we discuss how information professionals have traditionally tackled the issues of representing genre in settings where topical representation is the norm. Finally, we address the issues of studying the efficacy of identifying genre in large digital collections. Because genre is often an implicit notion, studying it in a systematic way presents many problems. We outline a research protocol that would provide guidance for identifying Web document genres, for observing how genre is used in searching and evaluating search results, and finally for representing and visualizing genres.

Collective Learning in Distributed Groups (Research in progress) - Annabi, Hala; Kevin Crowston (Syracuse University School of Information Studies, 2003)
Keywords: FLOSS

A coordination theory approach to process description and redesign - Kevin Crowston; Osborn, Charley (Organizing Business Knowledge: The MIT Process Handbook, 2003)
Managers must understand, influence, and redesign organizational processes to improve business performance. In this paper we present a technique for documenting a business process. The technique has six steps: defining process boundaries, collecting data, determining actors and resources, determining activities, determining dependencies and model verification. While similar to other processmapping techniques, our approach is novel in incorporating ideas from coordination theory, thus the attention to dependencies. As a result, the technique is useful both for documenting a process and suggesting ways in which the process could be redesigned. We present an extended illustration with the hope that the technique can be used by readers of this article.
Keywords: Coordination

Defining Open Source Software project success - Kevin Crowston; Annabi, Hala; James Howison (Proceedings of the 24th International Conference on Information Systems (ICIS), 2003)
Information systems success is one of the most widely used dependent variables in information systems research. In this paper, we identify a range of measures that can be used to assess the success of Open Source Software (OSS) projects. We identify measures based on a review of the literature, a consideration of the OSS development process and an analysis of the opinions of OSS developers. For each measure, we provide examples of how they might be used in a study of OSS development.
Keywords: FLOSS

The evolution of high-reliability coordination mechanisms for collision avoidance - Kevin Crowston (The Journal of Information Technology Theory and Application (JITTA), 2003)
Resource sharing and allocation are important coordination problems in most processes and organizations. They are especially critical in transportation systems, where the resource to be shared and allocated is the space through which various vehicles move and the problem is ensuring that vehicles do not conflict in their use of the space—that is, that they do not collide. Transportation systems are interesting because they accomplish this resource allocation in a highly reliable and often highly distributed fashion. In this paper, we apply coordination theory to analyze collision avoidance as a coordination problem. Coordination theory suggests that coordination problems are created by dependencies among activities and resources that constrain how the activities can be performed. To avoid or overcome these constraints, additional work must be performed in the form of a coordination mechanism that manages the dependency. From this perspective, transportation systems can be viewed as collections of mechanisms for allocating a scarce resource, namely the space through which vehicles move. The claim of coordination theory is that having identified the type of dependency involved in transportation systems, we can consider alternative coordination mechanisms and more importantly, the tradeoffs between them. More interestingly, we can analyze how the use of information technology differentially affects the costs of different mechanisms thus shifting the tradeoff. As well, the range of coordination mechanisms identified may have implications for resource allocation in other kinds of organizations.
Keywords: Coordination

Organizing Business Knowledge: The MIT Process Handbook - Malone, Thomas W.; Kevin Crowston; Herman, George (MIT Press, 2003)
Keywords: Process

The social embeddedness of transactions: Evidence from the residential real estate industry - Sawyer, Steve; Kevin Crowston; Rolf Wigand; Allbritton, Marcel (The Information Society, 2003)
Information and communications technologies (ICT) are becoming pervasive in the residential real-estate industry and their usage is affecting the work lives of real-estate agents. Drawing on data from a regional study of the residential real-estate industry in the United States, we focus on the disintermediation or, more accurately, the reintermediation of real-estate agents in the sales process. Using data collected from interviews, direct observation, and archival records, we examine how real-estate agents are (1) taking advantage of new ICT in their work, and (2) protecting themselves from others wishing to displace their position in the real-estate value chain. Our analysis of this activity draws on two contrasting theoretical perspectives to illuminate the roles of residential real-estate agents: transaction cost and social capital. The results of this study provide insights into the ways in which ICT are used to build and draw on the social relationships that underpin the actual transactions, to help guide the process of buying/selling a house, and to invoke expertise as needed.
Keywords: Computer-Mediated Communication; Coordination; Real Estate

The social structure of Open Source Software development teams - Kevin Crowston; James Howison (The IFIP 8.2 Working Group on Information Systems in Organizations Organizations and Society in Information Systems (OASIS) Workshop, 2003)
Open Source Software development teams provide an interesting and convenient setting for studying distributed work. We begin by answering perhaps the most basic question: what is the social structure of these teams? Based on a social network analysis of interactions represented in 62,110 bug reports from 122 large and active projects, we find that some OSS teams are highly centralized, but contrary to expectation, others are not. Furthermore, we find that the level of centralization is negatively correlated with project size, suggesting that larger projects become more modular. The paper makes a further methodological contribution by identifying appropriate analysis approaches for interaction data. We conclude by sketching directions for future research.
Keywords: FLOSS

A Taxonomy of Organizational Dependencies and Coordination Mechanisms - Kevin Crowston (Organizing Business Knowledge: The MIT Process Handbook, 2003)
Interdependency and coordination have been perennial topics in organization studies. The two are related because coordination is seen as a response to problems caused by dependencies. Past studies, however, describe dependencies and coordination mechanisms only in general terms, without characterizing in detail differences between dependencies, the problems dependencies create or how the proposed coordination mechanisms address those problems. This vagueness makes it difficult or impossible to determine what alternative coordination mechanisms might be useful in a given circumstance or to directly translate these alternative designs into specifications of individual activities. In this paper I develop a taxonomy of dependency types by considering possible combinations of activities using resources. The taxonomy includes task-resource dependencies and three types of task-task dependencies: shared resources, producer-consumer and common output. For each type of dependency, alternative coordination mechanisms are described. I conclude by discussing how the taxonomy helps to analyze organizational processes and suggest alternative processes. Although you will perform with different ingredients for different dishes, the same general processes are repeated over and over again. As you enlarge your repertoire, you will find that the seemingly endless babble of recipes begins to fall rather neatly into groups of theme and variations... --Child, Bertholle and Beck (1981, p. vii)
Keywords: Coordination

Using the service encounter model to enhance our understanding of business-to-consumer transactions in an electronic environment. - Massad, Nelson; Kevin Crowston (Proceedings of the 16th Bled eCommerce Conference, 2003)
The aim of this paper is to provide an alternative perspective to enhance our understanding of the transactions between customers and service providers in an electronic environment. The service encounter literature is well established in the Marketing field and provides an alternative model to explore online business-to-consumer transactions. The taxonomy of antecedents of satisfaction developed from this model has been tested over time, across respondents (i.e., customers’ perspective vs. employees’ perspective), and across different settings. This taxonomy, however, has been mostly restricted to the bricks-and-mortar environment. Based on the analysis of a pretest sample of customer-reported online experiences, the taxonomy has the potential to enhance our understanding of business-to-consumer online transactions. The next step is to carry out a complete study in order refine the taxonomy to account for the electronic context.
Keywords: E-commerce

Discontinuities and continuities: A new way to understand virtual work - Mary Beth Watson-Manheim; Chudoba, Katherine M.; Kevin Crowston (Information, Technology & People, 2002)
"Virtual’’ is a potent buzzword, freely applied to many situations, with many meanings. In this exploratory study, we develop a more precise understanding of "virtual’’ to describe changing work environments. Specifically, we propose a framework to classify work environments based on the type of discontinuities involved. Discontinuities are gaps or a lack of coherence in aspects of work. The framework allows us to compare research across different topics and work settings. We use the framework to classify 75 published articles on virtual work environments or earlier, related research streams. We observed that many studies were simultaneously addressing existing or emerging continuities, factors or strategies for overcoming discontinuities. The focus of "virtual’’ is on changes in the work environment; however, our analysis suggests the need to be equally aware of factors that have not changed and which may become more critical with the introduction of discontinuities.
Keywords: Discontinuity; Virtuality

Discontinuities and Post-Bureaucratic Organizing: A Framework and Research Propositions - Chudoba, Katherine M.; Kevin Crowston; Mary Beth Watson-Manheim (Academy of Management Conference, 2002)
Some organization theorists argue that prevailing theories of organizing are based primarily on detailed observations of bureaucratic work, but that the nature of work today is sufficiently different to bring the applicability of these theories into question. They note in particular the growth in white collar and service workers, the rise of contingent work and the increased application of computer technologies. While various kinds of non-bureaucratic work such as project-based work and non-traditional employees such as contractors is not new, the pace and intensity of work enabled by communications technologies suggest that a postbureaucratic theory of work may be appropriate. Indeed, virtual has become shorthand for novel work arrangements involving telecommuters or virtual organizations. We propose that an increasingly important characteristic of non-bureaucratic work settings is the fact that the workers in these environments face discontinuities, that is, a lack of coherence in aspects of their work, such as the work setting, task, relations with other workers or managers. In this paper, we argue that studying how discontinuities have been managed in a variety of settings may offer insights into the nature of post-bureaucratic work. The first contribution of this paper is a framework that illuminates commonalties in diverse non-bureaucratic work settings and thus suggests how the existing research in these settings might be integrated. Based on this framework, we then consider how various existing theories might be integrated into a theory of post-bureaucratic organizing. We conclude by proposing a set of questions for future research based on this perspective.
Keywords: Discontinuity; Virtuality

A New Perspective on "Virtual":Analyzing Discontinuities in the Work Environment - Mary Beth Watson-Manheim; Kevin Crowston; Chudoba, Katherine M. (Proceedings of the 35th Hawaii International Conference on System Sciences (HICIS), 2002)
The word "virtual" has become a potent buzzword and, as such, is freely applied to many situations, with many meanings. As a result, it, like other buzzwords, is in danger of meaning nothing. In this paper, we develop a more precise understanding of the use of the word "virtual" to describe changing work environments. Our specific contribution is to propose a framework to classify these different work environments based on the type of discontinuities involved. This framework enables us to compare research across a variety of different topics and work settings and further develop a foundation for future research investigating managing and working in this new environment. In this paper; we test our framework by using it to classify 75 published articles on virtual work environments and from earlier research streams.
Keywords: Virtuality

Open source software projects as virtual organizations: Competency rallying for software development - Kevin Crowston; Scozzi, Barbara (IEE Proceedings Software, 2002)
The contribution of this paper is the identification and testing of factors important for the success of Open Source Software (OSS) projects. We present an analysis of OSS communities as virtual organizations and apply Katzy and Crowston’s competency rallying (CR) theory to the case of OSS development projects. CR theory suggests that project participants must develop necessary competencies, identify and understand market opportunities, marshal competencies to meet the opportunity and manage a short-term cooperative process. Using data collected from 7477 OSS projects hosted by the SourceForge system (http://sourceforge.net/), we formulate and test a set of specific hypotheses derived from CR theory. The empirical data analysis supports the majority of these hypotheses, suggesting that CR theory provides a useful lens for studying OSS projects.

The effects of market-enabling Internet agents on competition and prices - Kevin Crowston; MacInnes, Ian (Journal of Electronic Commerce Research, 2001)
The Internet offers a vision of ubiquitous electronic commerce. A particularly useful feature is the ability to automate the search for price or other information across multiple vendors by using an “agent” to retrieve relevant information. The use of agents has the potential to dramatically reduce buyers’ search costs. We develop a framework that suggests that vendors who sell products with many differentiating factors beyond price will tend to accept agents, while vendors of commodities or branded goods will tend to resist them unless they have lower costs than their competitors. Empirically, we found that agents seem to be accepted for differentiated goods, but resisted for more commoditized goods, though not universally. An analysis of prices from one agent shows that 1) a small number of vendors tended to have the lowest prices and 2) while divergence in pricing remains, price dispersion declined over the period studied.
Keywords: Electronic Commerce; Internet Agent; Market-enabling

Genre based navigation on the Web - Roussinov, Dmitri; Kevin Crowston; Nilan, Michael; Kwaśnik, Barbara H.; Xiaozhong Liu; Cai, J. (Proceedings of the 34th Hawai'i International Conference on Systems Science (HICSS), 2001)
We report on our ongoing study of using the genre of Web pages to facilitate information exploration. By genre, we mean socially recognized regularities of form and purpose in documents (e.g., a letter, a memo, a research paper). Our study had three phases. First, through a user study, we identified genres which most/least frequently meet searchers' information needs. We found that certain genres are better suited for certain types of needs. We identified five (5) major groups of document genres that might be used in an interactive search tool that would allow genrebased navigation. We tried to balance the following dual objectives: 1) each group should be recognizable by a computer algorithm as easily as possible 2) each group has a better chance of satisfying particular types of information needs. Finally, we developed a novel user interface for a web searching that allows genre-based navigation through three major functionalities: 1) limiting search to specified genres 2) visualizing the hierarchy of genres discovered in the search results and 3) accepting user feedback on the relevancy of the specified genres.

Identifying document genre to improve web search effectiveness - Kwaśnik, Barbara H.; Kevin Crowston; Nilan, Michael; Roussinov, Dmitri (The Bulletin of the American Society for Information Science and Technology, 2001)

Information and communication technologies in the real estate industry: Results of a pilot survey [Research in progress] - Rolf Wigand; Kevin Crowston; Sawyer, Steve; Allbritton, Marcel (Proceedings of the European Conference on Information Systems (ECIS), 2001)
We have been studying the growing use of information and communication technologies (ICT) in the residential real estate industry and the effects of this use on how realtors work. Earlier stages of our project involved qualitative research to develop a better understanding of the industry, the work of realtors and their use of ICT. In this paper we report on the results of qualitative research and a pilot of a survey intended to gather large-scale data on realtors and ICT use.
Keywords: Real Estate

Investigating the interplay between structure and information and communications technology in the real estate industry - Kevin Crowston; Sawyer, Steve; Rolf Wigand (Information, Technology & People, 2001)
Information and communication technologies (ICTs) are reshaping many industries, often by reshaping how information is shared. However, while the effects and uses of ICT are often associated with organizations (and industries), their use occurs at the individual level. To explore the relationships between individual uses of ICT and changes to organization and industry structures, we examined the residential real estate industry. As agents, buyers and sellers increase their uses of ICT, they also change how they approach their daily work. The increasing uses of ICT are simultaneously altering industry structures by subverting some of the realtors' control over information while also reinforcing the existing contract-based structures. This structurational perspective and our findings help to explain why information intermediaries persist when technology-based perspectives would suggest their disappearance.
Keywords: Real Estate

Cutting out the middleman: The disintermediating potential of the Web in real estate in the US and New Zealand - Kevin Crowston (2000)
Keywords: Real Estate

How do information and communication technologies reshape work? Evidence from the residential real estate industry - Sawyer, Steve; Kevin Crowston; Allbritton, Marcel; Rolf Wigand (Proceedings of International Conference on Information Systems (ICIS), 2000)
We are exploring how information and communication technology (ICT) use affects the work lives of real estate agents, the process of selling/buying houses, and the overall structure of the residential real estate industry. Earlier stages of our work involved intensive field research on how real estate agents use ICT. In this paper, we report on the design and analysis of a pilot survey of 868 agents intended to investigate their ICT use more generally. Analysis of the 153 responses to this survey sheds light on how ICT use supports information control, enables process support, and helps agents to extend and maintain their social capital.
Keywords: Real Estate

The on-line Ph.D. as computer-supported cooperative work - Kevin Crowston (Decision Line, 2000)
This issue’s column by Professor Kevin Crowston is in response to Professor Peter Carr’s column about the online Ph.D. that appeared in the last issue of Decision Line (Vol. 31, No. 3). Dr. Crowston summarizes the previous arguments and then discusses current research on the subject of applying computer-supported work to graduate education. For instance, he emphasizes the importance of face-to-face interaction for particular kinds of collaborative tasks. I hope you find this next entry in the debate about “online” versus “face-to-face” Ph.D. programs to be a stimulating and enlightening exercise as you contemplate the important questions surrounding the future of doctoral education.
Keywords: Computer-Mediated Communication; Learning

Process as theory in information systems research - Kevin Crowston (Proceedings of the IFIP TC8 WG8. 2 International Working Conference on the Social and Organizational Perspective on Research and Practice in Information Technology, 2000)
Many researchers have searched for evidence of organizational improvements from the huge sums invested in ICT. Unfortunately, evidence for such a pay back is spotty at best (e.g., Brynjolfsson 1994; Brynjolfsson and Hitt 1998; Meyer and Gupta 1994). On the other hand, at the individual level, computing and communication technologies are increasingly merging into work in ways that make it impossible to separate the two (e.g., Bridges 1995; Gasser 1986; Zuboff 198). This problem—usually referred to as the productivity paradox—is an example of a more pervasive issue: linking phenomena and theories from different levels of analysis. Organizational processes provide a bridge between individual, organizational, and even industrial level impacts of information and communication technologies (ICT). Viewing a process as the way organizations accomplish desired goals and transform inputs into outputs makes the link to organizational outcomes. Viewing processes as ordered collections of activities makes the link to individual work, since individual actors perform these activities. As well, process theories can be a useful milieu for theoretical interplay between interpretive and positivist research paradigms. A process-centered research framework is illustrated with an analysis of the process of seating and serving customers in two restaurants. The analysis illustrates how changes in individual work affect the process and thus the organizational outcomes and how processes provide a theoretical bridge between work at different levels of analysis.
Reprinted in Malone, T. W., Crowston, K. & Herman, G. (Eds.) Organizing Business Knowledge: The MIT Process Handbook (pp. 177–190). Cambridge, MA: MIT Press, 2003.
Keywords: Process

Reproduced and emergent genres of communication on the World-Wide Web - Kevin Crowston; Williams, Marie (Information Society, 2000)
The World Wide Web is growing quickly and being applied to many new types of communications. As a basis for studying organizational communications, Yates and Orlikowski (1992; Orlikowski & Yates, 1994) proposed using genres. They defined genres as “typified communicative actions characterized by similar substance and form and taken in response to recurrent situations” (Yates & Orlikowski, 1992, p. 299). They further suggested that communications in a new media would show both reproduction and adaptation of existing communicative genres as well as the emergence of new genres. We studied these phenomena on the World Wide Web by examining 1000 randomly selected Web pages and categorizing the type of genre represented. Although many pages recreated genres familiar from traditional media, we also saw examples of genres being adapted to take advantage of the linking and interactivity of the new medium and novel genres emerging to t the unique communicative needs of the audience.We suggest that Web-site designers consider the genres that are appropriate for their situation and attempt to reproduce or adapt familiar genres.
Proceedings of the Thirtieth Hawaii International Conference on Systems Science (HICSS–30), Maui, HI, USA, January.

Constructing intelligent agents with Java: A programmerʼs guide to smarter applications [Book review] - Kevin Crowston (Internet Research: Electronic Networking Applications and Policy, 1999)

The effects of linking on genres of web documents - Kevin Crowston; Williams, Marie (Proceedings of the 32nd Hawai'i International Conference on Systems Science (HICSS), 1999)
Documents on the Web can be composed of multiple Web pages, suggesting the need to consider how linking between pages affects a document’s form. We illustrate this point by considering patterns of linking in a common genre of document, the Frequently Asked Questions file or FAQ. In a sample of 70 FAQs, we found four patterns of linking: no links, links within the page, links to pages on the same host and links to other hosts. We suggest that links that tie together document pieces simply recreate the already accepted FAQ genre, but links that provide navigation within the document or that link to other information sources begin to extend and adapt the FAQ genre to the needs and capabilities of the Web.

ICT in the real estate industry: Agents and social capital - Sawyer, Steve; Kevin Crowston; Rolf Wigand (Proceedings of the Advances In Social Informatics And Information Systems Track, Americas Conference on Information Systems, 1999)
For the past year, we have been involved in a study of the ways in which information and communications technologies (ICT) are becoming pervasive in the residential real estate industry and their effects on the work lives of real estate agents. Our initial results suggest that analyzing an agent’s social capital–the set of social resources embedded in relationships–provides insight into how real estate agents work and how that work is affected by ICT. Social capital has three components: structural, relational, and cognitive. ICT use affects all three components.
Keywords: Real Estate

Real estate war in cyberspace: An emerging electronic market? - Kevin Crowston; Rolf Wigand (International Journal of Electronic Markets, 1999)
In this paper, we explore how electronic commerce, the World-Wide Web in particular, is affecting the real estate industry. Real estate is a promising setting for studying electronic commerce because it is an information-intensive and informationdriven industry; transaction-based, with high value and asset-specificity; with many market-intermediaries (agents and brokers who connect buyers and sellers rather than buying or selling themselves); and experiencing on-going information technology (IT) related changes. We analyze a real estate transaction to suggest where IT might change the process of buying or selling a house and discuss several current ventures in this area. This analysis suggests that Web-based commerce is eroding the long-enjoyed information monopoly of real-estate agents and electronic commerce applications have the potential to drastically change current practices in the real-estate industry, including the disintermediation of agents.
Keywords: Real Estate

Tools for inventing organizations: Toward a handbook of organizational processes - Malone, Thomas W.; Kevin Crowston; Lee, Jintae; Pentland, Brian; Dellarocas, Chrysanthos; Wyner, George; Quimby, John; Osborn, Charley; Bernstein, Avi; Herman, George; Klein, Mark; O'Donnell, Elissa (Management Science, 1999)
This paper describes a novel theoretical and empirical approach to tasks such as business process redesign and knowledge management. The project involves collecting examples of how different organizations perform similar processes, and organizing these examples in an on-line "process handbook." The handbook is intended to help people: (1) redesign existing organizational processes, (2) invent new organizational processes (especially ones that take advantage of information technology), and (3) share ideas about organizational practices. A key element of the work is an approach to analyzing processes at various levels of abstraction, thus capturing both the details of specific processes as well as the "deep structure" of their similarities. This approach uses ideas from computer science about inheritance and from coordination theory about managing dependencies. A primary advantage of the approach is that it allows people to explicitly represent the similarities (and differences) among related processes and to easily find or generate sensible alternatives for how a given process could be performed. In addition to describing this new approach, the work reported here demonstrates the basic technical feasibility of these ideas and gives one example of their use in a field study.
Reprinted in Malone, T. W., Crowston, K. & Herman, G. (Eds.) Organizing Business Knowledge: The MIT Process Handbook. Cambridge, MA: MIT Press, 2003. Reprinted in Malone, T. W., Laubacher, R. & Scott Morton, M. S. (Eds.). Inventing the Organizations of the 21st Century. Cambridge, MA: MIT Press, 2003.
Keywords: Coordination; Handbook; Process

Communicative style and gender differences in computer-mediated communications - Kevin Crowston; Kammerer, Ericka (Cyberghetto or Cybertopia: Race, Class and Gender on the Internet, 1998)
This chapter reports on an experiment that explored how gender interacts with communicative style to affect decisions to participate in computer-mediated communications (e.g., Usenet newsgroups). Although some field studies indicate that style has a differential effect on men and women, the data fails to confirm this hypothesis. However, significant differences were found in interest in topics, as well as effects of style across all subjects, which have implications for the design of future studies on this topic.
Keywords: Computer-Mediated Communication

Coordination and collective mind in software requirements development - Kevin Crowston; Kammerer, Ericka (IBM Systems Journal, 1998)
The purpose of this study was to understand how the group processes of teams of software requirements analysts led to problems and to suggest possible solutions. Requirements definition is important to establish the framework for a development project. Researchers have proposed numerous requirements development techniques, but less has been done on managing teams of requirements analysts. To learn more about group processes within such teams, we studied two teams of analysts developing requirements for large, complex real-time systems. These teams had problems ensuring that requirements documents were complete, consistent, and correct; fixing those problems required additional time and effort. To identify sources of problems, we applied two theories of collective action, coordination theory and collective mind theory. Coordination theory suggests that a key problem in requirement analysis is identifying and managing dependencies between requirements and among tasks. Most requirements methods and tools reflect this perspective, focusing on better representation and communication of requirements. The collective mind perspective complements these suggestions by explaining how individuals come to understand how their work contributes to the work of the group. This perspective suggests that deficiencies in actors' representations of the process and subordination to collective goals limit the value of their contributions.
Keywords: Software Development

Price Behavior in a Market with Internet Buyer's Agents (Research in progress paper) - Kevin Crowston (Proceedings of the International Conference on Information Systems (ICIS), 1998)
Keywords: Electronic Commerce; Internet Agent

Use of the web for electronic commerce in real estate - Kevin Crowston; Rolf Wigand (Proceedings of the Association for Information Systems Americas Conference, 1998)
In this paper, we will explore the ways in which electronic commerce, the World-wide Web (WWW) in particular, is affecting the real estate industry. Real estate is a promising setting for studying electronic commerce because it is an information-intensive and information-driven industry; transaction-based, with high value and asset-specificity; market-intermediary (agents and brokers connect buyers and sellers rather than buying or selling themselves); and experiencing on-going information technology (IT) related changes. In this paper, we apply a coordination theory framework to suggest where IT might change the process of buying or selling a house. Electronic commerce applications have the potential to drastically change current practices in the real-estate industry, including the disintermediation of agents. Web-based commerce is eroding the long-enjoyed information monopoly of real-estate agents. We illustrate this potential by reviewing a number of existing real estate websites that demonstrate the possible impact of electronic commerce on this industry.
Keywords: Real Estate

Comparative diffusion of the telephone and the world wide web: An analysis of rates of adoption - Chen, H.; Kevin Crowston (Proceedings of the WebNet `97–-World Conference of the WWW, Internet and Intranet, 1997)
This paper investigates the diffusion process of the World Wide Web technology by means of a comparison with telephone diffusion at the end of the nineteenth century. The Web technology has diffused at exponentially around the world. In contrast, the telephone technology, a similar innovation of interactive communication technology imbued with typical uncertainty and impedance, took several decades to diffuse. This paper diagnoses the fundamental differences between these two innovations by analyzing their innovation perceived attributes, such as relative advantage, compatibility, complexity, trialability and observability, and attempts to explain the determinants of their rates of adoption.
Keywords: Computer-Mediated Communication

A coordination theory approach to organizational process design - Kevin Crowston (Organization Science, 1997)
An important practical problem for many managers is finding alternative processes for performing a desired task, for example, one that is more efficient, cheaper, or that is automated or enhanced by the use of information technology. Improving processes also poses theoretical challenges. Coordination theory provides an approach to the study of processes. In this view, the design of a process depends on the coordination mechanisms chosen to manage dependencies among tasks and resources involved in the process. In this paper, I use coordination theory to analyze the software change process of a large mini-computer manufacturer. Mechanisms analyzed include those for task assignment, resource sharing, and managing dependencies between modules of source code. For each, I suggest alternative mechanisms and thus alternative designs for the process. The organization assigned problem reports to engineers based on the module that appeared to be in error, since engineers only worked on particular modules. Alternative task assignment mechanisms include assignment to engineers based on work- load or market-like bids. Modules of source code were not shared, but rather "owned" by one engineer, thus reducing the need for coordination. An alternative resource sharing mechanism would be needed to manage source code if multiple engineers could work on the same modules. Finally, engineers managed dependencies between modules informally, relying on their personal knowledge of which other engineers used their code; alternatives include formally defining the interfaces between modules and tracking their users. Software bug fixing provides a microcosm of coordination problems and solutions. Similar coordination problems arise in most processes and are managed by a similar range of mechanisms. For example, diagnosing bug reports and assigning them to engineers may have interesting parallels to diagnosing patients and assigning them to specialists. While the case presented does not formally test coordination theory, it does illustrate the potential of coordination theory for exploring the space of organizational processes. Future work includes developing more rigorous techniques for such analyses, applying the techniques to a broader range of processes, identifying additional coordination problems and mechanisms and developing tools for collecting and comparing processes and automatically suggesting potential alternatives.
Keywords: Coordination

Virtual Operations and Agile Organizations: workshop proceedings (ASLIB, 1997)
European Agility Forum, August 21-22 1997
Keywords: Virtual Organization

An approach to evolving novel organizational forms - Kevin Crowston (Computational and Mathematical Organization Theory, 1996)
A key problem in organization theory is to suggest new organizational forms. In this paper, I suggest the use of genetic algorithms to search for novel organizational forms by reproducing some of the mechanics of organizational evolution. Issues in using genetic algorithms include identification of the unit of selection, development of a representation and determination of a method for calculating organizational fitness. As an example of the approach, I test a proposition of Thompson's about how interdependent positions should be assigned to groups. Representing an organization as a collection of routines might be more general and still amenable to evolution with a genetic algorithm. I conclude by discussing possible objections to the application of this technique.
Keywords: Coordination

Market-enabling Internet agents - Kevin Crowston (Proceedings of the International Conference on Information Systems (ICIS), 1996)
The growth of the Internet offers a vision of ubiquitous electronic commerce. A particularly exciting feature of Internet commerce is the ability to automate the search for price or other product information across multiple suppliers by using an “agent” to retrieve the relevant information. The use of such agents has the potential to dramatically reduce buyers’ search costs. We argue that such agents effectively transform a diverse set of offerings into an economically efficient market and that their use should therefore be analyzed in these terms. In this paper, we present a simple model of the competitive effects of agents used to support purchasing. The model suggests that agents can be successful for diversified goods but resisted for commodities and near-commodities. We illustrate our model by analyzing the situation of current electronic commerce ventures on the Internet.
Keywords: Internet Agent

Evolving novel organizational forms - Kevin Crowston (Computational Organization Theory, 1994)
A key problem in organization theory is to suggest new organizational forms. In this paper, I suggest the use of genetic algorithms to search for novel organizational forms by reproducing some of the mechanics of organizational evolution. Issues in using genetic algorithms include identification of the unit of selection, development of a representation and determination of a method for calculating organizational fitness. As an example of the approach, I test a proposition of Thompson's about how interdependent positions should be assigned to groups. Representing an organization as a collection of routines might be more general and still amenable to evolution with a genetic algorithm. I conclude by discussing possible objections to the application of this technique.
Other publications: Crowston, K. Evolving novel organizational forms. In Ingemar Hulthage (Ed.), Computational Organization Design: Papers from the 1994 Spring Symposium (Technical Report SS-94-07) (pp. 35–42). Menlo Park, CA: AAAI Press.
Keywords: Coordination

The interdisciplinary study of coordination - Malone, Thomas W.; Kevin Crowston (Computing Surveys, 1994)
This survey characterizes an emerging research area, sometimes called coordination theory, that focuses on the interdisciplinary study of coordination. Research in this area uses and extends ideas about coordination from disciplines such as computer science, organization theory, operations research, economics, linguistics, and psychology. A key insight of the framework presented here is that coordination can be seen as the process of managing dependencies among activities. Further progress, therefore, should be possible by characterizing different kinds of dependencies and identifying the coordination processes that can be used to manage them. A variety of processes are analyzed from this perspective, and commonalities across disciplines are identified. Processes analyzed include those for managing shared resources, producer/consumer relationships, simultaneity constraints, and tank/subtask dependencies. Section 3 summarizes ways of applying a coordination perspective in three different domains: (1) understanding the effects of information technology on human organizations and markets, (2) designing cooperative work tools, and (3) designing distributed and parallel computer systems. In the final section, elements of a research agenda in this new area are briefly outlined.
Reprinted in Olson, G. M, Malone, T. W., and Smith, J. B. (Eds.) Coordination Theory and Collaboration Technology (pp. 7–50). Mahwah, NJ: Lawrence Erlbaum Associates, 2001. Reprinted in Malone, T. W., Crowston, K. & Herman, G. (Eds.) Organizing Business Knowledge: The MIT Process Handbook. Cambridge, MA: MIT Press, 2002.
Keywords: Coordination

Using a Process Handbook to design organizational processes - Dellarocas, C; Lee, Jintae; Malone, Thomas W.; Kevin Crowston; Pentland, Brian (Computational Organization Design: 1994 AAAI Spring Symposium, 1994)
Technical Report SS-94-07
Keywords: Handbook; Process

Modelling coordination in organizations - Kevin Crowston (Artificial Intelligence in Organization and Management Theory, 1991)
Keywords: Coordination

Towards a Coordination Cookbook: Recipes for Multi-Agent Action - Kevin Crowston (MIT Sloan School of Management, 1991)
This thesis presents the first steps towards a theory of coordination in the form of what I call a coordination cookbook. My goal in this research is hypothesis generation rather than hypothesis testing: I attempt to develop a theory of coordination grounded in detailed empirical observation. I am especially interested in using this theory to identify ways of coordinating that may become more desirable when information technology is used to perform some of the coordination. I address the following question: how can we represent what people do to coordinate their actions when they work together on common goals, in a way that reveals alternative approaches to achieving those goals? To answer this question, I study groups of people making engineering changes to complex products as an example of a coordination-intensive task. I perform detailed case studies of the change process in three organizations: an automobile manufacturer, a commercial aircraft manufacturer and a computer system software developer. To analyze these cases, I develop a technique for describing the behaviour of the members of an organization, based on research in distributed artificial intelligence (DAI). I first develop a data-flow model of the change process to identify what information was used and how it was processed by the different members of the organization. Then, using ideas from DAI, I model what each individual must have known about the task and the rest of the organization to act as observed. To develop a theory of coordination, I generalize from these specific individuals to the kinds of tasks they performed. I develop a typology of interdependencies between organizational tasks and objects in the world (including resources and products). This typology includes four categories of coordination needs, due to interdependencies between: (1) different tasks, (2) tasks and subtasks, (3) tasks and objects in the world and (4) different objects. I then re-examine the cases to identify the coordination methods used to address these needs. (These coordination methods are similar in spirit to the weak problem solving methods of cognitive science.) I represent each method by a set of what I call coordination recipes that identify the goals, capabilities and knowledge of the individuals involved. In some cases, consideration of the possible distributions of these elements suggests approaches other than those actually observed. This framework allows an analyst to abstract from a description of how a particular organization performs a task to a description of the coordination needs of that task and a set of alternative coordination methods that could be used to address those needs. The results of my thesis should be useful in several ways. A better understanding of how individuals work together may provide a more principled approach for designing new computer applications, for analyzing the way organizations are currently coordinated and for explaining perceived problems with existing approaches to coordination. By systematically exploring the space of possible coordination strategies, we may be able to discover new kinds of organizations—organizations in which humans and computers work together in as yet unimagined ways.
Keywords: Coordination; Process

What is coordination theory and how can it help design cooperative work systems? - Malone, Thomas W.; Kevin Crowston (Proceedings of the Third Conference on Computer-supported Cooperative Work (CSCW), 1990)
Reprinted in D. Marca and G. Bock. (Eds.) Groupware: Software for Computer-Supported Cooperative Work (pp. 100–115). Washington: IEEE Press, 1992.
Keywords: Coordination

How do experienced Information Lens users use rules? - Mackay, Wendy E.; Malone, Thomas W.; Kevin Crowston; Rao, Ramana; Rosenblitt, David; Card, Stuart K. (Proceedings of the Conference on Human Factors in Computing Systems (SIGHI), 1989)
The Information Lens provides electronic mail users with the ability to write rules that automatically sort, select, and filter their messages. This paper describes preliminary results from an eighteen-month investigation of the use of this system at a corporate test site. We report the experiences of 13 voluntary users who have each had at least three months experience with the most recent version of the system. We found that: 1. People without significant computer experience are able to create and use rules effectively. 2. Useful rules can be created based on the fields present in all messages (e.g., searching for distribution lists or one's own name in the address fields or for character strings in the subject field), even without any special message templates. 3. People use rules both to prioritize messages before reading them and to sort messages into folders for storage after reading them. 4. People use delete rules primarily to filter out messages from low-priority distribution lists, not to delete personal messages to themselves.
Proceedings Reprinted as ACM SIGCHI Bulletin, Volume 20, Issue SI.
Keywords: Computer-Mediated Communication

Information technology and work organization - Kevin Crowston; Malone, Thomas W. (Handbook of Human-Computer Interaction, 1988)
Reprinted in Thomas J. Allen and Michael S. Scott Morton. (Eds), Information Technology and the Corporation of the 1990s (pp. 249–275). New York: Oxford, 1994.
Keywords: Information Technology; Organization

Intelligent Software Agents - Kevin Crowston; Malone, Thomas W. (BYTE, 1988)
Keywords: Internet Agent

Cognitive science and organizational design: A case study of computer conferencing - Kevin Crowston; Malone, Thomas W.; Lin, Felix (Human Computer Interaction, 1987)
Many researchers have investigated and speculated about the link between information technology and organizational structure with very mixed results. This paper suggests that part of the reason for these mixed results is the coarseness of previous analyses of both technology and structure The paper describes a new and much more detailed perspective for investigating this link. Using concepts of object-oriented programming from artificial intelligence, the information processing that occurs in organizations is characterized in terms of the kinds of messages people exchange and the ways they process those messages The utility of this approach is demonstrated through the analysis of a case in which a reduction in levels of management is coupled with the introduction of a computer conferencing system The detailed model developed for this case helps explain both macro-level data about the changes in the organizational structure, and micro-level data about individuals' use of the svstem.
Originally appeared in Proceedings of the Conference on Computer Supported Cooperative Work (CSCW ʼ86) (pp. 43–61). Austin, TX: ACM, 1986. Reprinted in I. Grief (Ed.) Computer-supported cooperative work: A book of readings (pp. 713–740). San Mateo, CA: Morgan Kaufmann, 1988.
Keywords: Organizational Modelling

Cognitive science and organizational design: Assessing the impact of information technology on organizations - Kevin Crowston (Data Base, 1987)
Keywords: Information Technology; Organization

Assessing the impact of information technology on enterprise level performance - Kevin Crowston; Treacy, Michael E. (Proceedings of the Sixth International Conference on Information Systems (ICIS), 1986)
Implicit in most of what we do in MIS is the belief that information technology (IT) has an impact on the bottom line of the business. Surprisingly, we rarely know if this is true. It is very difficult to trace and measure the effects of information technology through a web of intermediate impacts upon enterprise level performance. In this paper, we review research that has been performed on enterprise level impacts of information systems, with a particular emphasis on research that has attempted to measure those impacts. We begin with a survey of articles published within the last ten years. This is followed by a discussion of the reference disciplines that underlie most of this work. From this we are able to draw conclusions about future directions for research in this area.
Keywords: Information Technology; Organization

Cognitive science and organizational design - Kevin Crowston; Malone, Thomas W.; Lin, Felix (Proceedings of the 1986 ACM conference on Computer-supported cooperative work (CSCW '86), 1986)
Many researchers have investigated and speculated about the link between information technology and organizational structure with very mixed results. This paper suggests that part of the reason for these mixed results is the coarseness of previous analyses of both technology and structure. The paper describes a new and much more detailed perspective for investigating this link. Using concepts of object-oriented programming from artificial intelligence, the information processing that occurs in organizations is characterized in terms of the kinds of messages people exchange and the ways they process those messages. The utility of this approach is demonstrated through the analysis of a case in which a reduction in levels of management is coupled with the introduction of a computer conferencing system. The detailed model developed for this case helps explain both macro-level data about thd changes in the organizational structure, and micro-level data about individuals' use of the system.
Keywords: Organizational Modelling