What Defines and Affects Replicability in Computational Communication Science?
Computational communication science (CCS) is a young but quickly growing field characterized by the use of digital traces and other media data (e.g., online news, social media) and methods suitable for collecting/generating (e.g., scraping, simulating) and analyzing (e.g., machine learning, natural language processing) those. As, for example, publication practices in the journal Computational Communication Research show, many CCS researchers have been advocating for and implementing open-science principles such as sharing data and materials. However, several challenges complicate making CCS research reproducible and replicable. These challenges presumably are a consequence of (a) the methods commonly used in the field, (b) the volatility of its topics of study and the data it makes use of, and (c) the increasing dependency on third-party data providers, such as search engines or social networking sites.
The aim of this project is to investigate the determinants and conditions for replicability in CCS. To achieve this, the project will (1) assess the potential replicability of research in CCS by means of a large-scale content analysis of publications in the field, and (2) test and evaluate the actual replicability of studies from this field via reproduction and replication of purposively selected studies.