Please visit psynet.dev for more information

Over the past decade, research in psychology, sociology, and economics has begun to incorporate online participant pools to varying extents. These online pools allow experimenters to increase the sample size and diversity of their participant groups, while also enabling experiments that would be nearly impossible to conduct in the lab, for example exploring interactions between thousands of participants within social networks. Such experiments include virtual worlds where networks of participants can interact in a way that simulates complex social systems (Wisdom et al. 2013, Shirado et al. 2017, Jayles et al. 2017), cultural transmission (Centola et al. 2015, Carr et al. 2020), governance decisions in online communities (Salganik et al. 2006, Balietti et al. 2016), large-scale perception research (Lahdelma and Eerola 2020), and online techniques for exploring high-dimensional perceptual representations in the human brain, which incorporate humans and machine-learning algorithms (Harrison et al. 2020). These new types of massive online experiments can help address the challenges of conducting basic and translational human behavior research during the COVID-19 pandemic while maintaining high standards of privacy and ethics, and also providing novel ways to study some of the most critical issues of our time, ranging from the transmission of fake news to the instability of politics within social networks (Lazer et al. 2018).

These new modes of experiments require new tools. Existing online platforms such as jsPsych, PsychoPy, Gorilla, PsiTurk, and OpenSesame are useful for creating single-player interactions, but provide very limited capacities for creating large-scale interactive experiments. In addition, existing platforms are limited in terms of their built-in security and privacy mechanisms and restricted in terms of the automation of recruiting, payment, setting up servers and managing data (e.g lioness-lab, nodeGame, Wextor, TurkServer), as well as their ability to combine pre-existing materials such as pre-screening tasks, questionnaires, stimulus types (e.g., Empirica, Dallinger, oTree, Breadboard). Moreover, some tools rely on subscription and proprietary code (e.g., Labvanced). To address these limitations, the Computational Auditory Perception Research Group at the Max Planck Institute for Empirical Aesthetics is developing a platform that enables the running of large-scale interactive studies involving hundreds or even thousands of human subjects. Our platform provides built-in modular components that can be easily woven together to form complex experimental designs, such as adaptive algorithms that sample from subjective probability distributions (MCMCP, Sanborn & Griffiths 2008; GSP, Harrison et al. 2020), while also supporting a broad range of stimulus and response types, including audio, video, multiple choice, free text, sliders, and WebGL. In the coming year we plan to make these techniques accessible to the scientific community as a mature, fully documented, and user-friendly open-source software package that we call PsyNet. We have already developed a prototype of this technology and used it to design, deploy, and implement massive interactive online experiments that include interactions between multiple participants. 

At its full form, PsyNet will provide nine core features that together transform the design of large scale virtual lab experiments: (1) Efficiency and reusability; (2) Automatic recruitment and compensation; (3) Database services & efficient storage; (4) Automatic replication of the entire experimental pipeline based on source code; (5) Security and privacy, specifically maintaining consistency with GDPR regulations and MPG ethical standards; (6) Built-in support for diverse recruiting, thereby significantly diversifying the population of participants (Henrich et al. 2010); (7) Automatic data quality assurance; (8) Extensive real-time experimental monitoring dashboard; (9) Open-ended integration with powerful cloud-based computation engines. The synergic combination of these features in PsyNet enables the development and deployment of large-scale experiments that would be extremely difficult otherwise to perform.

Thus far, we have developed a prototype of PsyNet (see architecture in Figure 1 and dashboard in Figure 2) and have already used it for several experiments (Harrison et al., 2020; see also preliminary results in Figure 3). While our prototype has mainly relied on external recruiting services (Amazon Mechanical Turk), the final product will also allow researchers to deploy PsyNet on a stand-alone server with increased privacy and security.  

PsyNet builds on the Dallinger framework (Dallinger Readthedocs), an open-source software project initially developed by Jordan Suchow (Suchow), Tom Morgan and Tom Griffiths. We collaborate with the software consultancy Jazkarta (Jazkarta) to keep Dallinger maintained and to develop new Dallinger features. In 2019 we organized a three-day workshop training researchers how to use Dallinger in their own research (Dallinger Workshop).