Research Group Computational Auditory Perception

What is universal in human perception and cognition, and what is culturally contingent? How are our mental representations shaped by our experiences?

The Computational Auditory Perception Research Group studies the internal representations that support and shape our sensory and cognitive abilities, and how those representations are themselves determined by both nature and nurture, by biology and socio-cultural interactions. The methodologies we develop employ techniques from machine learning, such as deep generative synthesis algorithms, alongside a significant data-intensive expansion of the scale and scope of experimental research both by means of massive online experiments and fieldwork in locations around the globe. Most of the group’s work thus far has involved computational approaches to high-level cognition with a focus on audition and music, but also, more recently, vision.

Our research program has three foci:
1) Internal representations  2) The universality and diversity of human perception, and  3) The data science of cultural artifacts. 

###

1. Internal Representations 

Human perception is rich, multi-dimensional and contextual. (Consider, for example, the ways in which emotion is conveyed by the voice: by pitch, volume, and many other parameters.) Yet behavioral methods, biased by their limitation to one-dimensional and simplified stimulus spaces, typically produce an impoverished understanding of human perception. Inspired by Monte Carlo Markov Chain techniques borrowed from machine learning and physics, our research program addresses this gap by developing new adaptive sampling methods, in which each successive stimulus depends on the subject’s response to the previous stimulus. Such processes allow us to sample from the complex and high-dimensional joint distribution associated with internal representations and obtain high resolution maps of perceptual spaces.

2. Understanding the Universality and Diversity of Human Perception

Traditional psychology experiments recruit participants who have access to computer technology and are located in industrialized countries such as the US and India. This sampling constraint severely limits our understanding of the roles of nature and nurture in human perception, as the similarities we find between participants may stem either from universal biological mechanisms or from comparable exposure. To overcome this limitation, we apply computational methods and analysis to data obtained in field research with diverse populations around the world. We also focus on developing new infrastructures for massive online experiments.

3. The Data Science of Cultural Artifacts

Simulated cultural evolution experiments give rise to predictions regarding the connections between social structure and the emergent features of cultural artifacts which can then be tested by a comparison with real-world data. This branch of the research group studies the emergence of social structure in both real and virtual worlds which iteratively interact with the behavior of experimental subjects.

    PsyNet: The Online Human Behavior Lab of the Future

    Over the past decade, research in psychology, sociology, and economics has begun to incorporate online participant pools to varying extents. These online pools allow experimenters to increase the sample size and diversity of their participant groups, while also enabling experiments that would be nearly impossible to conduct in the lab, for example exploring interactions between thousands of participants within social networks.

    more

    Iterated Learning

    When we perceive complex and/or ambiguous scenes, we rely on prior information in order to make sense of what we see or hear.

    more

    Experimenting with Governance in Virtual Worlds

    Culture is deep inside us, in our ability to speak, in our sense of belonging, in our values. The capacity of our brain to adapt to and integrate culture is what makes us human: from birth, our mind is set to absorb concepts, technologies and social conventions that accumulated over thousands of generations.

    more

    Cross-Cultural Perception

    Over 90% of psychology experiments between 2003-2007 were conducted on WEIRD subjects, hailing from Western, Educated, Industrialized, Rich, and Democratic societies (Arnett 2008). Henrich et al. (2010) have argued that these populations constitute an extremely biased sample across several critical dimensions, manifested in paradigms from basic visual and spatial perception to social cognition.

    more

    Gibbs Sampling with People

    As cognitive scientists, we are often interested in mapping the relationship between external stimuli (e.g., spoken sentences, musical chords, faces) and semantic features that the mind derives from these stimuli (e.g., happiness, sadness, pleasantness).

    more

    A Robust Cross-Platform Solution for Online Sensorimotor Synchronization Experiments

    Sensorimotor synchronization (SMS), the rhythmic coordination of perception and action, is a fundamental human skill that supports many behaviors, from repetitive daily routines to the highest forms of behavioural coordination, including music and dance (see Repp, 2005; Repp & Su, 2013, for reviews).

    more

    News

    New Findings on the Evolutionary Origins of Beat Synchronization

    [more]

    Events


    Online Event

    Musical traits and performance practice of Uruguayan candombe drumming: an approach from computational musicology [more]


    Online Event

    "Large-scale comparisons of entire social systems (or: The method of the social sciences)" [more]

    Activities


    Jacoby  presented a talk on "Universality and cross-cultural variation in mental representations of music revealed by large-scale comparisons of rhythm priors." [more]


    Jacoby  presented a talk on "Universality and cross-cultural variation in mental representations of music revealed by large-scale comparisons of rhythm priors." More information... [more]


    organized by the TEMPoral Organization of Speech (TEMPOS) research cluster at the Max Planck Institute for Psycholinguistics in Nijmegen, The Netherlands. [more]

    Press

    Gibt es ein Schlagzeuger-Gen? Erstmals haben Forschende im Erbgut danach gesucht.

    [more]

    A New Discovery About Spatial Representation in the Brain

    [more]

    Catch the Deutschlandfunk interview here.

    [more]

    Contact

    Nori Jacoby

    Dr. Nori Jacoby

    Research Group Computational Auditory Perception

    Research Group Leader

    +49 69 8300479-820

    E-Mail

    Diana Gleiß

    Diana Gleiß

    Research Groups

    Assistant

    +49 69 8300479-801

    E-Mail

    Team

    Manuel Anglada-Tort

    Dr. Manuel Anglada-Tort

    Research Group Computational Auditory Perception

    Researcher

    +49 69 8300479-822

    E-Mail

    Frank Hoeger

    Research Group Computational Auditory Perception

    Researcher

    E-Mail

    Erika Tsumaya

    Research Group Computational Auditory Perception

    Master Student

    +49 69 8300479-821

    E-Mail

    Harin Lee

    Research Group Computational Auditory Perception

    Visiting Researcher

    E-Mail

    Ofer Tchernichovski

    Ofer Tchernichovski

    Research Group Computational Auditory Perception

    Visiting Researcher

    E-Mail

    Thomas Langlois

    Research Group Computational Auditory Perception

    Visiting Researcher

    E-Mail

    Maria Zimmermann

    Research Group Computational Auditory Perception

    Scholarship holder

    E-Mail

    Alumni

    Peter Harrison

    Dr. Peter Harrison

    University Assistant Professor, University of Cambridge

    Raja Marjieh

    Raja Marjieh

    Graduate Student in Psychology, Princeton University