What is universal in human perception and cognition, and what is culturally contingent? How are our mental representations shaped by our experiences?

The Computational Auditory Perception Research Group studies the internal representations that support and shape our sensory and cognitive abilities, and how those representations are themselves determined by both nature and nurture, by biology and socio-cultural interactions. The methodologies we develop employ techniques from machine learning, such as deep generative synthesis algorithms, alongside a significant data-intensive expansion of the scale and scope of experimental research both by means of massive online experiments and fieldwork in locations around the globe. Most of the group’s work thus far has involved computational approaches to high-level cognition with a focus on audition and music, but also, more recently, vision.

Our research program has three foci:
1) Internal representations  2) The universality and diversity of human perception, and  3) The data science of cultural artifacts. 

###

1. Internal Representations 

Human perception is rich, multi-dimensional and contextual. (Consider, for example, the ways in which emotion is conveyed by the voice: by pitch, volume, and many other parameters.) Yet behavioral methods, biased by their limitation to one-dimensional and simplified stimulus spaces, typically produce an impoverished understanding of human perception. Inspired by Monte Carlo Markov Chain techniques borrowed from machine learning and physics, our research program addresses this gap by developing new adaptive sampling methods, in which each successive stimulus depends on the subject’s response to the previous stimulus. Such processes allow us to sample from the complex and high-dimensional joint distribution associated with internal representations and obtain high resolution maps of perceptual spaces.

2. Understanding the Universality and Diversity of Human Perception

Traditional psychology experiments recruit participants who have access to computer technology and are located in industrialized countries such as the US and India. This sampling constraint severely limits our understanding of the roles of nature and nurture in human perception, as the similarities we find between participants may stem either from universal biological mechanisms or from comparable exposure. To overcome this limitation, we apply computational methods and analysis to data obtained in field research with diverse populations around the world. We also focus on developing new infrastructures for massive online experiments.

3. The Data Science of Cultural Artifacts

Simulated cultural evolution experiments give rise to predictions regarding the connections between social structure and the emergent features of cultural artifacts which can then be tested by a comparison with real-world data. This branch of the research group studies the emergence of social structure in both real and virtual worlds which iteratively interact with the behavior of experimental subjects.