Welcome, all of you, to this symposium. We’ve been planning this for well over a year and it’s really exciting to have this all come together and to see you all here today. And I’m really grateful to be able to be the first speaker to kick off this set of talks that we’ll be having over the next few days. So, today I’m gonna be talking about communicating through color. And the initial question I would like to pose is why a symposium on color in the first place. It turns out, color’s a prime area for interdisciplinary discourse and a visual representation of what we just heard. We bring together lots and lots of different departments for this symposium. So, we’ve got history, English, literary studies, computer science, psychology, physics, and so on. And so the question is why is it the case that color is this prime area for interdisciplinary discourse? Why does color bring us all together?

I’d like to suggest there’s two main reasons, at least from my perspective, why color brings us together. The first is that colors activate a rich network of concepts in people’s minds, and these concepts are highly flexible and they can change, depending on the context. So, for example, redness is associated with the University of Wisconsin, but it can also signal chemical processes in beets, it can tell us to stop at an intersection, it can tell us about properties of cells in imaging, it can also tell us that a storm is coming. So, concepts are activated by both perceptual experiences, as well as verbal descriptions of color. Given that it can be activated by perceptual experiences, this brings together visually-oriented disciplines and given that concepts can be activated by verbal descriptions, this brings together verbally oriented disciplines. So, this brings together a lot of differed disciplines together then. Combining these points, color both perceived and described is a catalyst for a thought, abstract thought, across the arts, sciences, and humanities.

Okay, a second reason why I think color brings us together is that color arises from sources that spans the art sciences and humanities. And I use the word “arises” very loosely because I think we all would have very different definitions, or many of us would have different definitions of what color actually is across our different disciplines. But I think we can probably mostly agree on the following, which is that there’s physical properties in the world that predispose us to see colors. There are biological systems that process this physical input. There are psychological processes in the mind that integrate this input with prior knowledge. And then there’s things in the world, there’s things that create physical properties and some of those things include humans, which are probably gonna be influenced by these psychological processes. So, when humans create art, design, imaging, and data visualization, they’re creating things in the world. And there’s also cephalopods, which change color with camouflage. Stars, plants, geological structures, all of these are things that create physical properties in the world that allow us to see color.

So, if you highlight each one of these areas, we’ve got physics, biology, psychology, art, design, imaging, data visualization, cephalopods, stars, plants, and geological structures. All of these things are involved in what we think of as color. And so, I think that’s another reason why color brings us all together. So, color, I think, helps us answer basic questions in our respective fields, but we also need these fields to answer basic questions about color. So, for the remainder of this talk, I’m going to talk from the perspective of my field in psychology about the questions I’m interested in and how we go about answering these questions. So, to begin, let’s start with an exercise.

So take a moment to look at these crayons and think about which color you like the best. Okay, so now raise your hand if it was one of these colors. Look around, see if there’s a lot of you. Okay, raise your hand if it was one of these. Not many, and what about one of these. Okay, a little bit fewer. Most of them in the bluish range, so we’ll come back to that in a moment. So, now we’re gonna ask a different question. Which college scholarship fund would you donate to? Dennison College or Wilkes College. Raise your hand if it’s Dennison, so the one on the left. Raise your hand if it’s Wilkes, the one on the right. This one’s a little more distributed, I’ve never did this in public before, that didn’t work so well but that’s okay, I’ll still make the point with data in a little bit. We’ll come back to that. (laughing)

Okay, one more, so which bin is for recycling glass? So, here we’ve got a bunch of bins. You’ve got a bunch of objects, take a moment to think about which object would go in which bin, and I’m gonna ask you about a couple of them. So, think about which one would be the right one for a glass. Raise your hand if you think it’s this one. Really, someone from my, okay. Raise your hand if you think it’s this one. A lot more of you! (laughing) So, it turns out that is what people generally think, except for a person from my lab who knows what these data are and he said the different one anyway. So, these might seem like completely unrelated judgements. The first one was about evaluating colors, how much you like the colors. The other one was about evaluating colleges, which college you would donate to a scholarship fund. And the third one was about throwing things away. So, this is about interpreting color coding systems. So, although these may seem like unrelated judgments, I think there’s an underlying commonality and they all involve visual reasoning. So, you use visual reasoning to approach each one of these questions. And I define this as making conceptual inferences from visual information.

The visual information in these cases is color and we’ll talk more about what the kinds of conceptual inferences you might have been making for these different kinds of tasks. So, this brings us to the general question that we study in my visual reasoning lab here, at the University of Wisconsin in the Psychology Department and the Wisconsin Institute for Discovery. And our main question is how do people infer meanings from colors and how do those inferences influence people’s judgements about the world? So, to address this question, we’ve been developing what we call the Color Inference Framework to try to answer it. The idea of the Color Inference Framework is that this is a framework for how people form these conceptual inferences from colors, which we call color inferences. And how those inferences influence judgements about the world. The framework is predicated on the notion that people form color concept associations from their experiences in the world. So, what do I mean by this? Well, let’s actually start with a perceptual color space. So, in a perceptual color space, there are three dimensions, as most of you probably know. Hue, which is the circular dimension, like red, orange, and yellow. Lightness, which goes from the bottom to the top, so colors get lighter as they go higher in the space.

And chroma, which goes from the central spine outwards, so colors get more pure and vivid as they go further outward. So, here’s a perceptual color space, and this space represents all the colors that we can perceive. These are the samplings of the space but internal of the space represents every color we can perceive. Now, we can use this representation to talk about color concept associations by building color concept association spaces. So, here we have a color space like before, but now we have a weight on each point in this space that represents how strongly that color is associated with that particular concept. So, here’s a hypothetical color concept association space for bananas. So, there’s a lot of weight on the yellows, and the yellow greens, and browns. There’s very, very little to zero weight on reds and blues, which is why you don’t even see them in the diagram because they’re teeny tiny little orbs in there. This is a color concept association space for ocean. So, here there’s a lot of weight around the blues and blue greens, not much around the yellows, which we don’t generally associate oceans with yellows. These are relatively peaky spaces, meaning that a lot of the weight is concentrated on one particular part of this space.

But other concepts would have much more broadly distributed spaces. So, for example, tropical birds. Tropical birds can be associated with lots and lots of different colors. So, you’re gonna have weight distributed across this space. Now, the idea is that these spaces are continually updated in the mind with new experiences in the world. So, as we move about the world and experience new color concept associations, it can manipulate the weights in this space. For example, a young child who only knows about ripe bananas might only have weight on this very saturated colors, but then as she learns about rotten bananas and unripened bananas, her color association space will broaden. She also might make new spaces for ripe bananas and rotten bananas that are different from the overall banana space. So, with these color concept association spaces that we’re constantly building and updating as we move about the world, we can construct a color concept association network. And what this does is it’s a network that represents the association strength between all colors, in this case, colors that are represented by squares, and all concepts, which are represented by circles. So, the idea is that all colors we can perceive and all concepts we know about are linked together through this color concept association network. For example, a particular red might be associated with the University of Wisconsin, roses, strawberry, and foot with a sore on it. That looks really gross there, it’s supposed to.

There are other things in this space that will be really far away. So, bananas will be really far away from red, so it has very, very weak to zero weight on the associations with the colors that are near red. And our tropical ocean from before also is very far away in this space. But the idea is that this space connects all these colors and concepts. Now, going back to the framework. The idea is that people can form conceptual inferences from colors, or these color inferences, which influence their judgements about the world. And what I’d like to suggest is they do this by using this color concept association network and doing different operations on this same network that leads to different kinds of inferences and different kinds of judgements. So, for example, a first type of operation would be a pooling operation. So, here the idea is that each one of the objects that’s associated with a color, we know things about them.

We have valences for them, so most of us probably have a positive valence for the University of Wisconsin and maybe for roses, unless you’re allergic to them, and probably a negative valence for the gross looking foot. The idea then is that these valences get pooled onto the color and we get a summary statistic of these valences that is represented by that color. When you go to make a judgment about the color, it’s informed by that summary. So, if I ask you how much you like the color, you’re making an inference about all the objects, the valences of the objects that are associated with that color that get pulled onto the color and that leads to your judgment about the color. So, that’s our first kind of operation. Our second kind of operation is transmitting. This is kind of similar to pulling but instead of these valences just getting transferred onto the color, they get transmitted to some other object that is associated with that same color. So, let’s add a new object, this little greeble here, you probably don’t know much about this thing. And the idea is that your valences for these objects gets transmitted through the color as a conduit onto this novel object. And so, the judgment here is about the concept. You judge how much you liked this greeble, so if I ask you how much you like, based on how much you like the things that are associated with that greeble. And colors simply act as a conduit in this case. We have valences from objects transferring to other objects just based on their shared color association.

The third type of inference process is assigning. And this one’s a little bit different. So, imagine you’ve got some colors and you’ve got some concepts, and you’re trying to figure out what the mapping is between colors and concepts in a color coding system, like that recycling example I gave you earlier. The idea is that we figure out what the best assignments are given the association strengths between the colors and the concepts. And these lead to judgments about color concept mapping. If I ask you which one of these bins is for paper and which one is for trash, you’ll be doing this assignment inference process to figure out what that mapping is. Okay, so to recap our operations, we’ve got pooling, which leads to judgements about colors, transmitting, which leads to judgements about concepts, and assigning, which leads to judgements about color concept mappings. Starting with pooling, so the question here is what determines people’s color preferences? Why do people have color preferences in the first place? And why do people like some colors more than others? And before I started working on this problem, there were a few previous accounts.

One is that color preferences are hard-wired in the visual system. We have these weights on the cone visual system that just somehow give rise to our color preferences. But this account doesn’t tell us why we have the weights that we have or why this would be hard wired in the way that it is. Another previous account is that color preferences can be predicted by color emotions. And color emotions are these dimensions like active-passive, cool-warm, heavy-light. And it’s been found that people like colors that are more active, cool, and light. But it’s not clear why. Why would people like colors that are more active than passive, more cool than warm, and light than heavy. It’s also not clear that these are emotions. So, these are the previous accounts of colors preferences that were there prior to the work that we had done. But back in 2010, Steve Palmer and I developed the Ecological Valence Theory as a different account for where color preferences come from.

And the idea is that you like colors that remind you of things you like, like clear sky, and ripe fruit, and rolling hills, and sunsets. And you dislike colors that remind you things you dislike, like the sewage water, rotting fruit, and that’s mold, and I spared you the picture of the vomit. So, your preference for a given color then is determined by how much you like all of the objects that you associate with that color. How can we test a theory like this? In our initial study, we posed the question, can we predict color preferences based on how much people like the objects that are associated with those colors? And we did this by testing four different groups of people. So, the first group was presented with colors like these and they were asked to write how much they like each color on a scale from not at all to very much by sliding a cursor and clicking to record their response. A second group of people was presented with those same colors but their task was to write down all of the objects that they associated with those colors.

A third group was given the object descriptions that were provided by the second group, that was without colors, just the object description, and their task was to write how positive versus negative the objects were that were associated with each color. And finally, a fourth group was presented with a color along with the objects that were described as associated with that color and they rated how well the color on the screen matched the colors of the objects. And we did this to be able to down weight the effects of poor matches and uprate the effects of good matches. So, with these data, we were able to calculate what we call a Weighted Affective Valence Estimate, and what this is, is the single number that represents how much people like all of the objects that are associated with a particular color weighted by how good the match is. So, for each color, we have how much group one likes those colors and then we have these weighted affected valence estimates, which represent how positive the objects are that are associated with the colors.

The question is, do these waves predict people’s color preferences? So, at first we’re gonna look at color preferences. So, here on the y-axis, we have mean preference ratings and on the x-axis, we have hues, so, red, orange, yellow, H is chartreuse, or yellowish green, green, cyan, blue, and purple. And here are what the data looked like, and these different lines are for different saturation of lightness levels. And there’s two main patterns to take from this. One is that there’s this peak around blue. Remember, most of you raised your hand for the bluish colors. And in a trough around the yellowish green. There’s also a deep dip around dark yellow. So, if you remember what things were associated with dark yellow in the slide of pictures, we had our rotting food and not picture of vomit. So, now let’s look at the waves. Now, remember, the waves are measures of how much people like objects associated with the colors. The people who gave these data did not judge their color preferences at all. And these are what the waves look like. So, we have this peak around blue and the trough around yellowish green, and the deep dip around dark yellow, along with other commonalities between them. And these two data sets are highly correlated. So, what this tells us is that we can indeed predict how much people like colors based on how much people like objects that are associated with those colors. So, this is evidence for this pooling idea. That your preference for color is influenced by how much you like all these objects that are associated with the colors. And I’m using the word influence but in this study, this is totally correlation. So, we just find that there’s this correlation between color preferences and object preferences.

Can we provide evidence for a causal relation where our color preferences actually caused by preferences for objects that are associated with those colors? And we have several studies that suggest that that is the case but I’m gonna highlight one in particular. So, what we did is we had people come into the lab and we had them judge how much they liked each color, just like the previous participants did, but then we divided them into two different groups. So, one group then saw positive red images, like these bright raspberries and negative green images, that’s pond scum, and neutral other color images. And the other group saw negative red images, positive green images, and the same neutral other color images. And when they saw these images, they saw them multiple times and they did different kinds of tasks with them. So, they were asked to click on the center to say how complex they were, to judge how much they liked the objects in the images, and critically a different experimenter administered these tasks. So, we presented to participants as there was this color preference they needed first and then there’s this other spatial set of tasks that you’re gonna do. And then the original color preference experimenter came back in and said, “Hey, can I get some more color preference judgements from you?” And the participants would say sure, and they did another round of color preference judgements.

So, this really did feel like two separate sets of experiments. The participants generally did not know we were trying to manipulate their color preferences and we debriefed them at the end and if they had any idea that we were trying to, we excluded them from the analysis but this was very few participants. So, the question is can we actually change people’s color preferences by activating these associations in their color concept association network to influence their color preferences. So, here we’re looking at change in color preference on the y-axis where positive means that people like the color more after seeing the images. So, people who saw positive red images and negative green images had a significant increase in their preference for red and a minor decrease in their preference for green. And people who saw positive green images and negative red images had the opposite pattern. The controller neutral colors were in between. So, what this tells us is that color preferences can be changed by experiences with positive and negative-colored objects. So, by priming objects in people’s minds, we can selectively manipulate preferences for particular colors. We also, very recently, have neural evidence that color object associations are involved in color preferences, but I’m not gonna say any more about that because you’ll hear from Chris at 9:45 about this very new data.

Okay, so that was pooling and so now we’re gonna talk just a little bit about transmitting and assigning. So, for transmitting. The question is does preference for a concept that’s associated with particular colors transmit to concepts that are associated with similar colors? We address this question in the 2014’s Super Bowl. So, during the 2014 Super Bowl, the Seahawks, which were blue, green, gray and white were playing the Broncos, which were blue, orange, red, and white. And the question was does the team you’re rooting for during the Super Bowl– and we collected these data online during the Super Bowl– so people were watching and we know that they were watching because they told us they were. Does your preference for these teams generalize, or transmit to teams of division three colleges where you don’t know much of anything about them? So, pilot data suggests that people don’t know much about Endicott College or Utica College.

And so the question was does your Super Bowl-preferred team, your preference for that team, spread to these division three colleges? So, to test this idea, we had our two Division 3 colleges that were matched to the Super Bowl teams, as well as other colleges that were matched to other NFL teams, and we presented people with trials like this. So, we said, imagine there are two minutes left in the game. Who do you hope would win, in this case, Endicott College or Utica College? But they saw all pair-wise combinations of these teams. And afterwards, we gave them a football questionnaire. So, how much do you like the Seahawks and how much do you like the Broncos? And the question is can we predict their choices from this kind of bizarre task where they don’t know about these colleges based on who they’re rooting for in the Super Bowl. We also did a similar thing in 2015 but instead of asking about which college team they hoped would win in a Super Bowl, we asked which college they would donate to their scholarship fund. So, then we tried to move a little bit away from the football context and we found similar effects there. So, the question then is can we actually predict which college they will choose, and we can. So, here we’re looking at NFL difference scores. So, positive means that people like Seahawks more and negative means they like Broncos more. And the points are gonna be individual participant’s choices.

So here, people who liked the Seahawks more were more likely to choose the Endicott College and people who liked the Broncos more were more likely to choose Utica College. Even though they don’t know much about these colleges at all. So, the idea is that your valence for a particular concept is spreading to some other one through color as a conduit. Now, this could have really serious implications for the kind of judgements we make in the world. For example, in college admissions. If you’re an admissions officer at Yale and your rival is Harvard, are you gonna be more biased to favor college applicants who have letterhead from Exeter College, which is blue, which matches your school’s colors, as opposed to St. Paul, which is red, which matches the rival, Harvard’s, colors? So, there’s a lot of cases where our associations with colors might actually be influencing our judgements in ways we don’t realize because color is not relevant to the particular judgment. We’re activating this color concept association network that’s implicitly influencing our judgements. So, this is a question we’re following up with in the lab. So finally, assigning, which gives us judgements about color concept mappings. So this is especially relevant when we’re interpreting color coding systems, and we do this all the time.

For example, when you’re in an airport and you’ve got different colored terminals, you have to figure out which terminal maps onto which color. When you’re reading weather maps, and when you’re trying to figure out how to throw things away in different kinds of recycling bins. And it turns out that in different parts of the world and even different parts of the same campus, you’ll see different color coding systems for different kinds of bins. The question is what do these colors mean in these contexts? More specifically, how do colors map onto concepts? And you might be thinking, well this is a stupid problem. Can’t we just rely on legends or labels to interpret what colors mean? If there’s some label that tells me what the color is, then I can just read it and I know. Well, it turns out that color coding systems are harder to interpret when the labels do not match people’s predictions. So, it’s been shown by Lin, that if you show people graphs of fictitious fruit sales, they’re much faster at interpreting the graphs if bananas are yellow, and blueberries are blue.

Then if bananas are orange and blueberries are green, that’s even hard to say, because they are incongruent. So, the question though is what are people’s predictions in these color coding systems? Now, this might seem obvious when you’re talking about fruit because we know that bananas are generally yellow and blueberries are generally blue. But what about when there’s abstract concepts that do not have directly observable colors? What are people’s predictions there? And what happens when you’ve got multiple concepts that are associated with the exact same colors? How do we figure out how to do the mapping so that people can interpret the systems? We’ve been working on what we call Assignment Color Inference and the idea is that people predict assignments that optimize the color concept association. So, they figure out in their minds, they do an inference process where they figure out what the best mapping is between colors and concepts, given the colors that they have and the concept that they have. And by understanding assignment inference, we can actually create color coding systems that are easier to interpret. We’ll hear a lot more about this later this afternoon from when he talks about Modeling Color Inference: A Study of Color-coding Systems in Recycling.

So, to conclude, I’ve suggested that a key reason that many of us are united by color is that we use color to infer and communicate abstract meaning in our different disciplines. And the color inference framework provides an account for how. It’s an account for how people infer meaning for colors and how those meanings influence people’s judgements about the world. So, I’d like to acknowledge my collaborators on this work, my lab and these wonderful people who have provided valuable feedback on this work and funding sources. And thank you for your attention. (applause)