This is part two (of presumably five?) of Pentland’s Social Physics. I’m roughly following the sections that the book lays out, in order.

• Part 0: Some immediate personal/social context, what I’m thinking about before reading.
• Part 1 was on the basic premise of the book
• This part is about the foundations of Social Physics (Social Learning, Idea Flow, Engagement)
• Part 3 will be about Organizations (a favorite topic on this blog!)
• Part 4 will be about “Data-Driven Cities” (a recent favorite topic on this blog!)
• Part 5 will be about Policy (mostly avoided on this blog…)

Each part may not follow on the heels of the previous. I reserve the right to get distracted by some other thing.

Finally, all of the math is in the back of the book. Smart choice, Sandy. (Although this sometimes obfuscates how new some of these conclusions are- they smell like social science consensus, making the book feel like a retread when the real innovation is about the method, the computation, that brought about the reliable result).

—

Finding Good Ideas

Pentland agrees that the “singular genius” construct is not useful for creating more good ideas/decisions. He instead suggests that seekers of different (good) ideas/views, who benefit from many disparate streams of thought, are more likely to find serendipitous connections. They can then socialize that new idea, bouncing off of people from various backgrounds, before publicly proclaiming that new idea. Science, art, and leadership are all about “developing a compelling story about the world and then deciding to test it against reality.”

So now we ought to make this process a little more concrete. This exploration of ideas through people “is fundamentally a search of one’s social network”, so “a good place to start […] is by investigating the role of social interactions in how we find new ideas and use them to make decisions.”

The most productive people are constantly developing and testing a new story, adding newly discovered ideas to the story and then trying it out on everyone they meet. Like sculpting raw clay into a beautiful statue, over time their story becomes more and more compelling. Finally they decide it is time to act on it, to bring it into the light and test it against reality. To these people, the practice of harvesting, winnowing, and sculpting with ideas feels like play. In fact, some of them call it “serious play” [note: link mine]

—

It is known that the “wisdom of crowds” is most effective when the humans involved are giving their own judgment without social influence (that is, all of the judgments are independent). On the other hand, social influence can allow for the apparently-best ideas to spread through a population in the right doses. Periods of “idea harvesting” and “expert evaluation” are needed for the smartest groups- there must be idea diversity and pragmatic tests in order to develop the best ideas.

 

eToro, Social Learning and Idea Flow

“But how in the world does one go about discovering sufficiently diverse ideas?”

A very brief summary of the “eToro” study can be found here. In the book, Pentland explores the spectrum of traders in this platform that allows for both normal trades and “social trades” (placing trades that copy another user’s single-trade, or follow that user’s trades automatically). On one end, there are isolated users who suffer from an impoverishment of opportunities, and on the other end there are users tightly engaged in feedback loops, sharing the same ideas with each other over and over again.

The ten million trading activities were plotted, “idea flow rate” vs “return on investment” (corrected for changes in the market). The arc is essentially an upside-down U, anchored by the antisocials on one side and the echo-chamber on the other (not sure if significant, but the echo chamber suffered less than the isolated traders).

When traders had the right balance of diversity of ideas in their social network, their return on investment increases 30 percent over individual traders.

[Possible objection: I wonder if these traders in the middle happened to have “the right” social network- elite, closed groups of some kind? I don’t know.]

The wisdom effect is not unexpected; there are hints of this in ape troops and small human groups; it is also seen in simulations of networks of computer learning algorithms and in mathematical models of social learning. What postdoctoral student Erez Shmueli, Yaniv, and I found is that a community of social learners spontaneously forms what is called a scale-free fractal network- one whose connections are systematically more diverse than merely random- and, in addition, that connections in the network change over time in this same scale-free fractal manner. As a pattern of connections between learners becomes optimal, the performance of the entire crowd improves dramatically. The result is a fractal dance of learning that spins ideas into wisdom.

[This quote is heavily end-noted. May return later.]

Pentland references the work of Robert Kelly at Carnegie Mellon. From elsewhere:

The use of social networks for learning is not a new idea. A 1985 study studying differentiators between average and star performers at Bell Labs gives us an early hint.  Bell Labs only hired smart people, so brains weren’t an issue. The study examined why some of the smart people delivered exceptional performance and why other smart people were stuck in average. Robert Kelley of Carnegie Mellon found that Bell’s star performers invested in establishing a diverse set of relationships with experts in related fields. Average performers, on the other hand, limited their relationships to those in similar roles. When they needed help, star performers reached into an expert network that helped them understand a problem from diverse perspectives, while average performers tapped into people who had the same limited line of site.

Idea flow depends on both social and individual learning. People make decisions on a combination of personal and social information. When the personal info is weak, they lean on social information (and this also gives them confidence in their action). This can lead to overconfidence and group-think, especially if apparently-independent sources are actually getting their information from within the same echo-chamber after all.

Pentland and Altshuler started providing incentives to nudge behaviors in the eToro network and observe the effects. “We were able to increase the profitability of all the social traders by more than 6 percent, thus doubling their profitability.”

In this example, our tuning worked to break up the echo chamber, reducing the re-circulation of currently popular strategies and giving new strategies a chance to catch on. By reducing the rate of idea flow to allow greater diversity, we moved the social network back into its sweet spot and raised average performance. Through managing idea flows, our tuning of the network turned average traders- often the losers in our current financial system- into winners.

Altshuter and Pentland have applied the same logic to news reporters’ sources and ad campaigns, and have founded a company (Athena Wisdom) to tune other financial and decision-making networks. Pentland summarizes that social learning, diversity, and contrarians are all critical to the health of the network.

One disturbing implication of these findings is that our hyperconnected world may be moving toward a state in which there is too much idea flow. In a world of echo chambers, fads and panics are the norm, and its much harder to make good decisions. This suggests we need to pay much more attention to where our ideas are coming from, and we should actively discount common opinions and keep track of the contrarian ideas. We can build software tools to help us do this automatically, but to do so we have to keep track of the provenance of ideas. Older systems such as copyrights were a first step toward keeping track od idea flow, but much more uniform and lightweight mechanisms are required. [He promises more on this later].

 

“The Building Blocks of Collective Intelligence”

“…Companies have different types of idea flow- and therefore different abilities to learn both from inside and outside their communities. In each case, I think that the causes of excitement, boredom, or craziness have more to do with how tightly coupled people are to each other or how deep the splits are between divisions that with specific management techniques or the nature of the company’s work. In other words, it is the rates of idea flow- or the barriers to idea flow- that we must understand if we are to work well together.”

“I believe that we can think of each stream of ideas as a swarm or collective intelligence, flowing through time, with all the humans in it learning from each other’s experiences in order to jointly discover the patterns of preferences and habits of action that best suit the surrounding physical and social environment.”

“This is counter to the way most Westerners understand themselves, which is as rational individuals, people who know what they want and who can decide for themselves what actions to take in order to accomplish their goals. Could it be that our preferences and methods of action, the very things that define rationality, come from our community as much as from within ourselves? Are we, by the definition of economists, as much collectively rational as we are individually rational?”

 

Habits, Preferences, and Curiosity

This is fairly standard fare for books about social science and complexity, but Pentland is drawing almost entirely from his own methods and studies, and also I haven’t left behind notes on the other books that are coming to mind, so I won’t skip this section entirely. Pentland refers to two of his living labs to answer this question, “Social Evolution” and “Friends and Family”.

• “Longitudinal studies of participants in the Framingham Heart Study have suggested that social interaction is important in the spreading of behaviors ranging from obesity to happiness.”
• “For most people only a few peers are also friends and instead most are only acquaintances with which we have relatively little interaction.” “What we found was that weight change showed a very strong association with exposure to peers who gained weight but not to those who lost weight.”
• Exposure to behavior examples that surrounded each individual dominated every other variable they looked at (personal weight gain by friends, gender, age, stress/happiness, or all combined) “The effect of exposure to the surrounding set of behavior examples was about as powerful as the effect of IQ on standardized test scores.”
• “It might be asked how we can know that exposure to the surrounding behaviors actually caused the idea flow; perhaps it is merely a correlation. The answer is in this experiment we could make quantitative, time-synchronized predictions, which make other noncausal explanations fairly implausible. Perhaps even more persuasively, we have also been able to use the connection between exposure and behavior to predict outcomes in different situations, and even to manipulate exposure in order to cause behavior changes.” There are also quantitative lab experiments supporting this.
• “Therefore, people seem to pick up at least some habits from exposure to those of peers (not just friends).”
• Finding from the Social Evolution data set: “More exposure to similar views made the students more extreme in their [political] views.” This doesn’t necessarily mean that their friends or the people they talk politics with were predictive of their voting behavior. The whole stream of information, the peer group, was predictive, though maybe not their friends. Just before a political debate or major event, liberals/conservatives withdraw from each others’ social groups a bit, perhaps to feel a bit more comfort. This “selective exposure” reinforces our political views even further. “As Nobel Laureate Daniel Kahneman might have put it, we can consciously reason about which flow of ideas we want to swim in, but then exposure to those ideas will work to shape our habits and beliefs subconsciously.”
• App-downloading behavior was tracked in a community of young families (supplemented by surveys and interpersonal interaction patterns). They could “examine the data to see if they were independent decision, decisions driven by advertising, or decisions driven by exposure” (that is, people who already downloaded it). “People with similar characteristics (eg. age, gender, religion, employment, etc.) tended to download similar apps. This similarity effect, however, only gave us 12% accuracy in prediction. In contrast, when we analyzed participants’ exposure through all channels of communication […] we could predict which app a person would download with four times more accuracy.” “The search for new ideas and information, like the formation of new habits, appears driven primarily by social exposure.”
• Online discovery (e.g. of songs) was also studied. User’s behavior could be “accurately explained by a simple statistical model of social influence.”
• “The app and music cases are different from the health habits or political preferences, however. In both cases we could make good predictions about what people sampled but not what they would actually use or purchase. The effect of social exposure was informational, guiding their search for new apps or new music, but not normative. In other words, the particular app or music sampled often did not turn into a habit.”
• Social network idea flow incentives, Pentland argues, are more powerful behavior change tools than traditional individual incentives.

 

Habits versus Beliefs

• Pentland invokes Kahneman and Herbert Simon, as he should be expected to. A quick image key to Kahneman’s Thinking, Fast and Slow for the uninitiated here. Huge, important ideas, and maybe I should come back and write more on it, because I’m always referencing it off-hand.
• “But what do fast and slow thinking have to do with stream of idea sand collective rationality? The answer is that the process of learning is different for these two ways of thinking, and they are also used differently in daily life; these differences matter when trying to understand how communities build a collective intelligence.”
• Factual beliefs: Single exposure from a trusted peer is sufficient to convert.
• Habitual behavior/preference/interests: requires several exposures within a short period of time. [There’s a “serendipity”-like phrase I’m trying to pull up… is it “confluence”?  Will edit as I incubate on what I’m thinking…]. “Multiple exposures showing that a new behavior has a good outcome (such as social approval) are needed before we are likely to pick up the habit as our own.”
• “[As] Herbert Simon put it, our rational, conscious thinking is the program that invokes the habits of action and take care of all of the details of daily life, just as computer programs have subroutines that handle frequently used computations.” [Pentland spends a little time summarizing the concept of bounded rationality.]

 

“How Can We All Work Together?”

Humans, like many of our cousin creatures, create complex patterns of signalling in our vocalizations, body posture, etc. Exactly how to signal is different among animals, but what is common is the structure of decision-making process: “cycles of signaling and recruitment, until a tipping point is reached when everyone in the group accepts that a consensus has been reached.”

Mildly obvious study: Information over Facebook did directly affect political behavior, but the effect was “disappointingly small.” Facebook users who saw the messages *and* the faces of friends who already voted greatly improved the message’s effectiveness. Face-to-face relationships are strongest in this case. “Each act of voting on average generated an additional three votes as this behavior spread through the real-world, face-to-face social network.” Strong ties are important.

The FunFit study: everyone was assigned two buddies. Some were high-interaction and others were only acquaintances. (In this closed group, everyone was both a behavior-change target and a buddy). The buddies are given small cash rewards for the behavior of their targets, to drive engagement. The FunFit study, again, was based on sensors out in “the wild”, not a laboratory. Gigabytes of contextual data was also being collected. This scheme “worked almost four times more efficiently than a traditional individual-incentive market approach.” For the buddies who had the most interactions with their assigned target (instead of a mere acquaintance), the incentive worked almost eight times better than the standard market approach. People also maintained higher levels of activity once the incentives were removed.

We focus on changing the connections between people rather than getting people individually to change their behavior.

People shown their electricity usage compared to their neighborhood were more affected than people shown their electricity usage compared to national or state averages. Along FunFit lines, Pentland et al suggested that buddies would be given gift points when a person saved energy. “This social network incentive caused electricity consumption to drop by 17 percent, twice the best result seen in earlier energy conservation campaigns and more than four times more effective than the typical energy reduction campaign. The stronger the tie, the better the effect.

In digital social networks, bursts of engagement is much more effective for network use than the same amount of engagement over time. “Until people see there is a rush to adopt a new behavior, most group members will be reluctant to go along.”

—

Pentland has a short section on Subjugation and Conflict: “In a recent paper in Science, May Lim, Richard Metzler, and Yaneer Bar-Yam showed that between-group violence is likely to happen when the communities are poorly integrated, when one group can dominate the other, and, in addition, when the political or geographic boundaries fail to match demographic borders.” Pentland briefly reviews how this information connects with the previous results he’s shared. Near people are individuals, far people are representatives of the Other group. Politicians and Lawyers as a class are slimy, but my politicians/lawyers are swell people (same with racial groups, etc.)

Engagement requires interaction, cooperation, and trust (the expectation of future cooperation).

 

Final Thoughts

Some social scientists may ask what the fuss is about. Aren’t the experiments in the last three chapters just highlighting effects we already know about, such as homophily […] and social learning […]? Yes, but no one has really followed through on the computational effects of these well-known human behavior patterns: how these patterns of communicaton affect individual decision making and the fitness of the community. I have shown that these social universals produce a major increase in the collective intelligence of the community to act in a coordinated manner. Further, as we will see in the following sections of the book, these computational effects are central to the functioning of companies, cities, and society overall.

 

More later, on “Organizations”.