New Analytics of Culture

Organizational Culture. The New Analytics of Culture What email, Slack, and Glassdoor reveal about your organization by

Matthew Corritore, Amir Goldberg, and Sameer B. Srivastava

From the Magazine (January–February 2020)

Jean-Pierre Attal/Courtesy of Galerie Olivier Waltman

A business’s culture can catalyze or undermine success. Yet the tools

available for measuring it—namely, employee surveys and

questionnaires—have significant shortcomings. Employee self-reports

are often unreliable. The values and beliefs that people say are important

to them, for example, are often not reflected in how they actually

behave. Moreover, surveys provide static, or at best episodic, snapshots

of organizations that are constantly evolving. And they’re limited by

researchers’ tendency to assume that distinctive and idiosyncratic

cultures can be neatly categorized into a few common types.

Our research focuses on a new method for assessing and measuring

organizational culture. We used big-data processing to mine the

ubiquitous “digital traces” of culture in electronic communications, such

as emails, Slack messages, and Glassdoor reviews. By studying the

language employees use in these communications, we can measure how

culture actually influences their thoughts and behavior at work.

In one study, two of us partnered with a midsize technology company to

assess the degree of cultural fit between employees and their colleagues

on the basis of similarity of linguistic style expressed in internal email

messages. In a separate study, two of us analyzed the content of Slack

messages exchanged among members of nearly 120 software

development teams. We examined the diversity of thoughts, ideas, and

meaning expressed by team members and then measured whether it was

beneficial or detrimental to team performance. We also partnered with

employer-review website Glassdoor to analyze how employees talk

about their organizations’ culture in anonymous reviews to examine the

effects of cultural diversity on organizational efficiency and innovation.

Jean-Pierre Attal/Courtesy of Galerie Olivier Waltman

The explosion of digital trace data such as emails and Slack

communications—together with the availability of computational

methods that are faster, cheaper, and easier to use—has ushered in a

new scientific approach to measuring culture. Our computational-

lingustics approach is challenging prevailing assumptions in the field of

people analytics and revealing novel insights about how managers can

harness culture as a strategic resource. We believe that with appropriate

measures to safeguard employee privacy and minimize algorithmic bias

it holds great promise as a tool for managers grappling with culture

issues in their firms.

The Studies

Our recent studies have focused on cultural fit versus adaptability, the

pros and cons of fitting in, cognitive diversity, and the effects of

diversity on organizational performance. Let’s look at each in detail.

Fit versus adaptability.

When managers think about hiring for cultural fit, they focus almost

exclusively on whether candidates reflect the values, norms, and

behaviors of the team or organization as it currently exists. They often

fail to consider cultural adaptability—the ability to rapidly learn and

conform to organizational cultural norms as they change over time. In a

recent study two of us conducted with Stanford’s V. Govind Manian and

Christopher Potts, we analyzed how cultural fit and cultural adaptability

affected individual performance at a high-tech company by comparing

linguistic styles expressed in more than 10 million internal email

messages exchanged over five years among 601 employees. For

example, we looked at the extent to which an employee used swear

words when communicating with colleagues who themselves cursed

frequently or used personal pronouns (“we” or “I”) that matched those

used by her peer group. We also tracked how employees adapted to

their peers’ cultural conventions over time.

We found, as expected, that a high level of cultural fit led to more

promotions, more-favorable performance evaluations, higher bonuses,

and fewer involuntary departures. Cultural adaptability, however,

turned out to be even more important for success. Employees who could

quickly adapt to cultural norms as they changed over time were more

successful than employees who exhibited high cultural fit when first

hired. These cultural “adapters” were better able to maintain fit when

cultural norms changed or evolved, which is common in organizations

operating in fast-moving, dynamic environments.

These results suggest that the process of cultural alignment does not end

at the point of hire. Indeed, our study also found that employees

followed distinct enculturation trajectories—at certain times in their

tenure demonstrating more cultural fit with colleagues and at other

times less. Most eventually adapted to the behavioral norms of their

peers, and those who stayed at their company exhibited increasing

cultural fit over time. Employees who were eventually terminated were

those who had been unable to adapt to the culture. Employees who left

voluntarily were the most fascinating: They quickly adapted culturally

early in their tenures but drifted out of step later on and were likely to

leave the firm once they became cultural outsiders.

To further assess how cultural fit and adaptability affect performance,

Berkeley’s Jennifer Chatman and Richard Lu and two of us surveyed

employees at the same high-tech company to measure value congruence

(the extent to which employees’ core values and beliefs about a desirable

workplace fit with their peers) and perceptual congruence (how well

employees can read the “cultural code” by accurately reporting the

values held by peers). We found that value congruence is predictive of

retention—employees with it are less likely to voluntarily leave the

company—but is unrelated to job performance. We found that the

opposite is true of perceptual congruence: It is predictive of higher job

performance but unrelated to retention. These results suggest that

companies striving to foster a stable and committed workforce should

focus on hiring candidates who share similar values with current

employees. Employers needing people who can quickly assimilate and

be productive should pay greater attention to candidates who

demonstrate the ability to adapt to new cultural contexts.

The benefits of not fitting in.

When might it better to hire a cultural misfit? People who see the world

differently and have diverse ideas and perspectives often bring creativity

and innovation to an organization. But because of their outsider status,

they may struggle to have their ideas recognized by colleagues as

legitimate. In a recent study two of us conducted with V. Govind

Manian, Christopher Potts, and William Monroe, we compared

employees’ levels of cultural fit with the extent to which they served as a

bridge between otherwise disconnected groups in the firm’s internal

communication network. For instance, an employee might have

connections with colleagues that bridge both the engineering and sales

departments, allowing her to access and pass on a greater variety of

information and ideas.

Consistent with prior work, we found that cultural fit was, on average,

positively associated with career success. The benefits of fitting in

culturally were especially great for individuals who served as network

bridges. When traversing the boundary between engineering and sales,

for example, they could hold their own in technical banter with the

former and in customer-oriented discourse with the latter. People who

attempted to span boundaries but could not display cultural

ambidexterity were especially penalized: They were seen as both cultural

outsiders and social outsiders without clear membership in any

particular social clique. However, we also identified a set of individuals

who benefited from being cultural misfits: those who did not have

networks spanning disparate groups but instead had strong connections

within a defined social clique. By building trusting social bonds with

colleagues, they were able to overcome their outsider status and

leverage their distinctiveness. These results suggest that an effective

hiring strategy should strive for a portfolio of both conformists—or at

least those who can rapidly adapt to a company’s changing culture—and

cultural misfits.

Cognitive diversity.

Proponents of cultural diversity in teams presume that it leads to

cognitive diversity; that is, diversity in thoughts and ideas. But the

findings about whether cognitive diversity helps or hinders team

performance are inconclusive. Part of the problem is that these studies

use imperfect proxies for cognitive diversity, such as diversity in

demographics, personalities, or self-reported beliefs and values.

Moreover, this line of research has rarely looked at how diversity is

actually expressed in communications and interactions, which is

problematic given that team members are sometimes reluctant to share

their real feelings and opinions. Finally, cognitive diversity is often

assumed to be static, even though we know team dynamics frequently

change over a project’s life cycle.

In a new study, which two of us conducted with Stanford researchers

Katharina Lix and Melissa Valentine, we overcame these challenges by

analyzing the content of Slack messages exchanged among team

members of 117 remote software-development teams. We identified

instances when team members discussing similar topics used diverse

meanings, perspectives, and styles, and then analyzed the impact of that

diversity on performance. For example, in discussions of customer

requirements, different interpretations of the desired look and feel of the

user interface in some cases led developers to talk past one another and

fail to coordinate but in other cases sparked creative new ideas.

Our results indicate that the performance consequences of cognitive

diversity vary as a function of project milestone stages. In the early

stages, when the team is defining the problem at hand, diversity lowers

the chances of successfully meeting milestones. During middle stages,

when the team is most likely to be engaged in ideation, diversity

increases the likelihood of team success. Diversity becomes an obstacle

again toward the end of a project, when the team is deep into execution.

Cultural diversity and the organization as a whole.

We’ve seen that there are trade-offs associated with diversity in teams,

but how does it affect the performance of entire organizations?

Conventional wisdom holds that firms must choose between a

homogeneous, efficient culture and a diverse, innovative culture. A

homogeneous culture improves efficiency and coordination, the theory

goes, because employees agree about the norms and beliefs guiding

work, but the benefits come at the expense of fewer novel ideas about

how to accomplish tasks. In contrast, a heterogeneous culture sacrifices

the benefits of consensus in favor of healthy disagreement among

employees that can promote adaptability and innovation. The evidence

supporting this thinking, however, is scant and inconclusive.

In a recent study, we analyzed the language that employees used when

describing their organization’s culture (for example, “our culture is

collaborative,” “our culture is entrepreneurial,” and so on) in

anonymous reviews of nearly 500 publicly traded companies on

Glassdoor. We first measured the level of interpersonal cultural

diversity, or disagreement among employees about the norms and

beliefs characterizing the organization. We found that interpersonal

cultural diversity makes it difficult for employees to coordinate with one

another and reduces the organization’s efficiency as measured by return

on assets.

We then measured the organizations’ level of intrapersonal cultural

diversity. Those with high intrapersonal cultural diversity had

employees with a large number of cultural ideas and beliefs about how

to accomplish tasks within the company (measured as the average

number of cultural topics that employees discussed in their Glassdoor

reviews). For instance, employees at Netflix conceptualized the work

culture in terms of autonomy, responsibility, collaboration, and intense

internal competition. We found that organizations with greater

intrapersonal cultural diversity had higher market valuations and

produced more and higher-quality intellectual property via patenting,

evidence that their employees’ diverse ideas about how to do work led

them to be more creative and innovative.

Jean-Pierre Attal/Courtesy of Galerie Olivier Waltman

About the art: In his project Cells, photographer Jean-Pierre Attal explores the social urban archaeology of modern office towers, revealing the recurrence of patterns and postures found inside.

This suggests that organizations may be able to resolve the assumed

trade-off between efficiency and innovation by encouraging diverse

cultural ideas while fostering agreement among employees about the

importance of a common set of organizational norms and beliefs. Again,

consider Netflix: Although “multicultural” employees contributed to the

company’s diverse culture and drove innovation, the culture was

nonetheless anchored by core shared beliefs, such as the importance of

radical transparency and accountability, which help employees

coordinate and work efficiently.

Implications for Practice

How can these findings inform leaders’ understanding of culture as a

tool for improving the performance of employees, teams, and the

broader organization?

First, managers can increase retention by hiring candidates whose core

values and beliefs about a desirable workplace align well with those of

current employees. However, too much emphasis on cultural fit can

stifle diversity and cause managers to overlook promising candidates

with unique perspectives. Hiring managers should look for candidates

who demonstrate cultural adaptability, as these employees may be better

able to adjust to the inevitable cultural changes that occur as

organizations navigate increasingly dynamic markets and an evolving

workforce.

Hiring managers should also not overlook cultural misfits. They can be

wellsprings of creativity and innovation. But to make sure they flourish

inside the organization, managers should consider assigning them to

roles in which they are likely to develop strong connections within

particular social groups. That’s because misfits need the trust and

support of colleagues to be seen as quirky innovators rather than

outlandish outsiders.

Second, leaders should be mindful that the expression of diverse

perspectives in teams needs to be managed. Cognitive diversity is

essential for generating novel, innovative solutions to complex

problems, especially during the planning and ideation phases of a

project. However, the expression of diverse perspectives can quickly

become a liability when the team needs to focus on execution and meet

looming deadlines. It is during these times that team members have to

unify around a common interpretation of the problem and come to

agreement about what needs to get done to solve it. Leaders must be

adept at switching back and forth, learning when and how to promote

the expression of divergent opinions and meanings and when to create a

context for convergence.

An important distinction is warranted here. The term “diversity” is

often used to connote variation in the demographic makeup of a firm’s

workforce. This has been particularly the case in recent years, as

companies have tackled pernicious problems such as the

underrepresentation of women and minorities in decision-making

positions in organizations. In our work, we use “cultural diversity” to

refer to variation in people’s beliefs and normative expectations,

irrespective of their demographic composition. As we pointed out

earlier, demographic and cultural diversity are related, but a

demographically homogenous group may be culturally diverse, and vice

versa. Our research on cultural diversity is relevant to but ultimately

independent of efforts to increase gender, race, and ethnic diversity in

firms.

Third, leaders should foster a culture that is diverse yet consensual in

order to promote both innovation and efficiency. Such a culture is

composed of multicultural employees who each subscribe to a variety of

norms and beliefs about how to do work. These diverse ideas help

employees excel at complex tasks, such as dreaming up the next

groundbreaking innovation. Managers should encourage employees to

experiment with different ways of working—extensive collaboration for

some tasks, for example, and intense competition for others. At the

same time, a culture should also be consensual in that employees agree

on a common set of cultural norms—shared understandings—that helps

them successfully coordinate with one another. Leaders can signal the

importance of these norms during onboarding and in everyday

interactions, just as leaders at Netflix do by rewarding employees for

sharing their mistakes with colleagues in order to promote beliefs about

the value of transparency.

A New Management Tool

Many of the tools we used in these studies are off-the-shelf products,

and there is great potential for managers to use them to help solve

practical challenges inside organizations. For instance, Stanford PhD

candidate Anjali Bhatt is working with two of us to demonstrate how

language-based culture measures can be used to anticipate the pain

points of postmerger integration. We are studying the merger of three

retail banks, and analysis of emails has revealed stark differences in the

rates of cultural assimilation among individuals. Such tools can be used

diagnostically to assess the cultural alignment between firms during

premerger due diligence, as well as prescriptively during integration to

identify where and how to focus managerial interventions.

Yet the accessibility of these tools also raises important ethical concerns.

In our work, we maintain strict employee confidentiality, meaning that

neither we nor the organization is able to link any employee to any

specific communication used in our studies. We also strongly advise

against using these tools to select, reward, or punish individual

employees and teams, for at least four reasons: Accurately predicting

individual and team performance is considerably more challenging than

estimating average effects for broad types of individuals and teams;

culture is only one of many factors influencing individual and team

performance in organizations; algorithmic predictions often create a

false sense of certainty in managers; and finally, giving any algorithm

undue weight can have unintended consequences—for instance,

exacerbating human biases that negatively affect women and members

of underrepresented social groups.

Algorithms make estimates, but it is ultimately humans’ responsibility to

make informed judgments using them. Managers must be vigilant about

keeping metadata anonymous and must regularly audit algorithmic

decision-making for bias to ensure that the use of language-based tools

does not have unintended adverse consequences on culture itself—for

instance, by breeding employee distrust.

These important ethical questions notwithstanding, we believe that

these tools will continue to generate insights that allow managers to

finally manage the culture as a strategic resource, and ultimately lead to

more culturally diverse and inclusive teams and organizations.

A version of this article appeared in the January–February 2020 issue of Harvard Business Review.

DISCLAIMER

QUICK LINK

WE ACCEPT

CONTACT US