Field Manual for the Learning Historian

Chapter One: Introduction to Learning History theory and practice

What is a Learning History?

In recent years, the idea of building a "learning organization" has gained currency in management circles. Many senior managers, in particular, have come to recognize that, with the right approach to collective learning, their enterprises can continually gain new talents and capabilities even as they weather the vicissitudes of fate. Managers in middle levels, meanwhile, have embraced the "learning organization" idea because it encourages people to follow their own aspirations and, in the process, boost organizational performance. This implies that people can reclaim a little bit of the spirit of community and personal involvement that has been leached out of conventional business decision-making.

But even the most fervent "learning organization" enthusiasts have difficulty demonstrating a link between organizational learning efforts and key business results. The leaders of all learning and change efforts in organizations sooner or later run up against the challenge of proving the value of their efforts' accomplishments. The same is true for other types of "change" and "transformation" efforts. Executives authorize millions of dollars for organizational learning, reengineering, re-invention, or quality improvement -- and then grapple unsuccessfully with the problem of assessing their investment.

Assessment is also vital for the participants in learning efforts. They need to judge the value of their past experience, if only to help their organizations move forward and to develop their judgment and skills further.

Moreover, the rest of the company also needs to understand the experience of its learning efforts to date. They will, after all, need to build upon that experience. How do they replicate the first successes, and avoid repeating the first mistakes? How do they spread the sense of potential achievement through the rest of the organization? How do they overcome the disdain for anything "not invented in our part of the company"? Companies have found it notoriously difficult to institutionalize the learning of its subgroups, to help the rest of the organization develop.

Finally, successful learning efforts generally require people to rise above their conventional blinders to add new ways of thinking and new forms of behavior to their repertoire. But these sorts of changes are misunderstood. They may be seen as evidence of cultishness, as window-dressing that isn't backed up by action, or as well-intentioned but misguided attempts at change. To really make sense of a learning effort, people throughout the organization need to see it through the various perspectives of people who have been involved with it firsthand ,so that they can come to terms with it based on actual data (not just on the gossip that reaches them), and make sense of it in a way that is credible to them.

In short, when an organization has been through a learning or change process, people throughout the organization need a feedback process that can provide guidance and support Yet reacting to the pressure of assessing learning can easily undermine any learning effort. As people become aware of being assessed and measured, the intrinsic motivation which drove them to learn is supplanted by an extrinsically motivated desire to look successful. Any feedback, mediated through an outside observer's eyes, will be tainted by this built-in set of distortions.

Learning histories were invented in response to these concerns and needs.

A "learning history" is a document,or a series of documents, possibly in audiovisual form ,that is disseminated in a deliberately structured manner. The document, and the dissemination, are both designed to help organizations become better aware of their own learning and change efforts.

The learning history presents the experiences and understandings of participants ,people who initiated, implemented and participated in organizational transformation efforts, or some collaborative learning experience ,as well as non-participants who were affected by these efforts.

The learning history tells the story in participants' own words, in a way that helps the rest of the organization move forward, without having to "re-invent" what a small group of learners have already discovered.

A learning history thus represents the organization talking to itself, in a safe and carefully structured way, about the things it needs to hear but hasn't yet listened to.

The history includes reports of actions and results. It shows readers how learning is an approach to get what they want, and it illustrates how others have achieved the results they wanted.

The history also includes descriptions of learning methods and techniques ,the intent, tools, and design of an intervention. The history tells the story of how people learned to collectively inquire in new ways, generate insights, and then take actions which weren't thought possible before.

Finally, the history includes descriptions of the underlying assumptions and reasoning that led to people's actions. In this way, the unwritten but powerful tacit knowledge and undiscussable myths are brought to surface, codified, and turned into a knowledge base. People can test their understandings against the perspective of others, without having to be in the same room at the same time. For this reason, we sometimes think of learning histories as dialogue, on a different time/space continuum.

The history includes the perspectives of a variety of people (including people who did not support the effort). No individual view, not even that of top managers, can encompass more than a fraction of what actually happens in a real organization, and this reality is reflected in the learning history. When participants discover that their own points of views are treated fairly in the learning history, they become better able to understand the many other people's perspectives that make up the learning effort.

Learning history work goes beyond writing a history that documents a project. It is a critical element in developing an organizational infrastructure to support learning. Its research, distillation and dissemination processes are designed to create new opportunities for organizational learning.

A learning history document becomes an artifact which is then used as a piece of directly observable data which becomes the basis for individuals, a team and an organization to share a common, collective history of what happened in the past, build on the learning of others, and have a new kind of conversation that helps them to move forward in their own learning process. The content of the learning history creates a context for a conversation that the organization wouldn't be able to hold otherwise.

Learning history practice provides a philosophical and methodological basis for addressing issues related to how we measure and assess organizational learning. The learning history draws upon established theories, techniques, and skills of action science intervention, oral history, anthropology, sociology, literature, and theater. The integration of these theories and techniques, using a philosophy consistent with organizational learning principles, is what makes learning histories unique.

The production of a learning history requires 15-90 person-days over the course of 3-6 months. The end result is a dynamic, interactive report that help the people of the organization reach a common understanding about their triumphs, their problems, and their efforts to change current reality .

The value of a learning history

Most learning history projects begin with three questions:

1. "How can we judge the success of our organizational change effort?"

2. "How can the rest of the organization benefit from this experience?"

3. "What do our efforts to date reveal about our opportunities for success and our potential for failure?"

We believe that most organizations already know what they need to hear in answer to these questions. But if they have not learned to listen, then history is destined to repeat itself.

These days, people in most organizations have been involved in change efforts, transformations, learning initiatives and innovative breakthroughs. They know the pitfalls that befell them, the value of their experiments and how the rest of the organization could benefit from their experience.

However, they lack a way to reflect on their story collaboratively, talk about it effectively, consider its implications, and communicate its "learnings" to others. This desire is expressed when managers say, "We need time to reflect," or "We need what we say commuicated to others," or, "We don't want to reinvent the wheel." How, they ask, do we get one part of an organization to learn from another? How do we keep from making the same mistakes over and over and over again

To instill organizational learning requires a deliberate attempt to institutionalize reflection in organizations as whole entities. Somehow, brilliant components and skeptical overseers must be given a channel for engaging each other thoughtfully and productively. This is particularly important as hierarchies devolve and managers try to institutionalize the organization's "strategic memory," so that anyone within a corporate body can draw upon the knowledge of the whole.

Reflection is rarely put into practice in business, because organizations are not equipped for it. For one thing, the time pressures of corporate life mean that there is no slack built in to the typical management process. Managers act collaboratively, but they lack the time to make sense collaboratively of their actions. Instead, they are continually pressed to skip directly into more action.

Moreover, reflection requires the difficult (and often counter-intuitive) task of building self-awareness. Most managers have little experience with inquiring openly into the successes and mistakes of the past. They face overwhelming temptation to cover up such inquiries, because there is generally an organizational norm, or an unspoken agreement: Confrontational issues (like mistakes) are not meant to be discussed. And, as Chris Argyris has noted, this unwillingness to discuss tough issues is, itself, undiscussable as well. A manager who tries to "swim upstream" and reflect collaboratively on past mistakes might be labeled a complainer; and even the attempt to protest the label will be discounted as complaining. The manager is placed in a terrible double bind.

In the absence of reflection, the organization looks elsewhere for its assessments. Outside consultants or "expert" staff members interview some employees and present back the results as a set of "answers." But this represents a tragically discouraging waste of effort and enthusiasm. The people of most large organizations already know what they need to hear without a cadre of experts to intermediate. Each member of the organization knows some aspect of the pitfalls that have befallen them, the ways in which the organization creates its own problems, the impacts of changed policies, and the means by which the enterprise could move forward into the future. They simply lack the opportunity to reflect on their experience together, and make sense out of the total experience of which each member holds a part.

"If there had not been a learning history at our company," said a participant at one project, "the learning effort would have stopped with the end of the pilot team. People would have dispersed and said 'It was good.' Or, 'It was bad.' There was data to support both points of view."

A learning history approach captures stories people tell about learning and change efforts and reflects them back to the organization and others.

Learning histories are labor-intensive, and can be expensive. A small project can be "captured" with two or three days' worth of interviewing, but a large corporate-wide project may require as many as 150 interviews. It might take 30-60 person-days to conduct those interviews, distill them, and present the results in a document and workshops. Including the time of internal people, the overall costs might stretch as high as $500,000.

This is significantly more expensive than a survey, but it should be seen in the context of a transformational effort that costs tens of millions of dollars. Without the learning history, there may be no effective way to judge the effectiveness of that much larger investment.

Furthermore, a learning history project provides several benefits that a survey does not:

  • It is a process, not just a product. The work does not end with the written document; indeed, the final dissemination effort is as carefully planned and implemented as any other part of the work. The learning history process establishes a vehicle for reflection in all of its stages, from the interviews to the final workshops.

  • Surveys tend to produce a set of "answers" that are endorsed (or not endorsed) by senior management. The learning history creates a reflective experience that encourages people to make up their own minds; it involves people throughout the company in thinking about the past in a way that helps plan for the future.

  • Surveys translate qualitative experiences into quantitative data. In doing so, they distort the experience and inevitably lose credibility and focus. The result: "We've measured everything but we haven't changed attitudes." The learning history provides a way of packaging and replicating the kinds of non-metric data that otherwise would be lost.

  • The learning history work trains a core group of internal people in the methods of reflective interviewing, distillation, and even writing, and brings them up to speed in an overview of the existing transformation process.

  • It starts a "buzz" going throughout the company, as people start to make sense of the new "mythic" stories that the learning history generates.

  • Learning histories make use of the skepticism that exists in organizations, by reproducing it faithfully in the context of both collective aspirations and current reality. "Dilbert's" voice is heard in the learning history; so is the voice of Dilbert's boss. This helps build judgment among both of them. "This may be one tool," said an internal learning historian at one company, "that keeps us from going to the next fad."

  • It creates a history of effective practices that go beyond "best practices:" to show not just what people did, but what they were thinking, what assumptions they had made, how they came to their decisions, what others thought about their actions, and how they expect to move forward from here. Instead of merely copying the best practices of others, people who read learning histories are now equipped to develop their own best practices.

  • The process builds actionable knowledge among organizations.

    Learning histories have existed, in the form developed here, for less than three years, but a body of "results" is beginning to emerge. At one corporation (AutoCo, a large automobile company), learning histories are credited with helping preserve the innovations of one key car launch team, so that other teams could build on their experience. In a large oil company, learning histories have become a critical component of the company's "roll-out," to help new business practices can be instilled into the workday of tens of thousands of people. A learning history of an educational institution has become a key component of the orientation of new people who join the institution's community; it is credited with providing the most credible understanding of the issues facing that institution. At a manufacturing facility, the opportunity to see their story told in learning history form has been a significant morale and involvement factor ,especially since the plant is being spun off from the parent company, and its managers and workers must develop their own organizational identity.

    This year, the first comprehensive learning history (AutoCo) was made available to the public, published by the MIT Center for Organizational Learning. It will be produced in book form next year. There is reason to believe that others will follow, and that the growing body of data will comprise a rich source of generalizable knowledge that spans organizational boundaries. "Learning histories can be to action science," Chris Argyris has said, "what a microscope is to the physical sciences."

    Anatomy of a learning history Project

    There are essentially seven steps in a learning history project. Each one has its own chapter in this Field Manual. They include: Planning, Research, Distillation, Writing, Validation, Dissemination, and Publication/Outreach.

    I. Planning: Determining the boundaries

    The planning stage delineates the range and scope of the project. It is conducted, typically, by a learning history advisory team, composed of the external team leaders and a group of "champions" within the company ,people who are willing to invest some effort in helping a learning history take root. This planning phase includes identifying "noticeable results:" tangible business outcomes that have already caught the attention of the organization. It also involves recruiting internal learning historians, and establishing the primary audience, audience, the scope of the inquiry, and the specific questions that must be addressed. This planning stage is not just a crucial logistic phase, but a key avenue of reflective participation.

    2. Reflective research: Interviews and data gathering

    The internal and external historians conduct reflective interview conversations with participants in the original learning effort (along with key outsiders, such as suppliers, contractors, and consultants), taking pains to gather perspective from every significant point of view. Between 50 and 200 people may be interviewed, depending on the scope of the inquiry. These interviews involve skill development for the internal interviewers, reflective opportunities for the interviewees, and information gathering for the entire project. Thus, the interviews build reflective capacity in the organization ,both for the learning historian and for the interviewee, who may never have had an opportunity to ruminate at work, at length, about his or her experience. This stage may also involve other forms of research, including observations, examination of documents, etc.

    3. Distillation: Establishing key themes and "plots"

    From the mass of data (interviews, observations, field notes, and documents), the internal and external learning historians cull meaningful themes, systemic understandings, and implications. This also builds analytic and synthesizing skills for internal learning historians. We have chosen the word "distill" carefully to convey the essence of this activity ,taking volumes of data from interviews, and then rectifying, purifying and refining the "raw data" into a form which the organization can hear. In this distillation effort, we balance our "research" imperative ,to keep our conclusions clearly rooted in the data ,with the "mythic" imperative ,to tell an archetypally moving story with the "pragmatic" imperative ,to tell the story in a way that it can be effectively read, heard and discussed in organizations.

    4. Writing: Production of a transitional object

    Next, the document must be produced. We base our writing format on the anthropological concept of the "jointly told tale." In this form of writing, the participants and the learning historians tell the story together, incorporating the participants' experience and passion, along with the learning historians' broader perspective and objective training.This approach places an enormous burden on the artistry of the editor. Every narrative tale has to be pared down to its compelling core, to draw an audience in, to be valid and representative, yet succinct and direct. The section on writing in this Field Manual describes the various techniques we use to fulfill our mythic, pragmatic and research imperatives in the written work.

    5. Validation: Reflective feedback

    In the validation stage, there are several sets of checks designed to reestablish validity. In quote-checking, participants see their quotes, make changes, and approve them, before anyone else sees them. Validation helps guarantee our protection of anonymity, and builds in another level of perspective as interviewees reconsider their statements.

    In addition, the learning historians conduct validation workshops with small groups of key participants (who have already been interviewed), along with a few other people from elsewhere in the organization. This allows the original participants to relive their experience in the company of others ,and to observe how it will be seen by the rest of the company.

    Finally, the learning history champions and team members go over the document in a manner designed to render it most useful to the organization.

    6. Dissemination: application and transferring learning

    The learning history manuscript is not handed out as a report; it would be simply put on a shelf and ignored. In carefully designed workshops, built around concepts of action research and learning transfer, people from every part of the organization read and discuss the document. How typical was this story? What pitfalls and resources exist for their own learning efforts? How can they use the Learning History's insights to increase their own capabilities? As comments are added, the report's story becomes part of the common understanding of the organization.

    7. Publication/Outreach

    Eventually, learning histories are packaged and presented for a wider external audience, with the organization's name disguised ,so that the organization itself, and the research community, can benefit from the building of knowledge about management and organizational change.

    Creating "reflectionable knowledge"

    A key goal of learning history work is to create the type of knowledge which inspires reflection, and which leads to meaningful and significant reflection.

    "Reflectionable knowledge" often exists in the form of stories. It provides a context which makes it easy to assimilate and think about new information. It makes explicit the multiple mental models which operate in a given social setting. The knowledge is expressed at different levels of abstraction - from observable data to interpretations, attributions, and generalizations in such a way that the communicator's thought processes are articulated.

    Reflectionable knowledge promotes further inquiry into these thought processes and into the differences between various participants' assumptions. Having reflected, we build a store of knowledge, so that when we return to reflect next time, we are less likely to reinvent the wheel or spin our wheels.

    The emphasis on "reflectionable knowledge" has led us to de-emphasize the document of the learning history. Our fundamental purpose is to create materials which promote a more effective form of conversation. The document itself is just a means to that end.

    Indeed, not just the document, but each element in a learning history process interviewing, observing, analyzing, writing, editing, circulating drafts, following up and conducting dissemination workshops is intended to broaden and deepen "reflectionable knowledge" throughout the organization. The learning history process provides an ongoing forum for collaborative reflection:

    Interviewing: The act of interviewing people and capturing their perspectives about the learning project is, itself, an intervention. Often people in organizations do not take the time to reflect on what they have done, or how they are going about a particular task. After a learning historian asks them to talk about their learning, either by themselves or with a partner, they often come away with a renewed depth of understanding. Later, when they see the report, they have an opportunity to compare other peoples' interpretations and experiences with their own.

    Document creation: In designing and writing the document, a team of learning historians assembles the mass of data it has collected, reads it carefully for patterns and themes, and decides on how to organize the material.

    This set of activities is also an intervention. By having interviewed multiple people ,those active in the project, those supportive but perhaps only bystanders, and those against the project the team of "learning historians" is in a position to understand and hold a broader and more diverse perspective than anyone else involved in the effort. That is one reason why insiders are always part of the learning history team: their growing capability benefits the organization.

    We advocate the "jointly-told tale" form for learning histories (as described in Chapter 4, "The Jointly-Told Tale"). This form provides an effective model, on paper, of the type of conversation which encourages reflection. It is a slower conversation, in which people can consider the "story" as told from a variety of distinct perspectives ,including their own.

    Dissemination: The learning history is a "means" to a better conversation. The printed learning history, even if it is incomplete or provisional, provides a foundation of observable data upon which conversations about the learning effort can take place. Discussing a learning history give non-participants an unprecedented collaborative view of what went on in the learning effort.

    In sessions held within the pilot team, team members can look at themselves and ask, "What was effective? What do we need to do differently?" When people jump to conclusions and generalizations, the question can be asked, "where in the learning history to that come from?" Thus the reasoning process, going from observable data to attributions and generalization, whether derived from data in the learning history or other aspects of a person's experience, is articulated through this conversation.

    In sessions held for other teams, organization members can ask themselves what similarities and differences exist; what they could do similarly or differently; and how much they could foster an environment of similar experimentation and learning.

    As people throughout the organization read and talk about the document, they become more interested in the project. Their interest leads them to inquire about their own methods, to learn new skills and tools associated with organizational learning, and to create new ways of working, which in turn lead to new results, and eventually, to new learning histories.

    In some cases, the learning history process offers the only institutionalized opportunity that a team, or an organization, has to reflect. The learning history process can be beneficial not only for the original participants, but also for researchers and consultants who advised them, and ultimately anyone who is interested in organizations' learning processes.

    Basic principles of learning history work

    We're coming to think that a number of generic principles apply to learning history work. These principles are presented here as a first take, not as the final word. As the methodology advances, we hope to continue refining these principles into a set of theories of organizational reflection.

    Organizations today have a choice - "Slash and burn" or "learn"

    Learning history work is inextricably linked with the premise that organizational learning is essential for managers faced with a turbulent environment. Organizations that can survive under an authoritarian regime have no need of learning histories or, for that matter, of change efforts to evaluate. But the experience of the past 30 years suggests that, in turbulent environments, authoritarian companies will not be able to sustain themselves.

    The alternative to command-and-control is collaborative learning ,the ability to expand an organization's capabilities in response to its own desired future and the state of current reality.

    Learning takes place from experience, but collective learning from experience is inherently problematic

    This is the basic dilemma of organizational learning. Individual experiences are inherently limited, by the biases of our own backgrounds and beliefs, and by the fact that we can only experience a narrow slice of time and space. If we confine ourselves to learning from experience, we will always be restricted to a narrow set of "learnings." This is problematic in organizations, where systems are spread over time and space, and may not seem to have much to do with each other on first glance. Learning histories are successful when they bridge individual experience, helping people draw common understanding from the syntheses of individual stories.

    Communication that fosters learning must embody research, mythic and pragmatic imperatives

    We're trying to deliberately balance three imperatives in our work: The pragmatic (telling the story so that managers can accept it and work with it), the research (telling a story which can be validated with the "data" of interviews and observations), and the mythic (telling a story which is powerful, compelling, and pure because it is true to the story's own needs.) No one can think all three ways at once. A communication creation process, to be effective, must cycle between these three imperatives.

    No one voice provides "the answer" - people accept other's viewpoints in the context of their own

    Perspective comes from many sources. No one outside expert or internal observer has the whole story. The enthusiasts, skeptics, and senior managers ,or, as systems-oriented therapist David Kantor puts it, the "initiators, opposers, followers, and bystanders" ,all have valid experiences. They all need to tell it together, in an explicit way so that the reader can see whose perspective is present in which part of the text. This gives the story validity. When people see that their point of view has been treated fairly, they recognize the credibility of the whole document.

    "You are not alone" - all particular instances are reflections of universal patterns

    "I know who's speaking," people often say about a paragraph in a learning history ,but they may, in fact, be wrong about the individual, the team, the organization, or even the country! Many of the patterns and systems in organizational life exist archetypally, independent of a particular culture or set of circumstances. By understanding these systemic interrelationships, people can better understand the hidden forces at play in their environment. As challenges and successes are drawn from the details in the learning history, a story that seemed unique and based on idiosyncratic personalities becomes universal ,and thus useful as a broadly applicable glimpse into the underlying systems at play.

    Organizations "know" what they need to hear but lack the capacity to listen

    Individuals throughout the organization, together, understand the "missing" information that the organizations need. Each individual has a piece of the puzzle, but as a whole, they lack systematic ways to combine their understanding together into a single story. Learning histories represent an alternative to calling in outside consultants to tell the organization what it already knows.

    Organizations need an established infrastructure for reflection.

    We believe that ad-hoc interventions will ultimately be limited. Learning histories seem to be most effective when tied in to existing infrastructures for reflection: Management training series, Learning centers, Team workshops, etc.

    Learning involves change, and change may be difficult.

    Learning Histories bring out difficult, tough stories that have been swept under the rug ,and try to do so in a way that the organization can hear. In our interviews, we feel it is crucial to get as many perspectives as possible on painful situations.

    Stories convey intangibles

    We once wrote a learning history about a team which developed a prototype engineering model. On the surface, the achievement was a "noticeable result:" a technical feat, which had paid off in financial savings, and probably duplicable as a "best practice."

    But to the engineers who put together the prototype, there was a remarkable story involved: A story of continually testing the waters, involving people, and learning to communicate in new ways. Some team members had to work differently with outside contractors (who were architects of the prototype), while others had to muster the courage to request an extra half-million-dollar budget. Still others learned to create an atmosphere of open inquiry, so that engineers could talk across functional boundaries, and make the prototype work.

    Until the stories of these half-dozen individuals were brought together, they were not aware of common causes or each others' contributions, and many others in the company were unaware of the entire process.

    Behind every "noticeable result," there is a story... And the story is more effective at conveying intangibles than any other form of communication...

    What is, and is not, a learning history

    In our work over the past three years, many different forms of "learning history" have been proposed, and quite a few of these have been tested. In addition, learning historians have been called upon to write a variety of different types of reports, some of which may or may not fit under the definition of "learning histories." Finally, a large number of intervenors and consultants have begun to include "learning history work" among their practice, and in many cases, their work fails to conform to our idea of what a "learning history" should be ,and yet, clients are enthusiastically appreciative.

    Why bother, then, to define a learning history? Why not let the term "learning history" simply refer to any reflective document that helps capture the "learning" from an organizational experience?

    Because we strongly believe, based on our experience and theory, that some types of reflective documents are more valuable in the long run than others.

    A number of processes and methodologies resemble learning histories in some respect: Project clinics, "lessons learned" reports, systems diagram reports, left-hand/right-hand columns from action science, "organizational memory" efforts, action reviews, and "reflective memoranda."

    All of these, however, lack the ability to reach the entire organization in a way that encourages reflection on the most significant aspects of the organization's experience.

    In developing learning histories, we are trying to move beyond the traditional "lessons learned" reports that people think of when they think of "learning histories." This standard engineering practice, archived in a library, may often lead to valuable technological cross-fertilization, and it may help prevent litigation or the repetition of old mistakes. It may even "capture a moment" in the history of an organization's learning.

    But such reports are all too often filed in desk drawers and forgotten. They rarely, if ever, lead to conversation about the cultural and interpersonal issues that lay underneath the success of a pilot team. They may spread technical and organizational-development ideas, but they don't allow a sense of "what really happened, underneath the report" to filter through an organization. Or if it does, it will do so by accident.

    We have set our sights on developing a methodology that will systematically help organizations get past the stumbling blocks that have prevented reflection in the past: The unwillingness to consider "bad news," the desire to shoot the messenger, the fragmented nature of decision making, and the frequent lack of common sense of purpose.

    As an innovation in learning infrastructure, a learning history should conform to the following nine guidelines. If it does not have these characteristics, then it may be a useful document; but, in our view, it is not a learning history.

    Use of "noticeable results"

    Noticeable results are events which people in the organization consider significant, whether or not the observers were involved with the learning effort. They are the hard measures that managers use as yardsticks for performance: a gain in value, a decline in errors, the ability to do something that had never been done before, or a clearly observable change in human behavior. They are not always positive; when the leaders of a project are dismissed instead of promoted, that is also a "noticeable result."

    Unless we acknowledge these evident facts, and using them as a springboard for the story, our learning histories lacks credibility. Learning histories without noticeable results may serve the "myth" of pilot team participants, but they will not be taken seriously by the rest of the organization.

    At the same time, we do not attribute causes to an organization's "noticeable results:" we do not say, "Here is the explanation of why this result happened." Instead, we use the "result" as a springboard into telling the underlying story ,both during interviews and during the presentation of the report.

    See Chapter 7: Noticeable Results.

    Intended for an audience broader than the participants in the story.

    Learning histories are intended to advance an understanding of the pilot team's experiences ,among members of the original pilot team, through the rest of the organization, and in the community of managers and practitioners as a whole. Unless it is designed to be viable for all three of these audiences, we do not consider it to be a learning history.

    There are, of course, conflicts between these audiences. For instance, the third audience (the general managerial audience) extends beyond the boundaries of the organization. The learning history must protect confidential information and individual privacy (which may mean disguising the name of the organization). At the same time, it must be set up so that, at some point in time, in some edited form, it can be released to the general public.

    See Chapter 6: Audiences for a learning history

    Data generated through reflective conversations

    The learning history depends on information drawn forth in settings where people can think through what they have set out to do, how expectations have been accomplished and/or shifted, and what has been learned. Without this type of input, the learning history will not develop a rich enough level of a rich enough level of content ,not just about events, but about the systemic structures underlying those events, and the mental models which exist below the surface of visible actions. In addition, reflective interviews give participants an opportunity to be open and expressive about their experience ,one of the major benefits of learning history work.

    See Chapter 9: The data gathering process.

    The Jointly-Told Tale

    The story told in a learning history should be told in the words of participants. They were present; they alone can describe what they were thinking, and what led them to their actions. Through the carefully-edited multiple voices of people involved in the story, the drama of the underlying structure of events comes through.

    But participants are not enough. The reader also wants external information: What is typical? What has been left out? What was the significance?

    Thus, we have developed the "jointly-told tale" form, borrowed in part from recent trends in cultural anthropology, in which participants and outside observers tell the story side by side.

    Without this form, the Learning History lacks distinction. It is just another report, rather than a collaboration between insiders' and outsiders' voices.

    We insist on the "jointly-told" nature of learning histories because it gives us the freedom to make assertions ,and yet know that we are protected, somewhat, from the charge of being "overbearing outside observers." The assertions come from the people of the organization.

    See Chapter 4: The jointly-told tale.

    The Two-Column Format

    Readers want to be told a succinct story. They want to be told what it means, what its implications are, and what they should do differently. As writers of learning histories, researchers need to account for their choices in asking questions, collecting, and selecting data. Readers should be told why particular quotes were chosen, how representative they were, and what interpretations and generalizations can be drawn from the narrative that is presented.

    We developed the two-column format as a vehicle for accomplishing all of these goals. Full column text is used for context setting and exposition. The right hand column is exclusively for primary data ,narrative from people involved in the change effort, written comments by participants, sections of memos or meeting transcripts, and speech excerpts, all edited to tell the story as a whole. The left hand column is used for more objectively interpretive material: notes on the questions which people were asked, conclusions and interpretations drawn by the researchers, attributions and generalizations, comments on how representative the elements of narrative may be, and implications of particular statements.

    In a two column format, complexity can be expressed which is not found in traditionally formatted reports. The two columns distinguish between relatively "objective" comments by non-participants on the left, and relatively "experiential" comments by participants in the story on the right. Reading the two column format feels unfamiliar to some people at first, but it leads (we believe) to deeper comprehension of complex learning and change efforts.

    See, within Chapter 4: The two-column format. And Chapter 10: Writing.

    A Team of Insiders and Outsiders

    A learning history reflects multiple perspectives including those of outsiders and insiders. Like any research into culture, these two perspectives are necessary for determining meaning. Outsiders will notice the peculiar ways in which an organization operates, ways which go unnoticed and are taken for granted by insiders. Insiders, however, often do not notice how their espoused values or beliefs are different from what is practiced.

    While outsiders are likely to notice these discrepancies, only insiders can provide an interpretation for discrepancies' significance and it's deeper meaning. A learning history effort requires a team with both outsider and insider membership.

    There are also pragmatic reasons for insiders to be an integral part of the learning history team. As part of an efforts to develop learning capabilities in organizations, companies need to take responsibility for the researcher's role. The learning history becomes part of the institutional feedback mechanism, an element of an "infrastructure for learning." If learning efforts expand, there will be a continual need for a people to teach others tools and methods for learning and reflecting on progress. As the formal learning initiatives spread, internal people who are trained and capable are needed to carry these efforts forward.

    See Chapter 5: Project design and planning.

    Linking attributions to observable data

    "The evaluations or judgments people make are not concrete or obvious," writes Chris Argyris. "They are abstract and highly inferential. Individuals treat them as if they were concrete because they produce them so automatically that they do not even think that their judgments are highly inferential." With this concern in mind, we have designed the learning history process so that judgments, inferences, and interpretations can always be linked, by the reader, to the data nearby.

    Simply asking people to "tell their story" would be problematic if comments were not linked to specific events and observable information. The story would take on the aura of gossip: Exaggerated myth, without being rooted in genuine detail.

    See, within Chapter 11: Attribution, interpretation, or generalization linked to description.

    A means to better conversation

    Learning histories should not be judged by the reports themselves. They should be judged by the quality of the conversation that they provoke.

    The learning history is conceived not as an ends in itself but rather as a means toward better conversation. This justifies the time and expense of the effort. There are three opportunities for reflective conversation: In the interviews, in various phases of distillation, and in the dissemination process. All three must be designed to draw managers into the reflective spirit and make full use of the information in the report.

    See Chapter 13: Dissemination.

    Distinguishing assessment, measurement, and evaluation from each other.

    The learning and improvement requires feedback of information that conveys what happened and how people are doing, the extent to which they have been able to achieve expectations, and the surfacing or reasoning and actions that contributed to final or intermediate outcomes. In modern organizations, feedback comes in the forms of assessments, measurements and evaluations. An understanding of different kinds of uses of information that provide feedback can help organizations that make concerted efforts to improve learn from their experience.

    See next section.

    Does a learning history qualify as assessment? Measurement? And/or evaluation?

    Any manager working toward creating a "learning organization" will sooner or later run up against a challenge of "proving" the value of what has been done. Researchers face the same question, "How do you prove what you hold to be true?" The conventional response is to turn to some form of assessment.

    "Assessment," however, is an emotionally loaded term. The word derives from the Latin root assessare ,to impose a tax, or to set a rate ,and it often seems to invoke a feeling of being persecuted by an auditor. People who contemplate assessment report palpable fear ,the word itself draws forth a strong, gut-level memory of being evaluated and measured. This results in defensive behaviors ,behavior which seeks foremost to protect oneself against the dangers of assessment. People play it safe; they restrict themselves from speaking frankly, making experiments, taking risks, or paying attention, because they know they may be punished in the assessment. They devote themselves to performing for the test, to make themselves look good. Thus the assessment, in itself, systematically and subtly defeats and limits the learning that it has been brought to measure.

    Yet without some form of assessment, it is difficult to learn from experience, transfer learning, or help organizations replicate achievements. One of the major questions in studying learning in business organizations is manager's requests and requirements for tangible, measurable evidence of an impact on their people's or organization's capabilities. How then can assessment be used to provide guidance and support for improving performance, rather than elicit fear, resentment, and resignation?

    Experience has shown that the first step, starting with the contacts with potential interviewees, is to distinguish "assessment" from "measurement" and "evaluation." The sort of assessments we make in Learning Histories are made of events and team activity, not of individual performance. Moreover, "assessment," in our context, is the comparison of reality to expectations. "Here is what we sought to do; and here is what we did." For example, a learning history might include a comment comparing the amount of money made by a new effort to what it was expected to make. By this definition, assessment is a fundamentally human, and necessary activity. It is a way of judging significance.

    We judge significance with words, not numbers. A problem occurs when "assessment" is conceived of as "measurement." Certainly, the benefits of measurement are undeniable: they allow the comparison of performance across a large number of teams, projects, processes, and activities. The concept of measurement is based on being able to ascribe relevant quantitative dimensions to phenomena which can reliably or repeatedly be observed. If we cannot measure a learning effort (or a quality, reengineering, or organizational change effort), then it is far more difficult to judge its improvement. Unless a learning effort is measured somehow, it probably cannot be improved.

    And measurement, even of qualitative processes, may be possible. Educator Robert Gahagan noted, "To say, `I know good art when I see it' is not sufficient. If you know it when you see it, you can describe what it looks like. If you can describe it, why can't we measure it? I work with elementary teachers who often want to measure such things as love, security and self esteem. When I ask them to tell me what those things look like, they immediately start describing activities. Then why can't we measure whether those activities are taking place?

    But the difficulty which measurement poses for learning in organizations can be illustrated by considering how businesses are measured. In business, accountants' measurements tell managers how the business performs. As Fred Kofman points out, accounting measurement is a form of language, and language determines what we perceive. The way in which corporations count "beans" indicates which type of "beans" are valued, and which are not, in a way so subtle that it determines the subconscious focus of peoples' attention. But then a learning effort begins, and creatively expands participants' horizons. Suddenly, what they "see" and do may clash with the types of perceptions encouraged by the existing accounting system. A rigid measurement scheme, like that of financial accounting, might not recognize the effort associated with people's learning, or people may limit their learning in order to comply with the perceptions a measurement system enforces.

    If we seek to assign value to the learning effort, instead of measuring performance, then we are well-advised to call our work "evaluation." Evaluation involves values and valuing, deriving from the Old French evaluer, "to value." Evaluation means to determine the worth of, to find the amount or value of, to appraise.

    But evaluation is not necessarily fair or beloved. People react more strongly to being evaluated than to being measured or assessed. Evaluation, they know, tries to judge them as a whole; but often it is not clearly defined who, or on what basis, value or worth is being determined, or what decisions and actions will follow on the basis of an evaluation.

    Evaluations are, by nature, subjective; but the evaluators rarely admit the subjectivity. Thus, different people evaluate the same "data" in vastly different ways, and they interpret the same evaluations with even greater disparity. Recently, a university report was released, evaluating one of the university's programs. One faculty member, reading the report, said, "It raised important issues, and helped define the Program; it will eliminate some controversy." A Program staff member said, "It was unfair; they pickedon the Program in an inaccurate way." An associate said, "The report didn't say anything new. We all knew that we were polarized at the Program." And a fourth person ignored every aspect of the report, except a line that accused the Program of publishing too little work. Since that wasn't true, the entire evaluation was suspect.

    Each of these people had read the same report; each came away with completely different opinions, planning to take completely different actions. That is the danger of evaluation.

    A challenge in the learning history process is to develop a method of assessment which frees people from the tyranny of a predetermined measurement and evaluation scheme. The learning history process does not deny the value of measurement, or the existence of measurement schemes in most organizations. Indeed, in learning history projects, all three types of judgment are made explicit:

  • We offer measurement of significant improvements if we can describe how and why the measurement systems themselves were developed, and what impact the measurement had on the people being measured.

  • We provide assessments, made by people throughout the organization, with a clear link between the assessment and the direct observable data.

  • We present overall evaluations, always with enough context that the reader (and interviewee) knows, beyond a doubt, that the evaluation stems directly from the sense and meaning that people make of their own experiences and work.

    Influences on learning history work

    A learning history combines research philosophies and techniques from ethnography, action research, organizational development and oral history.

    From ethnography come the science and art of techniques to investigate culture ,systematic approaches for participant observation, interviewing, and archival research ,used in understanding the day-to-day routines which make up people's lives. The ethnographic researcher defines him or herself as an outsider, a stranger on the outside developing an understanding of how those inside the cultural system make sense of their world.

    From action research and organizational development we have social scientist working with people in organizations to help improve them while as they capture data, reflect back, study the changes. Action research offers effective models and methods for exploring situations where the researchers are actively involved in changing the system they are helping. The typical action research intervention follows a cycle in which managers observe themselves acting and communicating, learn to recognize the assumptions inherent in their actions, build an understanding of the norms and values which drive those assumptions, and then plan new, more effective actions.

    A method to engage people in reporting on their experiences comes from the tradition of oral histories. Oral histories are often narratives which come from recorded in-depth interviews. The tradition of oral histories in transcribing the in-depth interview and using the voices of participants to record historical evidence provides a data collection method for rich, natural descriptions of the complex events. The oral history approach is used to rapidly capture the details of stories and employs the voice of the narrator to understand the way that they attribute meaning to their experience.

    The U.S. Army historian work

    The U.S. Army has a long-standing, in-depth "history" practice based upon the need to record the process of decision-making in the field.

    Thus, officers and staff are often taken to battlefields such as Gettysburg, to see firsthand what how the terrain dictated decisions, what the dilemmas were at the time, and how strategy and decision-making considerations played out into action.

    From the Army's experience, we have gleaned a renewed sense of the importance of understanding history for making effective decisions.

    The "Listening Project"

    Based at a non-profit organization called the Rural Southern Voice for Peace, this project has existed for 15 years. It is a community-based project which uses oral history techniques to organize communities. "The basic idea," wrote Stephanie Mills in the Essential Whole Earth Catalog, "is to go directly into divided communities and question people about the real content of their opinions, and to listen ,attentively and respectfully to the concerns that emerge."

    The Listening Project's work shows how learning histories can be a community-building tool, and how the process of interviewing and mutual reflection helps people appreciate each others' perspectives.

    The background of this Field Manual

    Our work conceiving and building learning histories began in March 1994, when we invited people in a half-dozen companies to join us for an ongoing practicum at MIT. Everyone was interested in the same question; How do you take the experience and understanding of a pilot team in an organizational learning effort, and make it relevant to the rest of the organization?

    Since then, this practicum ,the "learning historian pioneer's group" ,has met regularly in two- or three-day sessions to develop a practice for reaching those ends. During these sessions, based on our experiences, we have begun to document and evaluate the evolving body of theory, lore, and practice of learning history work.

    When we organized Reflection Learning Associates in mid-1995, as an independent venture for conducting learning history work, we began to incorporate material gleaned from RLA's workshops and consulting practice.

    This "field manual" represents our efforts to capture and communicate that body of work to date. It will be continually updated and expanded as the work is further developed and refined.