Theory of Change

Theory of Change emerged from the field of program theory and program evaluation in the mid 1990s as a new way of analyzing the theories motivating programs and initiatives working for social and political change.[3] Its earlier origins can be traced to Peter Drucker's articulation of Management by Objectives, popularized in his 1954 book The Practice of Management. Management by Objectives requires identifying higher-order Goals, and lower-order Objectives which, if achieved, are expected to result in the Goals being achieved. Theory of Change extends beyond Goals (commonly named Outcomes in Theory of Change terminology) and Objectives to include Impact – the anticipated result of achieving stated goals.

Theory of Change is focused not just on generating knowledge about whether a program is effective, but also on explaining what methods it uses to be effective.[4] Theory of Change as a concept has strong roots in a number of disciplines, including environmental and organizational psychology, but has also increasingly been connected to sociology and political science.[5] Within industrial-organizational psychology, Austin and Bartunek have noted that approaches to organizational development are frequently based on more or less explicit assumptions about 1) the processes through which organizations change, and 2) the interventions needed to effect change.[6]

Within evaluation practice, Theory of Change emerged in the 1990s at the Aspen Institute Roundtable on Community Change as a means to model and evaluate comprehensive community initiatives. Notable methodologists, such as Huey-tsyh Chen, Peter Rossi, Michael Quinn Patton, Heléne Clark, and Carol Weiss, had been thinking about how to apply program theories to evaluation since 1980. The Roundtable's early work focused on working through the challenges of evaluating complex community initiatives. This work culminated in a 1995 publication, ‘New Approaches to Evaluating Comprehensive Community Initiatives’.[3] In that book, Carol Weiss, a member of the Roundtable's steering committee on evaluation, hypothesized that a key reason complex programs are so difficult to evaluate is that the assumptions that inspire them are poorly articulated. She argued that stakeholders of complex community initiatives typically are unclear about how the change process will unfold and therefore place little attention on the early and mid-term changes needed to reach a longer-term goal.

Weiss popularized the term “Theory of Change” as a way to describe the set of assumptions that explain both the mini-steps that lead to the long-term goal of interest and the connections between program activities and outcomes that occur at each step of the way. She challenged designers of complex community-based initiatives to be specific about the theories of change guiding their work and suggested that doing so would improve their overall evaluation plans and would strengthen their ability to claim credit for outcomes that were predicted in their theory. She called for the use of an approach that, at first glance, seems like common sense: lay out the sequence of outcomes that are expected to occur as the result of an intervention, and plan an evaluation strategy around tracking whether these expected outcomes are actually produced. Her stature in the field, and the apparent promise of this idea, motivated a number of foundations to support the use of this technique—later termed “the Theory of Change approach”—in the evaluations of community change initiatives. In the years that followed, a number of evaluations were developed around this approach, fueling more interest in the field about its value and potential application.[3]

Between 2000 – 2002, the Aspen Roundtable for Community Change led the dissemination and case studies of the Theory of Change approach, although still mostly applied to the field of community initiatives. As the Aspen Roundtable concluded its leadership in the field and moved on to apply Theory of Change to such topics as structural racism, others expanded the visibility and application of Theory of Change into international development, public health, human rights and more. The visibility and knowledge of Theory of Change grew with the creation in 2002 of theoryofchange.org[7] and later of Theory of Change Online[8] software.

In the 2010s, interest increased with some reviews commissioned by Comic Relief in the UK, the Department for International Development in the UK, the Asia Foundation and Oxfam Australia to name a few.[9][10][11] The explosion of knowledge of the term, and demand for "theories", led to the formation in 2013 of the first non-profit dedicated to promoting and clarifying standards for Theory of Change. The Center for Theory of Change[7] houses a library, definitions, glossary and is licensed to offer Theory of Change Online by ActKnowledge[12] free of charge.

The use of Theory of Change in planning and evaluation has increased among philanthropies, government agencies, development organizations, universities, international NGOs, the UN, and many other major organizations in both developed and developing countries.[13] This has led to new areas of work, such as linking the Theory of Change approach to systems thinking and complexity. Change processes are no longer seen as linear, but as having many feedback loops that need to be understood. Consequently, Theory of Change is strengthening monitoring, evaluation and learning. They are also helping to understand and assess impact in hard to measure areas, such as governance, capacity strengthening and institutional development. Innovations continue to emerge.[14]

Despite the growing ubiquity of Theory of Change, especially in the development arena, understanding of the approach and the methods necessary to implement it effectively are not uniform. In fact, there is evidence of some confusion about what the term ‘Theory of Change’ actually means; in some cases, what some program developers describe as a Theory of Change is, in essence, simply a log frame, strategic plan or another approach that does not encompass the complexity of the Theory of Change approach. There is also inconsistent use of other common Theory of Change terminology (e.g., outputs, outcomes, impacts, etc.), which confounds effective Theory of Change design, evaluation, and learning.[15][16]

A Theory of Change is a high order, or macro, If-Then statement: If this is done, Then these are the anticipated results. The outcomes pathway is a set of needed conditions relevant to a given field of action, which are placed diagrammatically in logical relationship to one another and connected with arrows that posit causality. Outcomes along the pathway are also preconditions to outcomes above them. Thus, early outcomes must be in place for intermediate outcomes to be achieved; intermediate outcomes must be in place for the next set of outcomes to be achieved; and so on. An outcomes pathway therefore represents the change logic and its underlying set of assumptions, which are spelled out in the rationales given for why specific connections exist between outcomes and in the theory narrative.[17]

In the early days of Theory of Change, Anne Kubisch and others established three quality control criteria.[18] These are:

Plausibility

Plausibility refers to the logic of the outcomes' pathway. Does it make sense? Are the outcomes in the right order? Are the preconditions each necessary and collectively sufficient to reach the long-term outcomes and ultimate impact? Are there gaps in the logic?

Feasibility

Feasibility refers to whether the initiative can realistically achieve its long-term outcomes and impact. Does the organization have adequate resources? Does it need partners? Does the scope, expectations, or timeline of the theory need adjustment?

Testability

Testability refers chiefly to the indicators: Are they solid and measurable? Will they yield sufficient information to evaluate the success of the initiative? Will they be convincing to necessary audiences?

In addition to these three basic quality control criteria, ActKnowledge[12] has added another key criterion: Appropriate Scope. An actionable theory that can be communicated to the key audiences is dependent in part upon choosing the right scope: broad enough to leave no gaps in the model, yet focused enough on the opportunities and resources at hand. Appropriate Scope also integrates the evaluation concept of “accountability”. Many Theory of Change outcome pathways include an “accountability ceiling,” often a dashed line drawn across the pathway that separates outcomes the organization will monitor and claim credit for attaining from higher-order outcomes that are beyond its power to achieve—e.g., “a just society.”[19]

An Outcomes Pathway mapped out

An important first step in the process is identifying a workable long-term goal and long-term outcomes. The long-term goal should be something the initiative can realistically achieve and that everyone involved understands. A trained external facilitator is best to lead the group to consensus and specificity in this process.

Once a long-term goal is identified, the group then considers: “What conditions must be in place for us to reach the goal?” Any such necessary conditions should be shown as outcomes on the Theory of Change pathway, underneath the long-term outcome. These outcomes act as preconditions to the long-term outcome.

The process of identifying preconditions continues, drilling down the pathway by posing fundamental questions such as: “What has to be in place for this outcome to be achieved?” and “Are these preconditions sufficient for the outcome to be achieved?” In these sessions, participants may use markers, sticky notes, and chart paper to identify and organize outcomes, surface assumptions, develop indicators, and so on.

The messy group work is then usually captured by the facilitator in digital form, through which the content can be expanded, edited, printed, shared, and otherwise managed as the theory continues to be developed.

Theory of change in process and action

The ultimate success of any Theory of Change lies in its ability to demonstrate progress on the achievement of outcomes. Evidence of success confirms the theory and indicates that the initiative is effective. Therefore, the outcomes in a Theory of Change must be coupled with indicators that guide and facilitate measurement.

Indicators may be said to operationalize the outcomes – that is, they make the outcomes understandable in concrete, observable and measurable terms. The relationship of indicator to outcome can be confusing and may be clarified with this simple formula: “I’ll know [outcome reached] when I see [indicator].” For example, “I’ll know that teenagers in the program understand the prenatal nutrition and health guidelines when I see program participants identifying foods that are good sources of nutrition.” A graduated set of indicators can be used to measure and assess the extent to which an outcome has been realized.[20]

Ideally, every outcome on the outcomes pathway (below the dashed accountability ceiling) should have an indicator, but available resources often make that difficult to do. Many groups want to designate priority outcomes – that is, outcomes they know they need to measure if the theory is going to hold. These are the outcomes that must be operationalized (that is, made measurable by one or more indicators.) At a minimum, every outcome for which initial interventions will be designed should have at least one indicator.[17]

Though indicators are valuable, it may be necessary to do more in-depth data collection and analyses to assess whether and how outcomes and assumptions have been realized.[20]

As the origins of Theory of Change lie in the field of monitoring and evaluation, developments over the years have ensured that Theory of Change continues to be an invaluable method to conduct evaluations of many different types of projects and organizations. Often posing theory-based evaluation questions helps to focus evaluation efforts on key concerns. As well, there may be a need to pick the right indicators from among the many available, and one can use “monitoring questions” to select the indicators that will be most helpful. The monitoring questions take the form of “What do we really need to know in order to manage grant-making directed to the achievement of this outcome?[21] It is important to understand success beyond just knowing “what works”. Experience has shown that blindly copying or scaling an intervention hardly ever works. An important task for monitoring and evaluation is to gather enough knowledge and understanding so as to be able to predict – with some degree of confidence – how an initiative and set of activities might work in a different situation, or how it needs to be adjusted to get similar or better results. We also need to be able to combine evidence from a number of studies in order to build a stronger picture of what is taking place, how it is unfolding, and, most importantly, how context influences the initiative.

Just as development of a Theory of Change is a participatory process, a ToC-based monitoring and evaluation system can be designed in a participatory way. For example, grant managers can be involved in choosing the outcomes of greatest interest to them in their decision-making. Similarly, people on the ground can have input into which indicators to use and how to operationalize them, choices of instruments and methods of data collection, and which existing sources of data may be used in tracking indicators.[22]

As an ex ante planning tool, Theory of Change can make research design and implementation more realistic and relevant by facilitating critical reflection on the role of research in a change process. It can also make research more participatory, by identifying and engaging key stakeholders of the research, and enabling co-ownership of the research process and the research results (e.g., findings).[24] Theory of Change can accompany or be embedded within a research proposal.

As a monitoring tool, a Theory of Change helps identify useful indicators to assess progress and can facilitate adaptive management. It does this by stimulating learning about what strategies work in the specific research context, and where additional attention and resources need to be directed in order to achieve intended outcomes. This can inform adjustments to planned research activities, and also take advantage of unexpected opportunities by giving users a framework to determine whether an opportunity does or does not align with the purpose and objectives of the research project or program.[24]

Theory of Change serves as the main analytical framework in theory-based research evaluation. A Theory of Change also helps identify what data are necessary to test whether the change happened as hypothesized. As an ex post evaluation tool, evaluators can assess the actual achievements of research projects or programs against expected outcomes, increasing research transparency and accountability to results.[20][24]

There are two areas of work that, although not coordinated with Theory of Change, offer much to think about in making Theory of Change more focused and effective:

1. The Annie E. Casey Foundation proposes mapping an organization's social change work along three criteria: Impact, Influence, Leverage.

• The impact of your work is its program outcomes
• Your influence is how much other actors change as a result of your work
• Your leverage is how much investment others put into your model.[28]

To date, Theory of Change has not distinguished impact, influence, and leverage as types of outcomes, but it may be useful to do so as a way of focusing the Theory of Change on measurable achievements. Particularly, when using Theory of Change to guide monitoring and evaluation, the Casey rubric helps focus the group's attention on outcomes, which could, if achieved, be convincingly attributed to the group's work. Other than direct program-related outcomes (impact), the Theory would anticipate outcomes in influence and outcomes in leverage. This approach could thereby help to avoid mapping outcomes involving broad shifts in behavior and values among whole populations, which are easy to think about, but are very difficult to monitor and to attribute to any one program.

2. Another refinement, which directly addresses this problem of attribution, comes from Outcome mapping. This process distinguishes changes in state from changes in behavior, changes in “state” being just those broad shifts in economic conditions, policy, politics, institutional behavior, and so on, among whole populations (e.g., cities, regions, countries, industries, economic sectors, etc.). Measuring changes in state can exceed the capacity of any one actor's monitoring capabilities. Governments collect data on changes in state but, of course, the data may not be calibrated to measure the kinds of change anticipated in any one Theory of Change. Changes in state are also, as stated above, difficult to attribute to any one source.

In contrast, changes in behavior are much easier to monitor, and more easily related to a group's own work. The Outcomes Mapping focus on changes in behavior would tend to direct a Theory of Change toward outcomes like this, which are outcomes the change agent cares most about and which it can relatively easily monitor and evaluate. There would be proportionately less attention to outcomes such as “every child is within five minutes walk of a playground” or “residents are healthy”. Such “changes in state” are more difficult to monitor and to attribute with certainty.[29]

Does Theory of Change frustrate or complement strategic thinking? This is an ongoing and important discussion, especially as there is an increasing emphasis on and demand for strategy as grounded, flexible, and opportunity-driven. Some perspectives understand ToC as a fixed model that gets in the way of effective work and useful evaluation. However, Patrizi notes that ToC is only at odds with strategic behavior if an organization treats their ToC like any other fixed plan. As Patrizi writes: “Once assumptions [in a theory of change] are laid out, 1) foundations don't actually test those assumptions, and 2) they don't see using the model as a continuous process. It is, like, ‘Well, we did our ToC and now we are done’.[22] If the change model is instead treated as something to adjust as organizations learn what works from experience in the field, then the theory should not be at odds with strategic behavior. If strategy is about seizing opportunities and trying out what works, the Theory of Change model serves as a guiding frame of reference. A list is not a model; a list does not push practitioners to consider the goals as part of a systematic model of change, or to think critically—strategically—about how best to attain the outcomes along the pathway.

Given that things don't happen in a straight-line sequence – as things impact each other in multiple, partly unpredictable ways, with all kinds of feedback loops that aren't modeled in a top-down diagramming format – an important question is: How adequate is the linear Theory of Change model as a description of what's going to happen? One answer to the question is that Theory of Change does not, in fact, model how things happen; rather, it models how we believe things will happen. Theory of Change is a forecast that shows what conditions we believe must exist for other conditions to come into being. As it is forward looking and logical, Theory of Change reflects the way we think logically –that is, if a, then b—and chronologically—first this, then that. The linear format is therefore appropriate. It can be helpful to complement Theory of Change with a process model that shows how the Theory of Change fits into a larger, more cyclical scheme in which theory leads to action, which leads to monitoring and evaluation, which leads to adjustment of the theory, which leads to the next action, more monitoring and evaluation, and so on. Such a process model depicts the linear theory as a conceptual driver of change, which must, to remain useful, be accompanied not only by taking action but also by evaluation and recalibration.[19]

It is important to remember that it is often a high-level person who has endorsed and initiated the Theory of Change process as something of value, so they have “bought in” at the beginning. The challenge that emerges is, therefore, how to sustain their support while letting others develop the theory. This is similar to many other issues that come up in matters of hierarchy and delegation: effective leaders delegate and trust their teams to carry out the work. There are many variations on the model but usually it involves good measures of delegation and, conversely, of reporting back to get the leader's thinking as the work progresses. It is necessary to come to high-level people having laid out your best thinking: don't go to them without something concrete to which they can respond, but don't wait until everything is perfect, either.[1]