The Apex Blend
We believe that evaluation should be in service of learning and equity.
Apex defines evaluation as helping people use credible evidence to build, share, test, and evolve their mental models about their work and the world in which it takes place. Our portfolio is focused entirely on equity work and we align ourselves with organizations and programs addressing the wicked problems of poverty and income inequality, health and education disparities, and racism and discrimination. The broad spectrum of experience and expertise held by Apex allows us to select and adapt methods and approaches best suited to optimize evaluation and learning for programs and organizations. We call this adaptive approach to theories and methods the Apex Blend.
The Apex Blend utilizes a combination of theories and methods to inform and guide practice, including systems thinking, equitable evaluation, utilization-focused evaluation, developmental evaluation, principles-focused evaluation, Results-Based Accountability™, and data visualization.
Our approach is rooted in a simple model of systems thinking rooted in four interacting patterns – distinction-making, zooming in and out, grouping and sorting part-whole system structure, exploring relationships and non-relationships – all from one or more perspectives. This Distinctions, Systems, Relationships, Perspectives (DSRP) model was developed by Derek and Laura Cabrera and considers the complex factors inherent in the larger system in which a program is embedded. It is a foundational framework. Systems Learning™ appropriately matches evaluation methods to the mental models being tested, giving special attention to the role perspectives play in emergent learning.
Equity is embedded in all that we do at Apex, from business practices to our evaluation approach. We are part of the equitable evaluation movement which calls for using evaluation as a tool for advancing equity through culturally appropriate methods and a diversity of teams that reveal drivers of inequity and empower those most affected to shape and own how evaluation happens. Our take is simply that equitable evaluation is about telling more authentic, more compelling, and more impactful stories based on a credible body of evidence – credibility not from numbers, but from the testimony of witnesses closest to the ground.
Developed by Michael Quinn Patton, Utilization-Focused Evaluation places a premium on planning and implementing evaluations in ways that maximize the usefulness of both the process and products of evaluation for the audiences doing the work and for those making decisions to inform, guide, and measure progress and performance.
Developmental evaluation, also by Michael Quinn Patton, applies utilization-focused evaluation principles to innovative initiatives being implemented in dynamic and complex environments where participants, conditions, interventions, and contexts are in flux and pathways for achieving desired outcomes are subject to change. This method supports reality-testing, innovation, and adaptation in complex dynamic systems where relationships among critical elements are nonlinear and emergent.
Quinn Patton developed principles-focused evaluation to complement and focus utilization-focused and developmental evaluation practices on examining:
- Whether the principles that guide a program or organization are clear, meaningful, and actionable, and if so,
- Whether such principles are actually being followed and, if so,
- Whether they are leading to desired results.
Principles-focused evaluation is an evolution that helps navigate the volatile, uncertain, and dynamic complex environments that characterize equity work. It helps shift the perspective from, “Are we doing things right?” to, “Are we doing the right things?”
We follow principles from the Results-Based Accountability™ (RBA) framework with important adaptations to ensure equity. RBA involves:
- Using simple, clear, and consistent language and avoidance of noisy jargon;
- Making distinctions between programmatic performance that can be more closely attributed to the work versus population results to which the work may contribute;
- Using data to justify, guide, and monitor specific actions around some theory of change (what RBA calls “turning the curve” is the process of reality-testing the implementation plan);
- Attending to the story behind data to bring in additional context; and
- Developing specific evaluation questions and measures around three guiding questions: “How much are we doing? How well are we doing it? Is anyone better off?
This last question can be problematic for equity-focused efforts because it too often leads to a focus on longer-term outcomes. And while we strive for equitable outcomes, we contend that people are better off when they have equitable resources and opportunities. Therefore, we focus accountability on inputs rather than outputs. This is the lens through which we refine and adapt evaluation methods to ensure a balance between process, quality, and outcomes.
Data visualization is much more than making data look good. We focus on understanding our audiences and create deliverables with these populations in mind. We realize that streamlined documents highlighting key learnings with visualizations often stimulate thinking better than lengthy, dense reports. Our reporting is designed to make findings accessible, relevant, and actionable.
Apex mixes theories and approaches in order to create and implement evaluation plans for the diverse needs of our clients, the communities they serve, and our larger society. Michael Quinn Patton rightly noted, “Developments in the larger world reverberate within evaluation and lead to both calls for systems transformation and evaluation of the extent to which systems transformation is occurring. In the vision of a more just, equitable, and sustainable world, we are going to have to integrate evaluation theories and methods in order to respond to these new challenges.” In alignment with the trajectory described by Patton, we engage multiple evaluation approaches in support of and guided by a theory of transformation.
We believe that evaluation is fundamentally a learning endeavor that should elevate perspectives and scrutinize numbers. We agree with Deborah Stone in Counting: “Every number is born of subjective judgments, points of view, and cultural assumptions. Numbers are filled with bias through and through…”. Ultimately, numbers are stories, evoking Brené Brown’s adage that “stories are just data with a soul.”
Our Commitment to Equitable Evaluation
“Culturally responsive and equitable evaluation (CREE) requires the integration of diversity, inclusion, and equity in all phases of evaluation. CREE incorporates cultural, structural, and contextual factors (e.g. historical, social, economic, racial, ethnic, gender) using a participatory process that shifts power to individuals most impacted.” (Expanding the Bench)
Apex President and CEO Carlos Romero explains how organizations can foster internal support for CREE. In this portion of the CREE Learning Series, he shares Apex’s approach to building knowledge of CREE through various elements, from creating a CREE vision to applying methods that align with this approach.