One of the best moments of my recent trip to Bhutan was the pleasure of spending time and having fantastic conversations with Jim Rugh, co-author of the book ‘RealWorld Evaluation’.
The RealWorld Evaluation approach was developed specifically to address the need for practical strategies on how to ensure the highest level of methodological rigour, consistent with the circumstances under which the evaluation has to be conducted. Sage first published the book in 2006 and the 2nd edition in 2012. (See www.RealWorldEvaluation.org)
The essence of the approach lies in the fact that program plans and implementation realities are most times starkly different and subject to dynamic environments and scenarios. It is a well-known fact that evaluators face innumerable challenges. All too often, evaluations are an ‘after thought’ of programs, not designed until the end. Baseline data or comparison groups are not established. Time and resources available are limited. Evaluation is seen as a tool to ‘prove’, not to ‘improve’, and clients have prior expectations for what they want evaluation findings to say. Further, many stakeholders do not understand evaluation; distrust the process; or even see it as a threat or as being ‘judged’.
As a result of these kinds of constraints, many of the basic principles of rigorous impact evaluation design (comparable pre-test — post test design, control group, adequate instrument development and testing, random sample selection, control for researcher bias, thorough documentation of the evaluation methodology etc.) are often sacrificed.
RealWorld Evaluation is an integrated approach to ensure acceptable standards of methodological rigour, while operating under real-world budget, time, data and political constraints. It promotes seven core steps, each with checklists for identifying constraints and determining how to address them. A toolkit of practical “RealWorld” evaluation techniques is available at www.RealWorldEvaluation.org.
4th Wheel Social Impact will soon be hosting a one-day workshop on the approach. We are excited to spread this approach among personnel involved in designing, implementing, monitoring, documenting and evaluating social programs in India.
I am humbled to have been trained by Jim Rugh to conduct this workshop. I am now officially an FOJ (Friend of Jim), who is determined to share learning and benefits of this approach.
Thank you Jim. (For everything)!
– Article written by: Sharon Weir, Co-Founder, 4th Wheel.