5 Key Takeaways from Session 3: From Tools to Systems – Rethinking AI Use in Evaluation
- 2 days ago
- 5 min read
Over the past week, our gLOCAL Evaluation Week 2026 series explored how AI is changing evaluation practice and why human judgment remains indispensable.
In our final session, From Tools to Systems: Rethinking AI Use in Evaluation, the conversation shifted to a broader question:
How Can Organizations Move Beyond Experimenting With AI Tools And Start Building Systems That Create Lasting Value?
Bringing together perspectives from technology, digital transformation, and evaluation, the discussion explored what it takes to embed AI meaningfully into organizations while keeping human judgment, accountability, and trust at the center.
Meet the Speakers
Vijayendra Vasu – Co-founder & Head of Engineering, MeraBills; technology leader with over 25 years of experience in building scalable digital systems.
Swapnil Agarwal – Co-founder, Dhwani Rural Information Systems; works with organizations on digital transformation and technology adoption.
Himanshu Chaudhary – Principal Consultant – MEAL, The 4th Wheel; evaluation practitioner with expertise in evidence generation and learning systems.
Jayashri Ramesh Sundaram – Associate Manager – Monitoring and Evaluation, The 4th Wheel; moderator of the session.
Key Takeaways From The Webinar
Here are five key takeaways from the conversation.
1. Start with the Problem, Not the Tool
One of the strongest themes throughout the discussion was that organizations often begin their AI journey by asking: "How can we use AI?"
The panel suggested a different starting point:
"What problem are we trying to solve?"
Whether it is improving data analysis, strengthening monitoring systems, reducing reporting burdens, or supporting program participants, AI should be viewed as a means rather than an end.
Organizations that focus on the problem first are more likely to identify meaningful applications of AI. Organizations that focus only on the technology risk adopting tools without creating real value.
The message was clear:
“Don't build AI projects. Solve organizational problems”
2. AI Is Most Powerful When It Removes Repetitive Work
The panel shared several examples where AI is already creating tangible value.
From generating situation reports during disasters to supporting product catalog creation for small entrepreneurs, AI is proving particularly effective in handling repetitive, time-consuming tasks.
In evaluation and social impact work, this could include:
Summarizing large volumes of documents
Cleaning datasets
Monitoring incoming information
Supporting report generation
Benchmarking program data against secondary datasets
Automating routine administrative work
Rather than replacing professionals, AI allows teams to spend less time on repetitive processes and more time on analysis, decision-making, and strategy.
As several speakers noted, the goal is not to replace human expertise but to free up more time for it.
3. Trust Comes from Governance, Not Technology
A recurring theme throughout the discussion was trust.
Many organizations are beginning to experiment with AI, but far fewer have established systems that ensure AI-generated outputs can be trusted and used confidently.
The panel emphasized that trustworthy AI systems require:
Clear data governance policies
Human oversight
Transparent decision-making processes
Validation mechanisms
Accountability structures
An AI-generated insight should always be explainable, traceable, and linked back to evidence.
“Technology alone cannot create trust”
Trust emerges from the systems, processes, and governance structures built around that technology.
4. Data Security and Privacy Cannot Be an Afterthought
As organizations increasingly use AI, questions around data privacy and security become more important than ever.
The discussion highlighted the particular responsibility social sector organizations carry when working with sensitive information, including personal identifiers, community stories, photographs, and participant data.
Several practical recommendations emerged:
Minimize collection of unnecessary personal data
Remove identifiers wherever possible
Establish clear organizational policies for AI use
Be transparent with communities about how data will be analyzed
Seek expert support when handling sensitive information and building secure systems
One important reminder stood out:
“Data breaches are often far more expensive than investing in security upfront.”
As organizations adopt AI, responsible data stewardship must remain non-negotiable.
5. The Future Requires More Builders, Not Fewer People
Perhaps the most reassuring insight from the session was that AI does not eliminate the need for people. In fact, the panel argued that organizations will continue to need strong practitioners, critical thinkers, and domain experts.
AI can support implementation.
AI can automate routine tasks.
AI can accelerate learning.
But organizations still need people who can:
Define problems
Exercise judgment
Design solutions
Interpret findings
Build trust
Lead change
The role of junior professionals may evolve, but it will not disappear. Instead, AI has the potential to accelerate learning by allowing people to spend less time on repetitive tasks and more time developing higher-order skills.
As the discussion highlighted, today's junior professionals are tomorrow's leaders. Investing in human capability remains just as important in an AI-enabled future.
Final Reflection
Across all three sessions, one message consistently emerged:
AI is not simply another tool.
It is prompting organizations to rethink how information flows, how evidence is generated, and how decisions are made.
But successful adoption is not about using the latest technology. It is about building systems that connect people, processes, evidence, and decision-making in meaningful ways.As organizations continue to navigate this transition, perhaps the most important takeaway from the session was also the simplest:
Focus on the problem. Build the system. Keep humans at the center.
The final session of our gLOCAL Evaluation Week 2026 series challenged us to think beyond individual AI tools and consider a much bigger question:
How do we build organizations that can use AI responsibly, effectively, and sustainably From governance and data security to organizational learning and systems design, the discussion made one thing clear: successful AI adoption is not a technology challenge alone. It is a people, process, and leadership challenge.
If these conversations sparked new ideas for your organization, we invite you to continue exploring the themes discussed during the webinar.
Explore the 4th Wheel Blog for insights on AI in evaluation, digital transformation, organizational learning, impact measurement, MEAL systems, Theory of Change, and evidence-informed decision-making.
Many of the questions raised during this session are explored further through practical examples, reflections, and case studies from the social impact sector.
Watch the full webinar recording to hear the complete discussion, real-world examples, audience questions, and practical recommendations shared by the panelists.
Important Time Stamps
03:45 – Setting the context: From tools to systems
06:20 – AI hype versus real organizational value
14:05 – Where nonprofits should begin their AI journey
22:18 – Internal vs programmatic use cases for AI
31:42 – Building AI-ready evaluation systems
41:10 – Why organizations should focus on problems, not tools
50:25 – Real-world examples of AI adoption from MeraBills and Dhwani
1:01:15 – Human oversight, governance, and accountability
1:10:50 – Data privacy, security, and responsible AI use
1:22:30 – What AI means for junior professionals and future talent
1:31:00 – Closing reflections and key takeaways
Follow our Social Impact Dialogues and subscribe to our updates for future conversations on evaluation, evidence, technology, and systems change.
As we conclude this three-part series, one message stands out:
AI is not simply changing how we work.
It is challenging us to rethink how organizations learn, make decisions, build trust, and create value.
The future will not belong to organizations with the most AI tools.
It will belong to organizations that can combine technology, evidence, human judgment, and strong systems to solve meaningful problems.
Focus on the problem. Build the system. Keep humans at the center.
