Towards a Holistic Understanding of Evidence: A Working Paper

Synopsis

This report synthesizes reflections and recent literature to propose a rethinking of evidence generation in educational technology (EdTech). It advocates for three paradigm shifts to advance the discourse and practice in EdTech impact:
1) Evidence of impact should be understood as multidimensional, encompassing a variety of measures, not only experimental evidence.
2) Evidence collection should support an iterative and ongoing process of evaluating and improving impact over time.
3) Rigorous approaches are essential throughout the entire cycle of evidence to ensure validity and reliability.
Recognizing that stakeholders in the EdTech ecosystem have diverse incentives and metrics for success, the report argues that aligning these efforts toward a shared goal of improving educational equity can accelerate the production of sharable knowledge and foster wider, more meaningful impacts.

Published: September, 18, 2024

Authors: Natalia I. Kucirkova and David Dockterman