Companies’ maturity and types of evidence
At our center, we engage with EdTech companies at various stages of development, and a recurring inquiry pertains to the relationship between different types of evidence and the specific stage of product development. Perspectives on this matter vary significantly; some are founded on established research guidelines, while others are based on estimations.
To provide clarity, our research group led by Professor Natalia Kucirkova has undertaken a rigorous analysis, employing a systematic literature review and synthesizing findings from multiple reports on efficacy and effectiveness. This work has enabled us to establish a coherent framework that aligns the stages of research with product development phases. The results of this analysis are presented below.
Efficacy research types and companies’ maturity
Types of efficacy research studies mapped on weight of evidence and companies' maturity
The summary shows that typically, startups rely on theoretical frameworks to demonstrate potential value, focusing on logical models and research reviews. Series A/B companies move to correlational studies, aiming to show a relationship between their solution and outcomes. Series C+ companies implement experimental designs, offering the highest level of evidence to validate efficacy, often through controlled trials and prospective studies.
The process overall moves from conceptual validation to experimental testing, reflecting increasing company maturity and evidence rigor.
Effectiveness research types and companies’ maturity
Types of effectiveness research studies mapped on weight of evidence and companies' maturity
The illustration shows that Startups start by focusing on pedagogical compatibility and building user-centered products through co-design and expert reviews. As companies grow, they conduct observational studies to provide practical insights into product use, ensuring feasibility and usability. Series A/B and C+ companies aim for replicable evidence through rigorous and repeated studies, such as long-term evaluations or cost-effectiveness analyses, to demonstrate sustained impact.
Stages or iterations?
Please note that we use the financial stages of company maturity—such as startups, Series A, B, etc.—as a simplified way to illustrate a complex process. These stages represent the progression from developing a prototype and acquiring initial customers, to scaling in a specific context, and eventually expanding to broader implementation. This shorthand helps convey the evolving maturity of a company as it grows and gains market traction.
The stages of research can guide companies in developing a research roadmap and allocating budgets for studies. However, research is an ongoing, iterative process, and the individual stages should not be viewed in isolation. Just as a well-designed logic model can inform the entire cycle—from a conceptual study to a randomized controlled trial (RCT)—the results from an observational study can also contribute to future feasibility studies. In reality, these stages are part of a dynamic, continuous cycle that should contribute to a portfolio of studies, or what we call your evidence portfolio.
Cite this article as: Kucirkova, N. (2024). Companies’ maturity and types of evidence, Insights: International Centre for EdTech Impact.