How to Evaluate the Quality of Open Educational Resources

Open Educational Resources (OER) have opened up access to free learning materials for educators and students worldwide. But here’s the challenge – not all OER are created equal. With thousands of openly licensed textbooks, videos, lesson plans, and interactive modules available online, how can a teacher be confident that a particular resource is accurate, pedagogically sound, and worth using in their classroom? That’s where OER evaluation comes in. A thoughtful evaluation process helps educators filter out low-quality materials and select resources that genuinely support teaching and learning goals.

Why quality matters in OER 🔗

Unlike traditional textbooks that go through a publisher’s editorial and review pipeline, OER can be created and shared by anyone – from university professors to independent educators. This openness is a strength, but it also means there’s no automatic guarantee of quality. The idea that free educational materials are inherently lower quality is one of the biggest barriers to widespread OER adoption. If educators use inaccurate or poorly designed resources, it can negatively affect student learning and erode trust in open education altogether.

Quality evaluation is also essential because OER are often adapted and redistributed. A resource with factual errors or biased content can spread quickly across institutions and geographies. The UNESCO OER Recommendation (2019) specifically calls for regular quality assurance reviews of OER through evidence-based standards, benchmarks, and related criteria. For educators, evaluating OER isn’t just a nice-to-have – it’s a professional responsibility that protects the integrity of the learning experience.

There’s also a positive angle here. Research has shown that open and adapted textbooks were evaluated by K-12 teachers as having higher quality than copyright-restricted textbooks , particularly when teachers took the time to adapt them to their specific needs. This suggests that when educators actively engage in evaluating and customising OER, the results can actually surpass commercial alternatives.

General quality parameters for evaluating OER 🔗

Before diving into specific frameworks, it helps to understand the core quality parameters that apply to virtually any OER. These are the fundamental lenses through which any resource should be assessed.

Accuracy 🔗

The most basic requirement of any educational resource is that its content must be factually correct. Content should be accurate, free from errors, and unbiased. This means checking whether the information aligns with current knowledge in the field, whether data and statistics are properly sourced, and whether the resource avoids misleading claims. For rapidly evolving subjects like technology or medicine, currency of information is especially important – a resource that was accurate five years ago may now be outdated.

Relevance 🔗

A high-quality OER must align with the educator’s specific course objectives and the learners’ level. Educators should consider whether the content is at an appropriate reading level and whether it matches the expectations of their educational context. A brilliant university-level physics simulation, for instance, would be a poor fit for a secondary school classroom if the language and concepts are too advanced. Relevance also means the resource should cover the topics an educator actually needs, without significant gaps or excessive irrelevant material.

Accessibility 🔗

Accessibility refers to whether all learners – including those with disabilities – can effectively use the resource. Accessibility is critically important for all learners and should be considered in the design of all online materials. This includes compatibility with screen readers, availability of captions for video content, adequate colour contrast, and alternative text for images. Instructors planning to use OERs should keep in mind that materials must comply with federal and state accessibility requirements. Beyond disability access, accessibility also covers whether the resource works across different devices and internet speeds.

Interactivity 🔗

Interactivity measures how well an OER engages learners beyond passive reading or watching. Resources that include quizzes, discussion prompts, simulations, or hands-on activities tend to promote deeper understanding and better retention. Interactive elements encourage critical thinking and allow students to apply what they’ve learned in practical scenarios. Multimedia elements such as videos, podcasts, and interactive graphics often enhance engagement compared to text-only resources.

The TIPS framework for OER quality assurance 🔗

While the general quality parameters are useful, educators benefit from a more structured evaluation approach. The TIPS Framework, developed by Paul Kawachi and published by the Commonwealth Educational Media Centre for Asia (CEMCA), offers a set of 38 criteria that teachers can use when creating or evaluating OER. The acronym TIPS stands for four core dimensions: Teaching and Learning, Information, Presentation, and System.

Teaching and learning (T) 🔗

This dimension covers pedagogical effectiveness – whether the materials support clearly defined learning objectives, employ appropriate instructional design, facilitate active learning, and include mechanisms to assess student mastery. Key questions include: Are learning outcomes stated clearly? Does the resource support diverse learning styles? Are there assessment tools built into the material? With 16 criteria, this is the largest section of the framework, reflecting the central importance of pedagogy in OER quality.

Information and material content (I) 🔗

This dimension evaluates the accuracy, comprehensiveness, currency, and appropriateness of the substantive content. It asks whether the material is factually correct and free from bias, whether it covers the subject area thoroughly, and whether it stays current without becoming quickly obsolete. There are 7 criteria under this heading, focusing on the substance of what the resource actually teaches.

Presentation, product, and format (P) 🔗

Version 2 of the TIPS framework includes 8 criteria under this heading. Questions here include whether the material uses accessible language, whether the organisation is logical and intuitive, whether navigation elements are clear, and whether multimedia content functions properly. Essentially, this dimension checks whether the resource is well-designed and easy to use, regardless of how strong the underlying content may be.

System, technical, and technology (S) 🔗

This dimension evaluates technical quality, platform functionality, compatibility across devices and browsers, and compliance with technological standards. It ensures that materials work correctly in various technical environments and can take advantage of current technologies. A resource that only works on a specific browser or requires proprietary software scores poorly here. The TIPS framework includes 7 criteria under this dimension.

One important practical note: the TIPS Framework was validated through rigorous Delphi methodology by international OER experts, achieving a content validity index exceeding 0.80 – a statistically robust threshold for framework reliability. Not all 38 criteria need to be applied every time. Educators can select the most relevant criteria based on the type of OER and the context in which it will be used.

Evaluating OER using peer review and user ratings 🔗

Individual evaluation is important, but community-driven quality assessment adds another powerful layer. Two primary approaches stand out: formal peer review and crowd-sourced user ratings.

Formal peer review systems 🔗

The MERLOT (Multimedia Educational Resource for Learning and Online Teaching) platform operates one of the most well-established peer review systems for OER. Each discipline community within MERLOT has an editorial board that sets standards and manages the evaluation process. The process works in stages: first, editorial boards conduct a preliminary screening to identify materials worth reviewing. Then, two qualified peer reviewers are assigned to independently evaluate each selected resource using established evaluation criteria.

If there is a significant disagreement between the two reviewers, a third reviewer is brought in. The editor then creates a composite review report, which is shared with the original author for feedback before being published publicly. This structured, multi-step process mirrors traditional academic peer review and provides a high degree of reliability.

Other platforms take different approaches. OpenStax uses a pre-publication peer review model, similar to traditional publishing. The Open Textbook Library uses a 10-criteria rubric for evaluating OER, covering both content quality and usability. The key takeaway is that peer-reviewed OER have undergone a level of scrutiny that gives educators greater confidence in their quality.

User ratings and crowd-sourced reviews 🔗

As an alternative to formal peer review, crowdsourcing allows users to provide public reviews after they use OER materials. Many OER repositories include user reviews that rate materials on a 5-star spectrum, often including the reviewer’s name, occupation, and institution along with a detailed assessment following a rubric.

The quality of crowd-sourced reviews can vary, but repositories that provide a good framework or rubric for reviews can make them a valuable evaluation tool. They offer multiple perspectives on a given resource and tend to reflect experiences with the most current versions of materials. For educators, checking both formal peer reviews and user ratings provides a well-rounded picture of an OER’s strengths and weaknesses.

OER also offer a unique advantage for quality: because they can be freely edited, improvements and updates can be made continuously based on user feedback. This creates a cycle where community evaluation directly drives resource improvement – something that’s far harder with proprietary textbooks.

Institutional policies for OER quality assurance 🔗

While individual educators and community platforms play essential roles, institutions themselves need systematic policies to ensure OER quality at scale.

Establishing evaluation committees and guidelines 🔗

Educational institutions can set up formal OER evaluation committees that vet resources before they are adopted in classrooms. These committees can develop institutional guidelines specifying the criteria for selecting and endorsing OER, drawing on frameworks like TIPS or rubrics from organisations like Achieve. Training faculty on how to evaluate resources effectively is an equally important part of this process – an evaluation policy is only as good as the people implementing it.

Supporting national and international standards 🔗

The 2019 UNESCO Recommendation on OER is the first international normative instrument to address openly licensed educational materials and technologies in education. It outlines five action areas, including encouraging inclusive and equitable access to quality OER and developing supportive policies. The Dubai Declaration on OER, adopted in November 2024 at the 3rd UNESCO World OER Congress, further examines how to integrate AI and emerging technologies to advance OER creation, adaptation, and dissemination.

For OER to succeed at scale, robust mechanisms for capacity building, policy development, quality assurance, and accessibility are essential. Governments and institutions can work together by sharing best practices, aligning with international standards, and building OER repositories that include quality indicators such as metadata standards, licensing information, and peer review status.

Encouraging continuous improvement 🔗

Institutional policies should not be one-time checkboxes. They need to emphasise continuous improvement, encouraging OER creators to update and refine their materials regularly as educational needs and technologies evolve. Quality indicators for OER repositories – such as featured resources, user evaluation tools, peer review, clear authorship, standardised metadata, and specification of Creative Commons licences – all contribute to a healthier, more trustworthy OER ecosystem.

Institutions can also track the impact of OER adoption on student outcomes. Research comparing over 12,000 elementary students found no statistically significant differences in exam scores between those using open materials and those using commercial resources , suggesting OER can replace conventional materials without harming performance – while reducing costs and enabling local customisation.

Practical steps for educators evaluating OER 🔗

Bringing all of this together, here is a practical approach educators can follow when evaluating any OER:

Check the basics first. Look at the authorship – is the creator identified, and are their qualifications clear? Authority, accuracy, and quality are key criteria, and educators may need to investigate these more thoroughly when the source is unfamiliar. Verify the licence type to understand what you can and cannot do with the resource.

Apply quality parameters systematically. Assess accuracy, relevance to your course objectives, accessibility for all learners, and the level of interactivity. If you want a more detailed approach, use the TIPS framework to work through pedagogical, content, presentation, and technical dimensions.

Look for peer reviews and user ratings. Check whether the resource has been reviewed on platforms like MERLOT, the Open Textbook Library, or OER Commons. Multiple positive reviews from educators in your discipline are a strong quality signal.

Test the resource yourself. Consider whether the OER covers the content you want your students to learn, whether it’s accessible for your students’ level, and how it aligns with your learning objectives and weekly lessons. Navigate through it as a student would. Check that links work, multimedia loads properly, and interactive elements function as expected.

Consider adaptability. Evaluate whether the resource is published in a format that can be revised or remixed using freely available tools that run on all major platforms. A resource that’s easy to customise gives you significantly more value than one that’s locked into a rigid format.

The bigger picture 🔗

Evaluating OER quality isn’t just about protecting your classroom – it’s about strengthening the entire open education ecosystem. When educators evaluate critically, share their reviews, and provide feedback to resource creators, they contribute to a cycle of improvement that benefits everyone. The UNESCO OER Dynamic Coalition exists specifically to support this kind of collaborative approach, promoting networking and knowledge sharing around OER quality, policy, and sustainability.

As OER continue to grow in volume and influence, the ability to evaluate their quality becomes an essential skill for every educator. Frameworks like TIPS provide structure, peer review platforms provide community validation, and institutional policies provide systemic support. Together, these layers create a quality assurance ecosystem that makes open education not just affordable, but genuinely effective.

What do you think? How do you currently evaluate the OER you use in your teaching, and do you think a structured framework like TIPS could make that process more effective? What role should institutions play in ensuring OER quality – should they mandate evaluation standards, or leave it to individual educators?

References

1. https://www.unesco.org/en/open-educational-resources
2. https://www.w3.org/WAI/standards-guidelines/wcag/
3. https://oasis.col.org/handle/11599/562
4. https://info.merlot.org/merlothelp/MERLOT_Peer_Review_Information.htm
5. https://openstax.org/
6. https://open.umn.edu/opentextbooks/
7. https://www.achieve.org/