Checking Your Compass: OER Assessment
Using the OER Publishing Review Packet
Why Review an OER Before Publishing?
By the time faculty creators finish drafting an OER, they have usually spent months (or years) writing, adapting, organizing, and revising it. They know the material inside and out. While that is a strength, it can also make it harder to notice what a fresh set of eyes might catch – things like gaps in the content, inconsistent formatting, missing attributions, accessibility issues, or places where the learning activities do not quite align with the objectives. That is where a publishing review can help.
For the OER for Social Justice project, we reviewed each OER through five different lenses: content, copyediting, copyright and attribution, instructional design and accessibility, and peer review. Each reviewer looked for something different. Taken together, those perspectives helped faculty creators see what was already working and what needed attention before publication, and provided clarity for what to potentially prioritize now and for future iterations.
The OER Publishing Review Packet grew out of that process. It is not meant to be the one “right” way to review an OER. Rather, it offers an adaptable model that OER creators, librarians, instructional designers, and program administrators can borrow from or simplify depending on their own needs and resources. In our case, the full process involved several reviewers and a fairly detailed workflow. Another institution might use only one or two parts of the packet. A faculty creator working independently might use it as a self-check. The value is in the different perspectives it invites, not in reproducing our exact process.
What Is Included in the Packet?
Before getting to the five review sections, the packet provides a few tools that help everyone understand the process and stay organized:
- OER Review Brief: Gives reviewers a quick overview of the project, review goals, timeline, contact information, and communication process. It also includes a question log where copyeditors and faculty creators can raise concerns, ask for clarification, and track responses.
- Review Coordination: Helps multiple reviewers see who is working on which sections, what stage each review has reached, and where feedback or follow-up is still needed. This is especially useful when several people have editing access to the same Pressbook.
- Pre-Publishing Checklist: Gives faculty creators a chance to confirm that major development and publishing tasks are complete before the formal review begins. The checklist covers areas such as organization, learning objectives, accessibility, citations, attributions, authorship, licensing, and front matter.
After those shared planning and coordination pages, the packet moves into five separate review sections: content, copyediting, copyright and attribution, instructional design and accessibility, and peer review. Each section includes guidance suited to that reviewer’s role rather than asking everyone to evaluate the OER in the same way.
Depending on the section, reviewers may find:
- Directions for completing the review
- A rubric specific to their review with criteria to consider
- Space for comments and examples
- Annotations that explain the criteria in more detail, including examples
- A place to record questions or note changes made
The rubrics, with the exception of the Peer Review, ask reviewers to choose from Not Observed, Rarely, Sometimes, Consistently, or Not Applicable based on provided annotations. The Peer Review rubric offers descriptions for peer reviewers to choose from rather than providing annotations. Those choices are not intended to produce a score. Instead, they help reviewers describe how often evidence of a practice appears across the OER and give some structure to the feedback that follows.
The written comments are where much of the value lies. Reviewers are asked to explain what they found, point to specific examples, and suggest improvements the faculty creators can actually use. The annotations support that work by explaining why each criterion matters, what reviewers should look for, and what it might look like in practice.
What Areas Are Reviewed?
Content
The content review looks at how well the OER works as a learning resource. Reviewers consider whether there is enough content to support the course, whether the table of contents follows a logical sequence, whether chapters move from simpler ideas to more complex ones, and whether visuals are used purposefully.
In our project, a student and the principal investigator of the OERFSJ project (a library dean) completed this review. That pairing gave us two useful perspectives. The student could speak to how the OER felt from a learner’s point of view, while the library dean brought a broader view of organization, usability, and academic resources.
This review is not the same as peer review. It is less about judging the disciplinary depth of the content and more about asking, “Does this resource make sense as a book students are expected to learn from?”
Copyediting
The copyediting review focuses on grammar, spelling, punctuation, consistency, readability, formatting, and citation style. Copyeditors also review the faculty team’s style sheet so they can preserve the intended voice and terminology of the OER.
Our copyeditors worked directly in Pressbooks for routine changes. They corrected grammar, standardized formatting, and smoothed out awkward wording. When they encountered something that required a larger decision, such as an unclear passage, a missing citation, or a change that might affect meaning, they added a question to the OER Brief of the review packet instead of making the decision for the authors.
That balance was important. The goal was to polish the OER without flattening the faculty creators’ voices.
Copyright, Attribution, and Social Justice
This review brings together legal, ethical, and social justice considerations. Reviewers look at copyright, fair use, licensing, and Title, Author, Source, and License (TASL) attributions. They also consider recognitive, representational, and redistributive justice (Lambert,>>>).
As a brief reminder, recognitive justice asks whose identities, experiences, and ways of knowing are acknowledged. Representational justice asks whether people and communities are able to represent themselves rather than being spoken for. Redistributive justice considers whether learners can freely access, retain, share, revise, and remix the resource.
Because these issues often overlap, it made sense for us to review them together. For example, an image might be openly licensed and legally usable but still raise questions about whose perspective is represented or whether the image reinforces a stereotype.
In our process, librarians from the four participating institutions completed this review. They could also correct TASLs and update metadata directly in Pressbooks when the needed information was available.
Instructional Design and Accessibility
The instructional design portion looks at whether the parts of the OER work together. Do the learning objectives align with the content? Do learners get opportunities to practice their learning objectives? Does the feedback embedded in the practice help them understand why an answer is correct or incorrect? Is the technology there because it supports learning, or is it simply there because it can be?
The accessibility portion examines headings, themes, images, links, tables, audio, video, and formulas. Reviewers look for barriers that might prevent some learners from using the OER fully.
Our instructional design and accessibility reviewers were able to make changes regarding accessibility directly in Pressbooks. For instance, they might correct a heading level or add missing alt text. More substantial instructional design recommendations were left as feedback for the faculty creators to consider.
Peer Review
Peer review brings in disciplinary expertise. Subject-matter experts consider the accuracy, comprehensiveness, relevance, clarity, and scholarly quality of the OER.
For our project, external experts completed the peer reviews independently. Their role was different from that of the student, librarian, copyeditor, or instructional designer. They were asking whether the content met the expectations of the field.
There is great value in having different experts review the OER through their unique lens. A book can be well organized and easy to navigate but still contain gaps in disciplinary coverage. It can also be accurate and comprehensive but difficult for students to follow. Those are different problems, and they call for different reviewers.
When Should the Packet Be Used?
The packet can be useful at more than one point in the OER creation process.
During planning and training, faculty creators can look through the criteria to get a sense of what they may want to consider as they design the resource. This can draw attention to organization, accessibility, representation, licensing, and alignment before those issues become harder to fix.
During drafting, a sample review may be enough. A reviewer might look at the front matter, one early chapter, a chapter with images or multimedia, and a chapter with learning activities or adapted content. This will not reveal every issue, but it can help identify patterns. That early check can save a great deal of time. In our project, it was much easier to tell a team, “This issue appears in the sample chapters, so you may want to check the rest of the book,” than to wait until every chapter was complete and then ask for the same correction dozens of times.
Before publication, though, we recommend reviewing the whole OER. A full review gives a clearer picture of how consistently the criteria have been addressed and provides time to resolve remaining concerns before the resource is shared more widely.
How Can a Team Coordinate the Review?
A multi-lens review can involve several people working in the same OER and the same review document. Without a clear plan, that can become confusing quickly. In our process, each review group received directions for using its part of the packet and had a lead who could answer questions. Copyeditors, copyright and attribution reviewers, and instructional design and accessibility reviewers were given access to the editing area in Pressbooks because their work sometimes involved making direct changes. Content and peer reviewers worked independently and provided feedback instead.
We also used a shared coordination page (Review Coordination page in the packet) so reviewers could see who was working where and what had already been completed. A separate question log on the OER Brief in the packet gave reviewers and faculty creators a place to raise questions and track responses.
Coordinating a Team Review
Before the review begins, we recommend deciding who is responsible for each area, who may edit the OER directly, where questions should be recorded, and how reviewers will show that their work is complete. Using a shared status log like the Review Coordination page can prevent duplicated effort and reduce the risk of reviewers editing the same chapter at the same time.
Direct editing can be helpful, but it works best when everyone understands the boundaries. Reviewers should know what they may change, what should be left for the authors, and where to document anything that may need explanation.
How Should Creators Respond to the Review?
Receiving five reviews at once can be a lot.
Even thoughtful, constructive feedback may feel overwhelming when it arrives all at once, especially after faculty creators have already invested so much time in the project. A long list of comments can easily look like a long list of failures, even when that is not what the reviewers intended.
For that reason, the follow-up process matters just as much as the review itself.
We found it helpful to send a link to the completed feedback in the packet by email first and give faculty creators some time to read it. Then we invited them to schedule a short meeting with the project team. That gave them space to ask questions, clarify comments, and talk through what support they needed.
Those conversations often changed the way the feedback felt. A faculty creator might arrive worried that every suggestion had to be addressed immediately. In the meeting, we could help separate urgent concerns from longer-term improvements, suggest pieces of the work that the project team could take on for them, and provide examples of other OER that might model what reviewers were looking for.
Some issues may need to be resolved before publication, particularly those involving copyright, licensing, or accessibility. Other recommendations may significantly strengthen the current edition but still leave room for author judgment. And some ideas may be better suited to a future revision.
It can help to sort the feedback into a few broad categories:
- Issues to address before publication
- Changes the project team or reviewers can help complete
- Recommendations the creators may choose to address
- Ideas to revisit in a future edition
The purpose of the review is not punitive judgment, and it is not perfection. It is to help creators make informed decisions about how to strengthen the OER.
Just as importantly, recognize the vulnerability involved. Faculty creators are inviting several people to examine work in which they have invested knowledge, care, and often a good deal of themselves. That deserves encouragement. The purpose of the review is not punitive judgment, and it is not perfection. It is to help creators make informed decisions about how to strengthen the OER.
A good review process should leave faculty creators with more than a list of problems. Ideally, they should leave with clear priorities, a manageable plan, answers to their questions, and people who are ready to help.
How to Access the OER Publishing Review Packet
Download the openly licensed OER Publishing Review Packet and adapt it for your own OER project. You may use the full model, select only the review areas you need, or revise the materials to fit your institution’s publishing process.