In my experience, assessment in higher education has traditionally focused on the final product. Students submit an essay, report, presentation, or project, and that completed piece of work becomes the primary basis for evaluation. Yet, anyone who has undertaken a substantial piece of academic work knows that much of the real learning happens long before the final submission. Ideas evolve, feedback is received and acted upon, early assumptions are challenged, and initial plans are refined. The final product tells only part of the story.
What is Black Box Assessment?
This year, as part of the MSc in Project Management module MN6021 Research in Projects and Organisations, I am exploring a different way of thinking about assessment, informed by the concept of Black Box Assessment developed by Winstone, Gravett, and Elkington (2026) and their workshop at the Assessment in Higher Education (AHE2026) conference. Rather than focusing solely on what students produce, this approach encourages us to pay closer attention to how learning develops throughout the assessment process.
The term Black Box comes from the idea that much of the learning journey is often hidden from view. In traditional assessment, educators see an assignment brief at one end and a completed piece of work at the other. What happens in between can be difficult to observe. Students may read extensively, seek feedback, revise their ideas, rethink their assumptions, and refine their approach several times before arriving at a final submission. These activities are often central to learning, yet they can remain largely invisible.
The Research Design Portfolio in Practice
The Research Design Portfolio used in MN6021 has been designed to place greater emphasis on these processes. Students are encouraged to engage with peer feedback, reflect on their developing understanding, and refine their research designs across the module’s duration. The intention is not simply to assess what students know at the end of the module, but to support how they learn throughout it.
What interests me as both an educator and researcher is how students experience this process-oriented approach. Do opportunities for feedback and revision help students strengthen their research designs? Do students become more aware of their own learning as they work through the portfolio? How do they respond when their ideas change or when they encounter challenges that require them to rethink their plans?
Exploring Students’ Experiences of Learning
The study seeks to better understand postgraduate students’ experiences of a Research Design Portfolio assessment designed to make learning processes more visible. Through an anonymous survey and, potentially, follow-up focus groups, students will be invited to reflect on their experiences of feedback, revision, learning, and the development of their research design ideas.
One aspect of Black Box Assessment that particularly resonates with me is its recognition that learning is rarely linear. Research, project work, and professional practice all involve uncertainty. We try things, encounter obstacles, rethink our assumptions, and adjust our direction. In many professional settings, this ability to learn from challenges and adapt our thinking is highly valued. Yet traditional assessment systems can sometimes reward only polished outcomes, obscuring the important developmental work that took place along the way.
Opening the Black Box
By exploring students’ experiences of the Research Design Portfolio, I hope to gain a better understanding of how assessment can support learning as an active, iterative process. The findings may inform future enhancements to the module and contribute to broader conversations about authentic, process-oriented assessment in higher education.
Ultimately, the project is guided by a simple question: if learning happens through feedback, reflection, revision, and the refinement of ideas, how can assessment help make those experiences more visible? Opening the “black box” of learning may provide valuable insights not only into what students produce, but also into how they develop as learners and emerging researchers.
At the time of writing, I am awaiting the outcome of my research ethics application. In positive anticipation of approval, I have already designed the study instruments and prepared an anonymous Qualtrics survey that aligns with the project’s research questions. Once the necessary approvals are in place, and students on the module have completed their assessment work, they will be invited to participate voluntarily and share their experiences of engaging with the Research Design Portfolio. I look forward to learning from their perspectives and exploring how process-oriented assessment can help make learning more visible.
Until next time, Sandra
Featured image courtesy of WordPress ‘Generate with AI’
Declaration: I used my University of Limerick Microsoft Copilot account to polish my initial draft and keep within my word count guide.
References:
Winstone, N. E., Gravett, K., & Elkington, S. (2026). Black Box Assessment: rethinking integrity and learning for a time of Generative AI. Assessment & Evaluation in Higher Education, 1–18. https://doi.org/10.1080/02602938.2026.2661365
Sandra Flynn 