In a previous post, Opening the Black Box: making learning visible through assessment design, I reflected on the potential of Black Box Assessment and how a Research Design Portfolio might help make the processes behind student learning more visible. At that stage, the ideas were still largely theoretical. The module had only recently begun, students were finding their way around Portflow, and I was learning alongside them.
Now, as we draw closer to the end of module MN6021, I find myself reflecting on a different question: What am I noticing about student learning that I would not normally be able to observe through a traditional assessment?
Looking beyond the final portfolio
Traditionally, I would expect students to submit a completed research design portfolio towards the end of the semester. My role would then be to assess the final product and provide feedback after the fact. While that approach allows me to evaluate what students have produced, it offers limited insight into how they arrived there. This time feels different.
Through Portflow, topic-based Teams groups, and structured opportunities for feedback, I am seeing far more of the journey. Rather than encountering students’ thinking only at the point of submission, I am observing research questions being refined, methodologies being reconsidered, and ideas being developed over time. What has become visible is not just the outcome, or assessment product, but the process itself.
What the black box is revealing
One of the most noticeable differences has been student engagement with feedback. Early in the module, I invited students to submit their initial and refined research questions for feedback. Over half the class took up the offer. A few weeks later, I invited students to seek feedback on their proposed methodology. On this occasion, a few more students took up the invitation, with, for the most part, requests tending to be more focused and specific.
What struck me was not simply the number of requests but what those requests revealed. Students were exposing uncertainties, testing assumptions, and seeking guidance on decisions they were actively wrestling with. In a traditional assessment, many of these moments would remain hidden until the final submission, if they became visible at all. I have also noticed that the quality of feedback requests itself has improved over time. Students are becoming clearer about the support they need and are beginning to ask more targeted questions. That, in itself, feels like evidence of learning.
Viewing learning through the three windows
When designing the Research Design Portfolio assessment, I chose to focus on three of the Black Box Assessment windows identified by Winstone, Gravett and Elkington (2026): #2 research and evidence-gathering processes, #3 feedback engagement processes, and #4 reflection and metacognitive processes. Together, these windows provide opportunities to observe aspects of learning that are often hidden in traditional assessment. Through Portflow and peer discussion, I can see students refining research questions, engaging with feedback, reconsidering methodological choices, and reflecting on their progress in ways that would often remain invisible in a traditional assessment until final submission.
Feedback as part of learning
Another observation is that feedback has become part of the learning process rather than commentary delivered after learning has taken place. Students receive feedback, act on it, revise their work, and consider further questions. The emphasis shifts from correction to development.
Research design is inherently iterative. Questions are refined. Methodological choices evolve. Initial ideas are abandoned or adapted as understanding deepens. The portfolio approach allows these iterations to become visible in a way that would be difficult to capture through a single end-of-module submission.
Visible learning and visible non-engagement
Making learning visible has another consequence: it becomes easier to identify when students are not engaging with the process. Through Portflow and Teams, I can see patterns of participation that would otherwise remain hidden. Many students are actively refining their work, engaging with peers, and seeking feedback. Others appear to be working largely in isolation or postponing engagement until much later in the module.
This visibility is not about monitoring students. Rather, it provides opportunities to intervene, support, and encourage engagement before it is too late. It also highlights a challenge. Black Box Assessment assumes that students see value in making their learning visible. For many, that appears to be the case. For others, the instinct remains to focus exclusively on the final product.
Questions still to be answered
The experience has not been without challenges. Providing meaningful feedback takes time, and I am already wondering how scalable this approach would be with larger cohorts. There is also an important learning curve for students. Not everyone initially understands what useful feedback requests look like, and not everyone engages equally with peer support.
Yet despite these challenges, the most significant insight for me so far is a simple one.
The final portfolio is becoming less interesting than the process that leads to it.
These observations are, of course, my perspective as module leader and assessment designer. What remains unknown is how students themselves experience the portfolio process. Once the module is complete and final submissions have been received, I hope to explore students’ perspectives through an anonymous survey and follow-up focus groups. It will be interesting to discover where our perspectives align and where they differ.
For now, my strongest impression is that creating opportunities for students to make their learning visible has also made learning more visible to me. And perhaps that is one of the most valuable things that Black Box Assessment has to offer.
Until next time, Sandra
Images created by the author using a Canva template.
Declaration: I used my University of Limerick Microsoft Copilot account to polish my initial draft dictated using Apple’s Journal app on my new iPad.
References:
Winstone, N. E., Gravett, K., & Elkington, S. (2026). Black Box Assessment: rethinking integrity and learning for a time of Generative AI. Assessment & Evaluation in Higher Education, 1–18. https://doi.org/10.1080/02602938.2026.2661365
