Permanent Registered Reports Track At Computer Science Education

Recently, Aleata Hubbard Cheuoua, Eva Marinus and I acted as guest editors of a special issue at the Computer Science Education journal. The special issue was distinctive because it only accepted replication studies, and reviewed them as Registered Reports. Registered Reports are a new way to publish science: you peer review the study’s design, before the study has been carried out:

(Image Creative Commons Attribution-NoDerivatives 4.0 International License from COS.)

This minimises all kinds of scientific questionable research practices (such as fishing for significant results) and means that changes can be made at a meaningful stage. Often in classical peer review, reviewers ask for changes (e.g. “you should have collected data on prior experience”) that cannot be addressed after data collection has been completed.

I don’t want to spend a long time writing about Registered Reports here, but bottom line is: we completed the special issue, it went well, you can read our editorial, the issue itself, and a whole paper that we just published at ICER 2022 about the experience.

Permanent Registered Reports Track

As a result of the special issue and the paper, Aleata, Eva and I have been appointed as Associated Editors at Computer Science Education to handle a permanent Registered Reports track. The addition of the track is currently being processed by the publisher: we have written submission instructions that will appear on the journal website shortly and a new track is being added to the submission system. In the mean time, the draft of the submission instructions is in this Google Doc (although it is possible it may change slightly before becoming official.)

We would like to invite all researchers in computer science education research to consider whether to make their next study a registered report. It doesn’t fit all models of research (hence it’s an extra track, not a replacement) but for many studies it fits very well. If you’re at the planning stage for a new study, you could write up your planned method analysis, submit it as a registered report, and get expert feedback plus an accept-in-principle before you’ve even begun collecting the data. What’s not to like?

If you have any questions about registered reports at Computer Science Education, please feel free to contact Aleata, Eva and me. We are grateful to the editors-in-chief, Jan Vahrenhold and Brian Dorn for commissioning the special issue and for adding this as a permanent part of their journal.