[ Show as SlideShow ]
Partial Refereeing at Process IPAC Experience 2017
Contact: Ivan Andrian (Elettra-Sincrotrone Trieste S.C.p.A.)
Terminology/roles
- Author: submits
- Referee: reviews (multiple ref. per paper)
- Approver: decides when referees do not agree (Main Classification coordinator)
- Peer Review Board: supervises, coordinates, decides at last
Basic idea
- At abstract submission stage, any author can ask his paper to be considered for peer reviewing
- SPC/Peer Review Board accepts/rejects those proposals
- Every accepted paper get 2 referees assigned
- a maximum of two review iterations are foreseen
Numbers
- IPAC17 abstracts: 1426
- Peer review requested: 373 (26%)
- Peer review accepted: 123 (8.6%)
- Peer review published: 109 (7.6%)
- Max peer review papers allowed: 200 (14%)
- Goals: 3% PRAB, 15% PR, 82% JACoW.org
- https://oraweb.cern.ch/pls/ipac2017/referee.html
Timeline (referred to $ConfStart)
- - 20: authors to submit refereed paper (PDF)
- - 19: notification to approvers (MC coordinators)
- - 17: notification to authors about refereeing acceptance
- - 17: notification to referees of assignment
- - 13: reminder email to all referees
- - 11: referees to provide report
- - 10: approvers to review outcome and decide
Timeline (referred to $ConfStart)
- - 7: revised version from author
- - 6: reminder email to referees
- - 4 (Wed): upload of non-ref'd papers
- - 3: referees to provide updated report
- - 1: approvers to review outcomes/exceptions
- 0: $ConfStart (Sun): peer reviewing board checks any controversial issues and left over papers
Timeline (referred to $ConfEnd)
- + 27: (initial) deadline for submission to IOP
- + 44: all papers received by IOP
- + 61: all papers published by IOP
- good, used to be 3-4 months!
Benefits
- indexed by Scopus, Inspec, ISI Web of Science, Compendex and many others
- citations tracked via IOP publishing citing articles facility and Scopus
- 575 references in subset of 42 papers, many to journal articles
- e.g. 36 citations to PRAB. Extrapolation: expect about 1500 references, e.g. about 100 to PRAB
PRB thoughts...
- Need to improve email utility further
- e.g., allow rall parties to send email to each other, keeping anonymity
- High effort in referee assignment. With higher numbers, could be a mess
- Scalability issues
- <= 250 for IPAC2017
- <= 400 for IPAC2018?
- impossible over 50% of all
- Referees did great job, on time
PRB thoughts... (2)
- No automatic notification of referee nomination
- No check on possible referees in multiple MC
- Deadline of 3 weeks before heavy but necessary
IPAC2017 at IOP
http://iopscience.iop.org/issue/1742-6596/874/1
IOP search metrics
Conclusions
- Solid Peer Review Board is needed
- High workload even after the conference
- No impact on JACoW proceedings office
- Possible inconsistency on content of same paper (need to reformat for IOP)
- Evaluation of benefits needed in 5 years?
