IEEE Transactions on Robotics
Information for Associate Editors
1 General Information
Associate Editors (AEs), and the reviewers they recruit, play the most critical role in ensuring the quality and reputation of the IEEE Transactions on Robotics (T-RO). AEs are responsible for the fair and thorough review and evaluation of a paper’s originality, relevance, significance, technical correctness, organization and presentation, and references in a timely fashion. The purpose of this document is to describe typical duties, procedures, and best practices for T-RO AEs.
2 Role and Responsibilities
AEs are selected based on their scientific quality and judgment; international reputation; likely ability to recruit reviewers; diversity in research topic expertise, geography, and gender; and commitment to service and reliability. Typically successful AE candidates have demonstrated their active involvement in the scientific life of the IEEE Robotics and Automation Society (RAS), including publications in RAS journals and conferences.
The initial term of an AE is one year, which can be extended by two more years after a one-year checkpoint review conducted by the Senior Editors Panel. Terms may be extended by an additional year in cases of special needs. AEs can also serve for additional terms, but it is normally required that the AE takes a break for at least one year before rejoining the editorial board. This procedure gives opportunities to other members of the scientific community to participate in the editorial board.
An AE may be assigned to handle the review of a new paper up until the last day of their term. This means that an AE’s service generally continues beyond the official end of their term. An AE will handle the review of a paper until its final disposition (accept, reject, or revise and resubmit), so the revision of a conditionally accepted paper originally handled by the AE will be assigned to the AE even if their term has nominally ended. A revised and resubmitted paper is treated as a new paper, so it will not be assigned to the original AE if their term has expired.
The main responsibilities of AEs are:
- supervising the review process of papers assigned to them by the Editors;
- selecting Reviewers for the assigned papers, based on their scientific expertise and their reliability in providing timely and useful reviews;
- ensuring the timeliness and quality of the review process, by sending reminders and interacting with Reviewers when needed; and
- providing a recommendation to the supervising Editor by producing a report that builds on the Reviewers’ comments and includes the AE’s opinion on the paper.
While Editors strive to assign each paper to an AE with an interest in the paper’s topic, this is not always possible, for load balancing and other reasons. AEs must be able to handle the review of papers outside their primary area of expertise.
AEs are encouraged to familiarize themselves with the T-RO website, particularly Information for Authors, Submission Procedures, and paper types (Regular and Short), to better understand T-RO’s requirements of paper submissions.
3 The T-RO Review Process
The T-RO review process is managed using the PaperPlaza system. (“PaperPlaza” and “PaperCept” refer to the same submission and review management system, and the names are used interchangeably. Technically, the first is the name of the system, and the second is the name of the company under which it has been developed. Also, “paperplaza.net” is a domain name used by the IEEE Control Systems Society, and “papercept.net” is the domain name used by IEEE RAS.) PaperPlaza provides a wide variety of tools to help AEs manage the review process. Reviewer assignments, review entry, and AE reporting are all managed using PaperPlaza. To access the system, go to the PaperPlaza page and log in to T-RO. If you have forgotten your login information, you can retrieve it using the PIN management page. It may be useful to spend a few minutes looking over the help pages, and you can use the User’s Guide as a reference.
3.1 Summary of the Review Process
The Senior Editorial Board (consisting of the Editors, who are also called Senior Editors, and the Editor-in-Chief) conduct a preliminary evaluation of each submission. Some submissions are rejected or returned at this stage, usually because the submission violates a T-RO policy, clearly does not advance the state of the art, or is outside the scope of T-RO. This evaluation is typically concluded within two weeks of submission.
After that, the handling Editor assigns the paper to an AE. Within two weeks, the AE should have either recommended Editorial Reject or recruited Reviewers for a full review process (more details are given below). If the paper goes to full review, the target time to a decision is three months from the original submission date. This corresponds to approximately two weeks for the Senior Editorial review (Editor-in-Chief and Editor), two weeks for the AE to confirm Reviewers, five weeks to receive the reviews, one week for the AE to make a recommendation, and one week for the Editor to issue a decision. This leaves one to two weeks of buffer to accommodate late reviews or solicit emergency reviews when necessary.
Editorial rejects are often made by the Editor-in-Chief, but all other publication decisions are made by the Editors.
The highest priority is a fair and thorough review of each submission. Nearly as important for the reputation of the journal and the satisfaction of authors, however, is a timely decision.
3.1.2 “New” and “Evolved” Submissions
T-RO considers submissions that have evolved from previously-published conference papers. Such papers are said to be “evolved” (or to follow the “evolutionary publication model”). The journal submission must contain new results of substantive research significance and impact beyond the conference version. Authors of evolved papers must submit the previous conference paper(s) along with their submission, so it is available to the AE and reviewers, and they must provide a clear statement in the introduction of their journal submission explaining how the submission extends or is different from the earlier conference version(s). Failure to adhere to these guidelines, or failure to acknowledge the conference paper(s) from which the submission has evolved, should result in the paper being returned without review. More details can be found in the section on Submitting a Previously Published Conference Paper in the Information for Authors page of the T-RO website. Submissions that authors have disclosed as being evolved are annotated with the code “PCV” (“previous conference version”) in PaperPlaza.
If the Editor or AE determines that an evolved paper does not contain a “substantive” new contribution beyond the conference version, the paper should be returned. There is no single definition of a “substantive” new contribution, so there is some discretion, but example considerations can be found in the Submitting a Previously Published Conference Paper section in the Information for Authors page on the T-RO. If you are in doubt, please consult with the handling Editor, who may in turn wish to consult with the Editor-in-Chief.
If the submission is found to contain substantive differences or additions compared to the conference paper, the submission should proceed to review. In this case, the submission should be reviewed as if the conference paper did not exist. In other words, authors are free to incorporate the conference paper material however they wish without being penalized for overlap. This is the essence of the evolutionary publication model. Authors are responsible for ensuring that the copyright for the material being considered for T-RO publication either belongs to IEEE or can be transferred to IEEE.
Submissions that are not evolved from previous conference papers are called “new.” Example tests to distinguish “new” papers from “evolved” papers are given under the section Is My Paper "Evolved" or "New?" of the T-RO website. Authors of new papers published in T-RO are given the opportunity to present their paper at an upcoming conference. More details are given on the T-RO website.
3.2 Preliminary Steps for the AE
When a paper is assigned to an AE by an Editor, the AE should give the paper a quick read to roughly understand what it is about, its potential contributions, who the leading experts are in the topic area, and so on. If the submission has a “PCV” notation, the AE should check to see that the submission has a statement of the difference from the conference paper and confirm that the journal submission has substantive changes from the conference version (see the discussion above). If the authors disclose that the submission has evolved from a closely related conference paper (or if it later becomes clear that this is the case), but the submission either does not give a clear statement of the changes to the paper or does not have the previous conference paper among the submitted files, the paper should be returned without further review. Similarly, if the difference between the submission and the conference paper(s) is not substantive (see the discussion above), the submission should be returned without further review.
The AE should also check the ithenticate report for possible cases of plagiarism. Potential plagiarism cases should be investigated and reported to the handling Editor.
After this initial screening, the paper will usually fall into one of three categories:
- The paper has the potential to provide a contribution. The AE should initiate the review process and start securing Reviewers (see more details below).
- The paper clearly fails to provide a contribution to the state-of-the-art, or it has some other major flaws (e.g., plagiarism, out of T-RO scope, etc.). The AE should submit an Editorial Rejection recommendation to the supervising Editor within two weeks of receiving the paper to allow a quick response to the authors. See the section below on Preparing a Summary Report.
- The AE is unable to handle the paper for some reason, such as a conflict of interest. The AE should quickly inform the supervising Editor (within 2-3 working days) so that the Editor can assign the paper to another AE.
3.3 Securing Reviewers
The AE should invite experts in the field to serve as Reviewers for the paper, with the goal of ultimately obtaining three high-quality reviews. Selecting appropriate Reviewers is critical to a high quality, timely review process. Candidate Reviewers may include leading researchers in the topic area, colleagues who are familiar with the topic, authors of papers cited in the submission’s bibliography, authors of relevant papers that have been previously accepted, and authors that have submitted a paper to T-RO on a related subject. A good mix of senior and junior Reviewers is desirable as they provide reviews from different perspectives and at different levels of detail. For example, a junior Reviewer may carefully check the various technical details of the paper, but may also lack the depth to assess the significance of the contribution, while a senior Reviewer may be in a position to judge whether the paper brings a real contribution to the community. A mix of Reviewers from different geographical regions is also desirable.
During the process of submitting a paper to T-RO, authors confirm that they are willing to perform up to three quality, timely reviews for T-RO. Therefore, in general, T-RO authors should be willing to provide reviews. (Note, however, that some authors perform other significant service for the community, such as serving as an AE for T-RO!) This commitment is also stated on the Information for Authors page of the T-RO website. If a candidate Reviewer declines to provide a review and fails to provide a recommendation of an alternate Reviewer, the AE should consider following up to ask if she/he has a postdoc or senior PhD student who could perform the review.
It is very important that AEs do their best to identify the “right” Reviewers from the beginning, even if this means spending extra time before sending out review requests. This initial effort will almost certainly be repaid later by high-quality reviews that will ease the whole decision process. Often it is easier to recruit Reviewers whose self-interest is served by providing the review, e.g., the paper is on a topic that they would really like to see or say something about. Personalizing the review invitation letter, for example by commenting on why the selected Reviewer is the right choice for the paper, may increase the likelihood that the candidate Reviewer accepts the invitation. Having Reviewers who are interested in going over the manuscript alleviates the need to repeatedly send reminders and minimizes the likelihood that the AE must write detailed comments to supplement shallow and not-to-the-point reviews.
Generally it is not recommended to ask another AE to serve as a Reviewer, unless that AE is especially well positioned to review the paper.
Finally, it is the responsibility of the AE to oversee the review process and make sure that invited Reviewers quickly confirm (or decline) to provide a review, and that reviews are provided in due time. The AE should periodically check the status of each Reviewer (confirmed or not, review sent or not) and send personalized reminder emails via PaperPlaza as needed. Auto-generated reminder emails are typically sent from PaperPlaza, but such emails are less effective at generating a response.
3.4 Overseeing the Review Process
Three high-quality reviews is the target. After receiving a review, it is the AE’s responsibility to check its quality. In case of a substandard review (e.g., too short, shallow, offensive, etc.), the AE should ask the Reviewer to update or correct the review. If this is not possible (e.g., the Reviewer refuses to improve the review), and three quality reviews have not been obtained, the AE should attempt to find an emergency Reviewer with a shorter deadline to prevent significant delays in generating a recommendation.
It is generally recommended that the AE not also serve as a Reviewer, since the AE can provide her/his opinions in the recommendation letter. In cases where the AE has expertise in the area of the paper, and obtaining a third quality review would unduly delay the recommendation, it is acceptable for the AE to serve as a third reviewer.
Reviewers should provide information to the authors that demonstrate errors in the paper or that will be useful in revision. Anything that would be useful to the authors should be in the portion of the review that is visible to the authors, not in the confidential comments to the Editorial Board.
Reviews should be specific. If the results are already known, references to earlier papers that contain these or similar results should be provided. If the reasoning is incorrect or vague, where and why should be indicated. If the paper should be rewritten, suggestions should be given as to which parts to delete, amplify, or modify.
Reviews should be constructive. The ultimate goal of the review process is to provide constructive feedback to the authors on how their work could be improved, not just by noting deficiencies but also by indicating how they might be mended, when possible. The AE is responsible for ensuring that reviews avoid any diminishing or disrespectful remarks, even if the review is negative.
One review structure that works well consists of:
- a summary of the paper’s contribution, to demonstrate the Reviewer’s understanding of the paper;
- high-level comments on topics such as the paper’s originality, relevance, importance, technical correctness, and organization; and
- low-level comments, such as typo corrections, comments on figures and grammar, etc.
To avoid confusion, the Reviewer’s acceptance or rejection recommendation should not be included in the comments to the authors.
Sometimes papers are passed on to senior graduate students for review. Some excellent reviews may result from this. It is important, though, that the quality and professionalism of the review be maintained, and the originally assigned Reviewer should check the review personally before submission. If needed, Reviewers may include mathematical formulas, figures, tables, etc., in a PDF file. In this case, the Reviewer should ensure that the PDF file does not include any metadata or other indications of the Reviewer’s identity. The PaperPlaza software scrubs metadata from PDF files, which may cause annotations (such as comment bubbles, highlights, etc.) to also be scrubbed, so comments should not be transmitted by annotations to the original paper.
3.5 Preparing a Summary Report
Once a sufficient set of reviews has been obtained for a paper, the AE prepares a summary report that synthesizes these reviews and makes a specific publication recommendation to the Editor. In general, at least three thorough reviews are required to make a recommendation; however, if two thorough reviews are available, if these reviews have been provided by trusted experts, and if the two reviews are in agreement, then two reviews can be considered a sufficient basis for submitting a summary report and recommendation.
The summary report to the authors should be written in a form that allows the Editor to directly cut-and-paste it into the decision letter to the authors. The report should be impartial, to-the-point, and constructive. It should contain a brief summary of the paper, an analysis of the reviews and the AE's own opinions, and the issues that the authors must address in a revision if the paper may receive a "revise and resubmit" or "conditionally accept" decision. The comments to the authors should not give the AE's publication recommendation, since the Editor could issue a different decision. (The confidential comments to the Editor is the place to mention the publication recommendation, if needed.) If the recommended revisions include shortening the paper, specific suggestions should be given as to how to shorten the paper. A request to shorten the paper should be realistic. For example, If the recommendation is to shorten the paper from 12 pages to eight pages, it is not reasonable to ask for new results while providing little guidance on how to shrink the paper.
The AE's summary report should not include the publication recommendations of the reviewers, and it should include neither their ratings nor their review text. The ratings and reviewer recommendations are for internal use only, and the reviewers' reports to the authors are made available separately by PaperPlaza.
Remember, the AE’s job is not simply to parrot the Reviewers’ comments and average their recommendations. AEs have been chosen for their technical judgment, and this must be exercised when writing a summary report. Reviewers' opinions may be in conflict, some comments are more important than others, and some comments may be incorrect or safely ignored. The AE’s job is to generate a focused synthesis of the most important points and to provide clear feedback to the authors. AEs are also encouraged to do a quick search in IEEE Xplore to see if relevant T-RO papers have been cited and to suggest the authors examine or cite these papers as appropriate.
In addition to the summary report, the AE makes a publication recommendation to the Editor. PaperPlaza codes for the common recommendation types are AR and AS for Accept as Regular and Accept as Short; CR and CS for Conditionally Accept as Regular and Conditionally Accept as Short; RR and RS for Revise and Resubmit as Regular and Revise and Resubmit as Short; R for Reject; and RE for Editorial Reject.
The difference between a Short paper and a Regular paper is the number of pages needed to succinctly and effectively convey the contribution. A T-RO paper must contain a significant contribution regardless of the paper’s length. Short papers are not intended to be “less important” than Regular papers.
3.5.1 Accept (AR or AS)
When a paper is accepted, small modifications to the paper can be suggested but none is required prior to publication. The next step is for the authors to submit their final manuscript for proofing and publication. In practice, an Accept decision is rarely issued to the first version of a paper, but it is certainly allowed.
3.5.2 Conditional Accept (CR or CS)
This decision indicates that the paper is not yet ready for publication, but that it requires only minor modifications. If a paper is conditionally accepted, a revised manuscript must be submitted within 60 days of the date of decision. The revision has the same paper ID and the additional review cycle should take two months or less, requiring only the intervention of the previously assigned AE and, possibly, a subset of the previous reviewers (typically the ones asking for modifications).
It is inappropriate to issue a Conditionally Accept as Short recommendation for a long paper that would require trimming multiple pages. A significant change like this requires a Revise and Resubmit as Short decision.
The submission of a revision to a Conditionally Accepted paper must be accompanied by a separate file with a clear response to the AE’s summary and the reviews, including indications of how the paper has been modified in response.
3.5.3 Revise and Resubmit (RR or RS)
This decision provides a way to reject a paper while indicating to the authors that it may contain publishable results after a major revision. A revised manuscript is assigned a new paper ID, it is treated as a new submission, and it must go through a full review process. A reference within the system to the original paper and its reviews is preserved to facilitate the new review process. The Editor and AE who handled the previous version are often assigned to the revision, but this is not guaranteed.
Repeated Revise and Resubmit decisions should be avoided; after a first Revise and Resubmit, the second decision should be either (conditional) acceptance or rejection. A second Revise and Resubmit iteration requires a strong justification.
The submission of a revision to a paper with a Revise and Resubmit decision must be accompanied by a separate file with a clear response to the AE’s summary and the reviews, including indications of how the paper has been modified in response.
Do not use the Revise and Resubmit recommendation as a “soft” rejection. The Revise and Resubmit recommendation should be reserved for papers with real promise to be accepted after an appropriate revision. If the paper lacks this promise, the recommendation should be Reject. If an AE issues a Revise and Resubmit recommendation, there is a good chance the AE will handle a future version of the paper.
Be judicious in the use of the Revise and Resubmit as Short recommendation. In general, avoid the use of a Revise and Resubmit as Short recommendation for a Regular submission unless the paper would be improved by shortening. Do not use this recommendation simply to express reservations about the significance of the results. If the results will not be significant enough for publication in T-RO even after revision, the submission should be rejected. Revise and Resubmit as Short recommendations should be accompanied by a report that is specific as to how the paper should be shortened to meet the page limit for a Short paper.
Do not be biased to issue an Accept or Conditionally Accept recommendation to the revision of a paper that received a Revise and Resubmit decision. There is a natural tendency to “reward” authors with an Accept or Conditionally Accept decision if they have invested effort to revise their paper. There should be no such bias in reviewing the paper, which is effectively a new paper.
3.5.4 Reject (R)
T-RO is a comprehensive robotics journal with a limited page budget, and we can only publish novel papers that report on contributions of high research significance and impact, on topics that are of interest to the robotics research community. Reject is the appropriate decision for papers that do not meet these criteria.
Nothing prevents authors from resubmitting a revised version of a rejected manuscript, but the Reject decision signals to authors that a mere revision is also likely to be rejected, so a revision is discouraged. If authors do submit a revision, the submission must be accompanied by a separate file with a clear response to the AE’s summary and the reviews, including indications of how the paper has been modified in response.
3.5.5 Editorial Reject (RE)
AEs have the option to recommend rejection without sending the paper to reviewers. This option is appropriate in cases where the scientific content of the paper is clearly inappropriate for T-RO, a major technical error is found, the material does not advance the state of the art, and so on. The AE should provide a rationale for the decision, summarizing the paper and clearly explaining why it is not suitable for T-RO. As always, the report should avoid any disrespectful remarks and should be authoritative in its reasoning and conclusion. If the supervising Editor agrees with the analysis, the Editor will issue the decision and transmit the AE’s report to the authors, possibly with modifications. Such decisions should be issued quickly, within two weeks of assignment of the paper to the AE.