Skip to yearly menu bar Skip to main content


ECCV 2024 Reviewer Guidelines

Thank you for volunteering your time to review for ECCV 2024! To maintain a high-quality technical program, we rely very much on the time and expertise of our reviewers. This document explains what is expected of all members of the Reviewing Committee for ECCV 2024. For further details, please consult the ECCV 2024 Reviewer Tutorial Slides.
 

AnchorReviewing Timeline

  • Monday, April 1, 2024: Reviewers receive assigned papers; review period starts
  • Friday, April 26, 2024: Deadline for submitting reviews 
  • Saturday, May 18, 2024: Reviewers receive author rebuttals; discussion period starts
  • Saturday, May 18 to Monday, May 27, 2024: Discussion phase
  • Monday, May 27, 2024: Deadline for submitting final reviewer recommendations

The reviewing deadlines will be STRICT. Reviewers who submit timely and high-quality reviews will be considered for outstanding reviewer awards and recommended for future conferences. 
 

Reviewing In a Nutshell

Each paper that is accepted should be technically sound and make a contribution to the field. Look for what is good or stimulating in the paper, and what knowledge advancement it has made. We recommend that you embrace novel, brave concepts, even if they have not been tested on many datasets. For example, the fact that a proposed method does not exceed the state-of-the-art accuracy on an existing benchmark dataset is not grounds for rejection by itself. Rather, it is important to weigh both the novelty and potential impact of the work alongside the reported performance. Minor flaws that can be easily corrected should not be a reason to reject a paper. Above all, you should be specific and detailed in your reviews. Your discussion, more than your score, will help the authors, fellow reviewers, and Area Chairs understand the basis for your recommendation. You should include specific feedback on ways the authors can improve their papers. Be sure to avoid committing common reviewer errors, e.g. novelty fallacy; consult the ECCV 2024 Reviewer Tutorial Slides for more details and examples.

Check your papers

As soon as you get your reviewing assignment, please go through all the papers to make sure that (a) you have no obvious conflict of interest (see “Avoid Conflicts of Interest” below) and (b) you feel comfortable reviewing the paper assigned. If issues with either of these points arise, please contact the Area Chair right away as instructed in the detailed emails you will receive during the process.

Know the policies

Please read the ECCV 2024 Submission Policies carefully to familiarize yourself with all official policies the authors are expected to follow. If you come to believe that a paper may be in violation of any of these policies, please contact the Area and Program Chairs. In the meantime, proceed to review the paper assuming no violation has taken place and describe the potential policy violation in the confidential comments field.
Please also carefully read the LLM policy for reviewers in the Reviewer FAQs below. Before using any LLM in conjunction with your review duties, make sure you understand what is allowed and what is not allowed.
 

Ethics for Reviewing Papers

1. Respect anonymity in the review process

Our Submission Guidelines have instructed authors to make reasonable efforts to hide their identities, including omitting their names, affiliations, and acknowledgments. This information will of course be included in the final published version of the manuscript. Reviewers should not take active steps to discover the identity of the authors, and make all efforts to keep their own identity invisible to the authors.

With the increase in popularity of arXiv preprints, sometimes the authors of a paper may already be known to the reviewer. Posting to arXiv is NOT considered a violation of anonymity on the part of the authors, and in most cases, reviewers who happen to know (or suspect) the authors’ identity can still review the paper as long as they feel that they can do an impartial job. An important general principle is to make every effort to treat papers fairly whether or not you know (or suspect) who wrote them. If you do not know the identity of the authors at the start of the process, DO NOT attempt to find out the authorship of a submission.

2. Protect ideas

As a reviewer for ECCV, you have the responsibility to protect the confidentiality of the ideas represented in the papers you review. ECCV submissions are not published documents. The work is considered new or proprietary by the authors; otherwise they would not have submitted it. Of course, their intent is to ultimately publish to the world, but most of the submitted papers will not appear in the ECCV proceedings. Thus, it is likely that the paper you have in your hands will be refined further and submitted to some other journal or conference. Sometimes the work is still considered confidential by the authors' employer(s). These organizations do not consider sending a paper to ECCV for review to constitute a public disclosure. Protection of the ideas in the papers you receive means:

  • You should not show the paper to anyone else, including colleagues or students, unless you have asked them to write a review, or to help with your review.
  • You should not show any results, videos/images, code, or any of the supplementary material to non-reviewers.
  • You should not use ideas/code from papers you review to develop your own ideas/code.
  • After the review process, you should destroy all copies of papers and supplementary material and erase any code that the authors submitted as part of the supplementary, and any implementations you have written to evaluate the ideas in the papers, as well as any results of those implementations.

3. Avoid conflicts of interest

As a reviewer of an ECCV paper, it is important for you to avoid any conflict of interest. There should be absolutely no question about the impartiality of any review. Thus, if you are assigned a paper where your review would create a possible conflict of interest, you should return the paper as early as possible and not submit a review. Conflicts of interest include (but are not limited to) situations in which:

  • You work at the same institution as one of the authors.
  • You have been directly involved in the work and will be receiving credit in some way. If you are a member of an author's thesis committee, and the paper is about his or her thesis work, then you were involved.
  • You suspect that others might perceive a conflict of interest in your involvement.
  • You have collaborated with one of the authors in the past three years (more or less). Collaboration is usually defined as having written a paper or grant proposal together, although you should use your judgment.
  • You were the MSc/PhD advisor or advisee of one of the authors. Most funding agencies and publications typically consider advisees to represent a lifetime conflict of interest. ECCV, similarly to other computer vision conferences, has traditionally been more flexible than this, but you should think carefully before reviewing a paper you know to be written by a former advisor or advisee, especially a recent one.

While the organizers make every effort to avoid such conflicts in the review assignments, they may nonetheless occasionally arise. If you recognize the work or the author and feel it could present a conflict of interest, contact the Area Chair as soon as possible so they can find someone else to review it.

4. Be professional

Belittling or sarcastic comments have no place in the reviewing process. The most valuable comments in a review are those that help the authors understand the shortcomings of their work and how they might improve it. Write a courteous, informative, incisive, and helpful review that you would be proud to sign with your name (were it not anonymous).
 

How to Write Good Reviews

  • Take the time to write good reviews. Ideally, you should read a paper and then think about it over the course of several days before you write your review.
  • Short reviews are unhelpful to authors, other reviewers, and Area Chairs. If you have agreed to review a paper, you should take enough time to write a thoughtful and detailed review. Bullet lists with one short sentence per bullet are NOT a detailed review.
  • Be specific when you suggest that the writing needs to be improved. If there is a particular section that is unclear, point it out and give suggestions for how it can be clarified.
  • Be specific about novelty. Claims in a review that the submitted work “has been done before” MUST be backed up with specific references and an explanation of how closely they are related. At the same time, for a positive review, be sure to summarize what novel aspects are most interesting in the Strengths section.
  • Do not reject papers solely because they are missing citations or comparisons to prior work that has only been published without review (e.g., arXiv or technical reports). Refer to the FAQ below for more details on handling arXiv prior art.
  • Do not give away your identity by asking the authors to cite several of your own papers.
  • If you think the paper is out of scope for ECCV's subject areas, clearly explain why in the review. Then suggest other publication possibilities (journals, conferences, workshops) that would be a better match for the paper. However, unless the area mismatch is extreme, you should keep an open mind, because we want a diverse set of good papers at the conference.
  • The tone of your review is important. A harshly written review will be resented by the authors, regardless of whether your criticisms are true. If you take care, it is always possible to word your review constructively while staying true to your thoughts about the paper.
  • Avoid referring to the authors in the second person (“you”). It is best to avoid the term “the authors” as well, because you are reviewing their work and not the person. Instead, use the third person (“the paper”). Referring to the authors as “you” can be perceived as being confrontational, even though you may not mean it this way.
  • Be generous about giving the authors new ideas for how they can improve their work. You might suggest a new technical tool that could help, a dataset that could be tried, an application area that might benefit from their work, or a way to generalize their idea to increase its impact.
  • Please refer to the Ethics Guidelines and Suggested Practices for Authors page to find out how to handle some specific issues that may arise.

Finally, keep in mind that a thoughtful review not only benefits the authors, but also yourself. Your reviews are read by other reviewers and especially the Area Chairs, in addition to the authors. Unlike the authors, the Area Chairs know your identity. Being a helpful reviewer will generate good will towards you in the research community – and may even help you to win an Outstanding Reviewer award.
 

Reviewer FAQs

Q. Is there a minimum number of papers I should accept or reject?

A. No. Each paper should be evaluated in its own right. If you feel that most of the papers assigned to you have value, you should accept them. It is unlikely that most papers are bad enough to justify rejecting them all. However, if that is the case, provide clear and very specific comments in each review. Do NOT assume that your stack of papers necessarily should have the same acceptance rate as the entire conference ultimately will.

Q. Can I review a paper I already saw on arXiv and hence know who the authors are?

A. In general, yes, unless you are conflicted with one of the authors. See next question below for guidelines.

Q. How should I treat papers for which I know the authors?

A. Reviewers should make every effort to treat each paper impartially, whether or not they know who wrote the paper. For example: It is not OK for a reviewer to read a paper, think “I know who wrote this; it's on arXiv; they are usually quite good” and accept the paper based on that reasoning. Conversely, it is also not OK for a reviewer to read a paper, think “I know who wrote this; it's on arXiv; they are no good” and reject the paper based on that reasoning.

Q. There are well-established conferences with a rigorous peer review process, like ICIP and ICASSP, that publish proceedings with four-page papers. Suppose that a reviewer identifies a prior ICASSP or ICIP paper that has substantial overlap with an ECCV submission. Should this ICASSP/ICIP paper not be considered a “previous publication” under the ECCV 2024 Dual Submission policy?

A. If the prior paper is within four pages, the ECCV submission in question is not in violation of the Dual Submission policy and will not be administratively rejected. However, the reviewer still needs to use their judgment to determine whether the submission offers enough additional value to warrant acceptance at ECCV. If it does not add enough value over prior work, it can still be rejected on substantive, not on policy, grounds. Additionally, depending on the specifics of the case, it may be relevant whether the authors of the prior ICASSP/ICIP paper are the same or different from those of the current ECCV submission. If there is a significant possibility that the authors of the present submission might be different from those of the prior paper, in which case this could be an instance of plagiarism, then the reviewer should contact the Area Chair to investigate this possibility.

Q. Should authors be expected to cite related arXiv papers or compare their results?

A. Consistent with good academic practice, the authors should cite all sources that inspired and informed their work. This said, asking authors to thoroughly compare their work with arXiv reports that appeared shortly before the submission deadline imposes an unreasonable burden. We also do not wish to discourage the publication of similar ideas that have been developed independently and concurrently. Reviewers should keep the following guidelines in mind:

  • Authors are not required to discuss and compare their work with recent arXiv reports, although they should properly acknowledge those that directly and obviously inspired them.
  • Failing to cite an arXiv paper or failing to beat its performance SHOULD NOT be SOLE grounds for rejection.
  • Reviewers SHOULD NOT reject a paper solely because another paper with a similar idea has already appeared on arXiv. If the reviewer suspects plagiarism or academic dishonesty, they are encouraged to bring these concerns to the attention of the Area and Program Chairs.
  • It is acceptable for a reviewer to suggest that an author should acknowledge or be aware of something on arXiv.

Q. How should I treat the supplementary material?

A. The supplementary material is intended to provide details of derivations and results that do not fit within the paper format or space limit. Ideally, the paper should indicate when to refer to the supplementary material, and you need to consult the supplementary material only if you think it is helpful in understanding the paper and its contribution. According to the Submission Policies, the supplementary material MAY NOT include results obtained with an improved version of the method (e.g., following additional parameter tuning or training), or an updated or corrected version of the submission PDF. If you find that the supplementary material violates these guidelines, please contact the Area Chair.

Q. Can I request additional experiments in the authors' rebuttal? How should I treat additional experiments reported by authors in the rebuttal?

A. In your review, you may request clarifications or additional illustrations in the rebuttal. Reviewers should not request substantial additional experiments for the rebuttal, or penalize for lack of additional experiments. “Substantial” refers to what would be needed in a major revision of a paper. The rebuttal may include figures with illustrations or comparison tables of results reported in the submission/supplemental material or in other papers. However, papers should also not be penalized for supplying extra results; you can simply choose to ignore them.

Q. If a social media post shared information on an ECCV submission, does that signal a violation of anonymity?

A. No, it does not. A violation only occurs when an author or the corresponding paper, its arXiv page, its project page, its accompanying video, etc. explicitly identifies the paper as an ECCV submission. 

Q. A paper is using a withdrawn dataset, such as DukeMTMC-ReID or MS-Celeb-1M. How should I handle this?

A. Reviewers are advised that the choice to use a withdrawn dataset, while not in itself grounds for rejection, should invite very close scrutiny. Reviewers should flag such cases in the review form for further consideration by Area Chairs and Program Chairs. Consider questions such as: Do the authors explain why they had to do this? Is this explanation compelling? Is there really no alternative dataset that could have been used? Remember, authors might simply not know the dataset had been withdrawn. If you believe the paper could be accepted without the authors’ use of a withdrawn dataset, then it is natural to advise the authors to remove the experiments associated with this dataset.

Q. If a paper did not evaluate on a withdrawn dataset, can I request authors that they do?

A. It is a violation of policy for a reviewer or Area Chair to require comparison on a dataset that has been withdrawn.

Q. A paper is claiming a dataset as one of its contributions. How should I evaluate this claim?

A. If a paper submission is claiming a dataset as one of its contributions, there should be a reasonable expectation that the dataset will be made publicly available upon publication. You should use your judgment to evaluate the dataset claim accordingly. Note that this does NOT imply that all datasets used in ECCV submissions must be public, or that papers relying on non-public datasets must be rejected. The use of private or otherwise restricted datasets (e.g., for training or experimentation) DOES NOT constitute grounds for rejection. However, private or otherwise restricted datasets cannot be claimed as contributions in their own right, and you must evaluate the papers based on their other technical merits.

Q. What is the LLM Policy for referees in ECCV 2024?

A. Referees may use any device, including a Large Language Model (LLM), to polish their review wording, but must vouch for, and be responsible for, the accuracy of the review. It is a significant act of referee misconduct to allow an LLM to see a submission. The Program Chairs interpret showing a submission to an LLM as a deliberate referee violation of confidentiality.  The PCs reserve the right to report reviewer misconduct to future computer vision conferences. These conferences then may take actions, e.g., there was a recent PAMI-TC vote that CVPR reviewer misconduct may lead to a 2-year submission ban.

Q. What is the reasoning behind the LLM policy?

A. The action that most likely affects the credibility of ECCV is using an LLM to write the summary of the paper.  The summary is a necessary part of reviewing, because it compels the reviewer to show what they think the paper is about, and hence validates the review.  A summary that has been polished by an LLM that hasn't seen the paper is acceptable, as long as the referee vouches for the review.  A summary that has been prepared by an LLM – i.e. one where the LLM sees the paper – is profoundly damaging, because it may allow a review to be prepared without the referee trying to understand the paper.

Q. How will the LLM policy be implemented?

A. An author may complain to their Area Chair that a summary (and/or other parts of the review) have been prepared by an LLM that has seen the paper.  Such a complaint would need to be supported by an example summary (or other part of the review) prepared by the author giving the paper to an LLM. If this matches the referee’s comments sufficiently, the AC will pass the complaint on to the Program Chairs who are then entitled, but not required, to act. Complaints must not appear on the rebuttal PDF, but should be submitted separately as confidential comments to the Area Chair. The PCs strongly discourage frivolous complaints. Authors should be aware that a complaint to an AC about a review prepared by an LLM without reasonable evidence in support of that complaint, is wasting the ACs time.