Follow these steps to prepare to accept paper submissions.

1. Set up PC member accounts. Many PCs are divided into classes, such as “heavy” and “light”, or “PC” and “ERC”. Mark these classes with user tags. It’s also useful to configure tag colors so that PC member names are displayed differently based on class (for instance, heavy PC member names might appear in bold).

2. Set submission policies, including whether submission is blind.

3. Set submission deadlines. Authors first register, then submit their papers, possibly multiple times; they choose for each submitted version whether that version is ready for review. Normally, HotCRP allows authors to update their papers until the deadline, but you can also require that authors “freeze” each submission explicitly; only administrators can update frozen submissions. The only deadline that really matters is the paper submission deadline, but HotCRP also supports a separate paper registration deadline, which will force authors to register a few days before they submit. An optional grace period applies to both deadlines: HotCRP reports the deadlines, but allows submissions and updates post-deadline for the specified grace period. This provides some protection against last-minute server overload and gives authors some slack.

4. Set up the submission form, including whether abstracts are required, whether authors check off conflicted PC members (“Collect authors’ PC conflicts with checkboxes”), and whether authors must enter additional non-PC collaborators, which can help detect conflicts with external reviewers (“Collect authors’ other collaborators as text”). The submission form also can include:

   • PDF format checker. This adds a “Check format” link to the Edit Paper screen. Clicking the link checks the paper for formatting errors, such as going over the page limit. Papers with formatting errors may still be submitted, since the checker itself can make mistakes, but the automated checker leaves cheating authors no excuse.

   • Options such as checkboxes, selectors, freeform text, and uploaded attachments. Checkbox options might include “Consider this paper for the Best Student Paper award” or “Provide this paper to the European shadow PC.” Attachment options might include supplemental material. You can search for papers with or without each option.

   • Topics. Authors can select topics, such as “Applications” or “Network databases,” that characterize their paper’s subject areas. PC members express topics for which they have high, medium, and low interest, improving automatic paper assignment. Although explicit preferences (see below) are better than topic-based assignments, busy PC members might not specify their preferences; topic matching lets you do a reasonable job at assigning papers anyway.

5. Take a look at a paper submission page to make sure it looks right.

6. Open the site for submissions. Submissions will be accepted only until the listed deadline.

After the submission deadline has passed:

1. Consider checking the papers for anomalies. Withdraw and/or delete duplicates or update details on the paper pages (via “Edit paper”). Also consider contacting the authors of papers that were never officially submitted, especially if a PDF document was uploaded; sometimes a user will uncheck “The paper is ready for review” by mistake.

2. Check for formatting violations (optional). Search > Download > Format check will download a summary report. Serious errors are also shown on paper pages (problematic PDFs are distinguished by an “X”).

3. Prepare the review form. Take a look at the templates to get ideas.

4. Set review policies and deadlines, including reviewing deadlines, whether review is blind, and whether PC members may review any paper (usually “yes” is the right answer).

5. Prepare tracks (optional). Tracks give chairs fine-grained control over PC members’ access rights for individual papers. Example situations calling for tracks include external review committees, PC-paper review committees, and multi-track conferences.

6. Collect review preferences from the PC. PC members can rank-order papers they want or don’t want to review. They can either set their preferences all at once, or (often more convenient) page through the list of submitted papers setting their preferences on the paper pages.

   If you’d like, you can collect review preferences before the submission deadline. Select “PC can see all registered papers until submission deadline”, which allows PC members to see abstracts for registered papers that haven’t yet been submitted.

7. Assign conflicts. HotCRP automatically installs the authors’ declared conflicts. HotCRP does not automatically install other conflicts, such as conflicts indicated by PC members’ “Collaborators and other affiliations” or their review preferences. Use the manual assignment tool to search for potential missing conflicts, and use the automatic assigner to assign conflicts when PC members have entered preferences of −100 or less.

8. Assign reviews. You can make assignments by paper, by PC member, by uploading an assignments file, or, even easier, automatically. PC review assignments can be “primary” or “secondary”; the difference is that primary reviewers are expected to complete their review, but a secondary reviewer can delegate their review to someone else. You can also assign PC “metareviews”. Unlike normal reviewers, a metareviewer can view all other reviews before submitting their own.

   The default assignments pages apply to all submitted papers. You can also assign subsets of papers obtained through search, such as papers with fewer than three completed reviews.

9. Open the site for reviewing.

Chairs and system administrators can access any information stored in the conference system, including reviewer identities for conflicted papers. It is easiest to simply accept such conflicts as a fact of life. Chairs who can’t handle conflicts fairly shouldn’t be chairs. However, HotCRP does offer other mechanisms for conflicted reviews.

The key step is to pick a PC member to manage the reviewing and discussion process for the relevant papers. This PC member is called the paper administrator. Use the left-hand side of the paper assignment pages to enter paper administrators. (You may need to “Override conflicts” to access the assignment page.) A paper’s administrators have full privilege to assign and view reviews for that paper, although they cannot change conference settings.

Assigned administrators change conflicted chairs’ access rights. Normally, a conflicted chair can easily override their conflict. If a paper has an administrator, however, conflicts cannot be overridden until the administrator is removed.

Paper administrators make life easy for PC reviewers while hiding conflicts from chairs in most circumstances. However, determined chairs can still discover reviewer identities via HotCRP logs, review counts, and mails (and, of course, by removing the administrator). For additional privacy, a conference can use review tokens, which are completely anonymous review slots. To create a token, an administrator goes to an assignment page and clicks on “Request review” without entering a name or email address. This reports the token, a short string of letters and numbers such as “9HDZYUB”. Any user who knows the token can enter it on HotCRP’s home page, after which the system lets them view the paper and anonymously modify the corresponding “Jane Q. Public” review. True reviewer identities will not appear in HotCRP’s database or its logs. For even more privacy, the paper administrator could collect offline review forms via email and upload them using review tokens; then even web server access logs store only the administrator’s identity.

1. Collect authors’ responses to the reviews (optional). Authors’ responses (also called rebuttals) let authors correct reviewer misconceptions before decisions are made. Responses are entered into the system as comments. On the decision settings page, update “Can authors see reviews” and “Collect responses to the reviews,” then send mail to authors informing them of the response deadline. PC members can still update their reviews up to the review deadline; authors are informed via email of any review changes. At the end of the response period you should generally turn off “Authors can see reviews” so PC members can update their reviews in peace.

2. Set PC can see all reviews if you haven’t already, allowing the program committee to see reviews and scores for non-conflicted papers. (During most conferences’ review periods, a PC member can see a paper’s reviews only after completing their own review for that paper. This supposedly reduces bias.)

3. Examine paper scores, either one at a time or en masse, and decide which papers will be discussed. The tags system lets you prepare discussion sets. Use search keywords to, for example, find all papers with at least two overall merit ratings of 2 or better.

4. Assign discussion orders using tags (optional). Common discussion orders include sorted by overall ranking (high-to-low, low-to-high, or alternating), sorted by topic, and grouped by PC conflicts. Explicit tag-based orders make it easier for the PC to follow along.

5. Assign discussion leads (optional). Discussion leads are expected to be able to summarize the paper and the reviews. You can assign leads either paper by paper or automatically.

6. Define decision types (optional). By default, HotCRP has two decision types, “accept” and “reject,” but you can add other types of acceptance and rejection, such as “accept as short paper.”

7. The night before the meeting, download all reviews onto a laptop (Download > All reviews) in case the Internet explodes and you can’t reach HotCRP from the meeting place.

1. The meeting tracker can keep the PC coordinated. Search for papers in whatever order you like (you may want an explicit discussion order). Then open a browser tab to manage the tracker, navigate to the first paper in the order, and select “☟” to activate the tracker. From that point on, PC members see a banner with the tracker tab’s current position in the order:

   

   You can also view the discussion status on the discussion status page.

2. Scribes can capture discussions as comments for the authors’ reference.

3. Paper decisions can be recorded on the paper pages or en masse via search. Use decision settings to expose decisions to PC members if desired.

4. Shepherding (optional). If your conference uses shepherding for accepted papers, you can assign shepherds either paper by paper or automatically.

1. Enter decisions and shepherds if you didn’t do this at the meeting.

2. Give reviewers some time to update their reviews in response to PC discussion (optional).

3. Set “Who can see decisions?” to “Authors, PC members, and reviewers.” This will also make shepherd names visible to authors.

4. Send mail to authors informing them that reviews and decisions are available. The mail can also contain the reviews and comments themselves.

5. Collect final papers (optional). If you’re putting together the program yourself, it can be convenient to collect final versions using HotCRP. Authors upload final versions just as they did submissions. You can then download all final versions as a .zip archive. (The submitted versions are archived for reference.)