An overview of statistics:

  • In total, 1621 submissions were received right after the submission deadline: 1045 long, 576 short papers.
  • 13 erroneous submissions were deleted or withdrawn in the preliminary checks by PCs.
  • 25 papers were rejected without review (16 long, 9 short); the reasons are the violation of the ACL 2018 style guideline and dual submissions.
  • 32 papers were withdrawn before the review period starts; the main reason is that the papers have been accepted as the short papers at NAACL.
  • In total, 1551 papers went into the reviewing phase: 1021 long, 530 short papers
  • 1443 reviewers are involved in the reviewing process; each reviewer reviews about 3 papers on average.

Detailed statistics by area:

Area Area Chairs Submissions Reviewers Per reviewer
Dialogue and Interactive Systems 4 90 99 2.72
Discourse and Pragmatics 2 46 58 2.38
Document Analysis 3 104 92 3.39
Generation 2 59 64 2.77
Information Extraction and Text Mining 6 179 161 3.34
Linguistic Theories, Cognitive Modeling and Psycholinguistics 2 24 59 1.22
Machine Learning 3 115 96 3.59
Machine Translation 4 106 129 2.47
Multidisciplinary (also for AC COI) 3 76 Shared with other areas NA
Multilinguality 1 32 27 3.56
Phonology, Morphology and Word Segmentation 2 27 48 1.69
Question Answering 4 80 80 3.00
Resources and Evaluation 3 72 78 2.77
Sentence-level Semantics 3 90 73 3.70
Sentiment Analysis and Argument Mining 3 86 91 2.84
Social Media 2 61 55 3.33
Summarization 2 52 50 3.12
Tagging, Chunking, Syntax and Parsing 4 61 77 2.38
Textual Inference and Other Areas of Semantics 2 58 53 3.28
Vision, Robotics, Multimodal, Grounding and Speech 3 54 46 3.52
Word-level Semantics 2 79 71 3.34

Recruiting area chairs (ACs) and reviewers:

  • Recruit area chairs (Sep - Oct 2017): the programme co-chairs (PCs) first decided on a list of areas and estimated the number of submissions for each area, and then proposed a short list of potential candidates of ACs in each area. Candidates who have accepted the invitation constitute the AC committee.
  • Look for potential reviewers (Sep - Oct 2017): PCs sent out reviewer nomination requests in Sep 2017 to look for potential reviewers; 936 nominations were received by Nov 2017. In addition, PCs also used the reviewers list of major NLP conferences in previous one or two years and ACs nominations to look for potential reviewers. Our final list of candidates consists of over 2000 reviewers.
  • Recruit reviewers (Oct - Dec 2017): the ACs use the candidate reviewers list to form the shortlist for each area and invite the reviewers whom ACs selected. 1510 candidates were invited in this first round, and ACs continued inviting reviewers when they needed.
  • After the submission deadline: several areas received a significantly larger number of submissions than the estimation. PCs invited additional ACs for these areas, and also ACs invited additional reviewers as necessary. Finally, the Program Committee consists of 60 ACs and 1443 reviewers.

Assigning papers to areas and reviewers:

  • First round: Initial assignments of papers to areas were determined automatically by authors’ input, while PCs went through all submissions and moved papers to other areas, considering COI and the topical fit. PCs assigned one AC as a meta-reviewer to each paper using TPMS scores.
  • Second round: ACs looked into the papers in their area, and adjust meta-reviewer assignments. ACs send a report to PCs if they found problems.
  • Third round: PCs made the final decision, considering workload balance, COI and the topical fit.
  • Fourth round: ACs decided which reviewers will review each paper, based on AC’s knowledge about the reviewers, TPMS scores, reviewers’ bids, and COI.

Deciding on the reject-without-review papers:

  • PCs went through all submissions in the first round, and then ACs looked into each paper in the second round and reported any problems
  • For each suspicious case, intensive discussions took place between PCs and the corresponding ACs, to make final decisions