Best Practices for Managing More Efficient and Accurate Document Reviews

To borrow from Thomas Paine, in the electronic information age, document review “even in its best state, is but a necessary evil; in its worst state, an intolerable one.” The purpose of this article is to take a step back from the swirling complexities of modern litigation discovery and provide an overview of the basics of the document review process.

Attorneys engaging in this time-consuming and costly process, from those reviewing documents to those managing the reviewers, must be equipped with proper training guidelines to ensure that the review proceeds effectively, efficiently, and with maximum accuracy. Such guidelines include the importance of conducting productive custodian interviews, drafting and updating a written document review manual, and implementing quality control mechanisms. While not every document can be deemed responsive with 100 percent accuracy, nor every complex privilege determination discerned on a first-pass review, certain methods can and should be used to aid in a meaningful review process, while also promoting time and cost management.

Document Collection and Custodian Interviews

Like most successful endeavors, any effective document review process begins with advance preparation. Ideally, counsel should understand the information and documents they are likely to receive through custodian interviews and discussions with a client’s information technology department before they receive documents.

Locating all sources of relevant electronically stored information (ESI) poses one of the biggest logistical challenges. ESI may reside on virtually any electronic medium, ranging from computer hard drives to backup tapes and even, in more recent years, self-destructing messaging applications. Employee self-collection may have perceived cost benefits, but is generally fraught with potential for inefficiencies and abuse and is frowned upon by courts. Accordingly, counsel should interview key witnesses and the client’s IT personnel to determine sources of potentially relevant ESI. In addition, specific and careful preservation protocols (including litigation hold) must always be timely implemented to ensure that any review process starts with a data set that can reasonably be expected to contain the universe of accessible, relevant and responsive information.

Beginning the Document Review

Once counsel identifies and collects data/documents from the relevant ESI sources, they must review that information primarily for responsiveness and privilege before producing it to the requesting party. Undertaking a careful and purposeful review before production is essential to ensure that irrelevant documents, privileged communications, trade secrets, or other commercially sensitive data are not disclosed.

Obviously, the document review team must be knowledgeable about both the case and the document review platform. Comprehensive training, performed by designated “subject matter experts” on the litigation team, should cover case background, key issues, examples of important documents and relevant document requests, and the review platform. The review team should fully understand, among other things, how to define what is responsive to discovery requests and what information mandates a privileged designation. As part of this training, it may be helpful to discuss specific documents and walk through the appropriate tagging. Cutting corners on education of the review team risks an unreliable outcome, successful challenges by the adversary, and a very costly remedial review.

The Document Review Manual/Protocol

When conducting a linear review of datasets, it is beneficial to prepare a memorandum and checklists to assist reviewers in making consistent determinations. Among other things, a comprehensive document review manual (sometimes referred to as a protocol) should provide an overview of the background of a case, clarify the key factual and legal issues and the subject/scope of requests, and identify key players and their roles.

The manual should also include a list of document “tags” or “codes” to be used by the reviewers, which will correspond with the tags available on the document review platform. These allow the reviewers to tag each document for relevance/responsiveness, significance to the case, privilege, confidentiality, and case-specific issues. The coding should be grouped into fields in order to minimize confusion and the time reviewers spend scrolling through the coding panel. For example, a coding system could appear as follows:

1. Relevance
   • Relevant
   • Not Relevant
   • Hot Document
   • Further Review Required
2. Privilege
   • Attorney-Client Privileged
   • Work Product
   • Not Privileged
3. Confidentiality
   • Confidential
   • Attorneys’ Eyes Only
   • Not Confidential
4. Issue Tags
   • [These tags will be specific to the issues or document requests in each case]

Additional coding fields can be used, such as those providing for a “free text” (narrative) description for a privilege assertion, keeping in mind that the review process is invariably slowed by the invitation to reviewers to add free text in the coding process.

When establishing coding, counsel should determine whether fields should be mandatory or optional. A mandatory field, such as “relevance,” requires a reviewer to make a selection in that field before moving on to the next document. Mandatory fields should be used when the analysis applies to most or all of the documents. There should be at least one mandatory field to be used to ultimately determine if a document has been reviewed and can be moved along to the next step of the process. An optional field can be used for a field such as Issue Tags if that type of analysis applies only to a subset of documents in the review set.

In certain tagging groups, e.g., Relevance and Confidentiality, reviewers should be permitted to make only one selection, which are generally set up as “radio buttons.” In others, such as Privilege and Issue Tags, reviewers may be permitted to make more than one selection as appropriate.

Typically, reviewers should be instructed whether a document tagged as “Not Relevant” should also be tagged in the other categories; not coding the other categories will save time during the review, but a completely-coded document will allow for more accuracy in producing full families and save time if the document is later found to be relevant.

Counsel should update the document review manual periodically as they learn more about the case, as new members join the review team, and as instructions are amended. Finally, while the manual is privileged work product and should be designated accordingly, counsel should also understand that, like other privileged documents relevant to the electronic discovery process (i.e., litigation hold notices), the manual itself, or elements of it, may be discoverable or voluntarily disclosed at some point to address challenges to the review and production process.

Managing the Document Review

The attorneys managing the document review must confirm that document productions are accurate and complete. They should meet often with the review team to discuss questions about specific documents or tags, as well as common trends that the reviewers have observed.

There are various ways to make a document review more efficient, such as reviewing potentially privileged documents at the outset, batching documents by search terms, email-threading, and using privilege categories. In addition, Technology Assisted Review (TAR), which uses an algorithm to leverage decisions made by an attorney reviewer with significant knowledge of the case regarding a subset of documents to make predictions regarding each document in the data set, can be an extremely effective means of making ESI reviews more accurate and cost-effective.

The managing attorneys should maintain a log of decisions made, both as to substantive determinations and coding rationale. A separate log should be used for reviewers to pose questions for which managing attorneys can provide responses for all reviewers to see. These logs will promote consistency and accuracy in document coding and create a historical record for future reference.

A senior attorney should also periodically evaluate reviewers, both in terms of the rate of the review as well as coding accuracy. If a reviewer is moving too quickly through complicated documents, it may be a red flag that they are merely running word searches rather than performing a comprehensive review. If quality control reveals that one reviewer missed a key privileged document, it is worth the time to engage quickly and identify whether other privileged documents were missed. Not every document needs be reviewed by a senior person; rather, samples of each reviewer’s documents can be screened to gauge the accuracy of the coding. Such oversight can catch discrepancies early on and prevent the need to repeat certain sections of the review.

Second-Level Review

Once the first-level review is complete, counsel should consider performing a second-level review to ensure the quality of a document review and coding accuracy. In so doing, the senior attorney may spot check a random selection of documents for accuracy. Counsel typically begin with a second review of designated “hot” documents and may expand to reviewing other responsive documents to collect information about the case. A second-level privilege review should be performed in most cases, and with smaller teams to increase consistency.

Production

Once the second-level review is complete, a subset of relevant/responsive, non-privileged documents will be identified for production. Counsel should confirm that only documents intended to be produced are included in the production set. One method of assuring this is to run quality control checks against documents marked as potentially privileged or families that may contain both relevant and non-relevant documents.

The timing and format of the production will also need to be considered. When selecting a production format, counsel should consider court rules regarding format and the needs of the particular case, all of which can and should have been agreed to by the parties in an ESI Protocol. See R. 4:103-2(c)(3); Fed.R.Civ.P. 16(b)(3)(B)(iii). In sum, the rules permit that the requesting party specify form of production, such as native format or TIFF or PDF images with or without metadata. The responding party may object and propose a different form or, if no form is originally requested, produce in the form in which the data is normally kept, or another “reasonably usable” format. R. 4:18-1(b)(1)-(2); Fed.R.Civ.P. 34(b)(2)(E)(i)-(iii).

Manner of production is another important consideration. Federal Rule of Civil Procedure 34 and New Jersey Rule 4:18-1 require that a party must “produce documents as they are kept in the usual course of business or must organize and label them to correspond to categories in the request.” These mandatory designations are often ignored by litigants, which can lead to expensive corrective measures. Courts have generally applied these requirements to ESI and have held that the “usual course of business” option cannot be satisfied absent sufficient information regarding the ESI’s original source, responsible custodian and file path. In the absence of such information, a party will be required to designate which documents respond to specific documents demands. One method of addressing this issue in the document review process is to incorporate coding of the specific demands (or demand topics) to which the documents pertain into the review protocol, but this can be time consuming.

Finally, constant communication with your vendor or in-house litigation support department is critical. They can be a wealth of knowledge on more technical production questions and can provide quality control searches to confirm that the production consists of the correct documents. Keep in mind that processing the production may take some time, usually 24 to 48 hours depending on the size of the production, so the vendor must have the final production set in advance of any production deadlines.

Conclusion

While document review will never be completely painless, the more planning and effort counsel puts into gathering information about the matter and training and managing the document reviewers, the better situated counsel will be to meet production deadlines, manage costs and avoid mistakes.

---

Reprinted with permission from the April 15, 2019 issue of the New Jersey Law Journal. © 2019 ALM Media Properties, LLC. Further duplication without permission is prohibited. All rights reserved. For information, contact 877-257-3382 or reprints@alm.com or visit www.almreprints.com.