What's Old is New Again: Why Document Review Protocols and Custodian Interviews Still Matter in AI-Driven eDiscovery

With new AI advances in the legal field seemingly occurring with every passing day, it is often assumed that the attorney’s role in eDiscovery will inevitably diminish. However, in our digital and increasingly virtual world, some of the most traditional “analog” eDiscovery tools — such as the written document review protocol and custodian collection interview — will continue to be important in the future. This may seem counterintuitive, but as technology evolves, the human element in eDiscovery becomes even more critical. Attorneys play a key role in managing and optimizing AI-driven document reviews and the collection process to ensure defensible, effective outcomes.

The Document Review Protocol and Use of AI 

eDiscovery vendors are increasingly deploying AI-based solutions that automate aspects of traditional document review. With cutting-edge technology, a generative AI model can ingest a document review protocol written by counsel and translate it into prompts, which the model then uses to score documents for responsiveness, privilege, and other key dimensions. Rule-based prompts derived from the protocol guide the AI in analyzing and categorizing documents. This approach holds the promise of largely eliminating the need for human review to “train” the model, a process that has traditionally involved attorneys manually coding a sample set of documents to teach machine learning-based technology-assisted review (TAR) systems how to distinguish relevant from non-relevant material.

As AI technology improves, becomes more cost-effective, and gains greater acceptance by authorities and the courts, this methodology may become standard. However, its effectiveness hinges on the quality of the document review protocol. The protocol that the attorney drafts must distill responsiveness, privilege, and other dimensions into clear, structured rules that prompt the AI model to generate reliable results. It also must be internally consistent. Attorneys will also need to validate AI-driven outcomes through rigorous testing and iteration, making modifications to the protocol to improve results. This may require attorneys to conduct some training of the model to enhance precision and ensure accuracy. In sum, the creation and improvement of the protocol will necessitate linguistic, analytical, and logical skills that are within the domain of the attorney drafter. 

Moreover, while AI holds promise for streamlining document review, privilege determinations pose unique risks. AI models may struggle with nuanced privilege calls, increasing the risk of misclassification or inadvertent waiver. Attorneys will need to closely monitor AI outputs, implement robust quality control measures, and consider best practices for privilege review in an AI-driven workflow.

New Communications Apps and the Custodian Interview

New communication technologies present challenges for document collection. Ephemeral messaging applications such as Snapchat, Signal, and Telegram, which automatically delete messages by design (though many of these technologies offer settings that allow for preservation), have drawn scrutiny from various federal authorities, including the Federal Trade Commission, the Antitrust Division of the Department of Justice, the Securities and Exchange Commission, and the Commodity Futures Trading Commission. Authorities increasingly view these tools as sources of potentially responsive information that must be preserved.

Given the widespread use of personal messaging applications alongside employer-provided platforms, custodian interviews are often the only way to determine what communication tools an individual uses. Furthermore, the interview is essential to understanding to what extent the custodian uses an application for personal versus business purposes. More often than not, it is a mix of the two.

Custodians frequently do not understand how these applications function or that their messages may be automatically deleted. Similarly, many are unaware that their smartphone settings may be configured to delete text messages automatically after a set period (e.g., 30 days).  

A diligent interviewer with a detailed interview outline and an advanced understanding of how the apps in question work is essential for uncovering such issues. For example, during the interview, counsel may need to guide the custodian through checking and, if appropriate, modifying phone settings to comply with preservation obligations.

Beyond ephemeral messaging, custodian interviews are also critical for identifying the use of collaboration tools such as Slack and Microsoft Teams, especially because the company IT department may not always be aware of all platforms employees use for business purposes. For example, business teams sometimes adopt new tools on their own without IT’s knowledge. Additionally, custodian interviews help assess whether employees conduct business communications on personal devices. These factors further underscore the continued importance of a thorough and strategic interview process.

Conclusion

Advances in technology present both opportunities and challenges in eDiscovery. However, the document review protocol and custodian interview remain foundational tools for managing AI-driven document review and assessing the scope of electronic collections. Attorneys must develop strong protocols, oversee AI implementations, and conduct effective custodian interviews to navigate these evolving challenges.

© Arnold & Porter Kaye Scholer LLP 2025 All Rights Reserved. This Blog post is intended to be a general summary of the law and does not constitute legal advice. You should consult with counsel to determine applicable legal requirements in a specific fact situation.