Revisiting the Relationship Between Keyword Search and Technology-Assisted Review in Judge Scheindlin’s National Day Laborer Opinion

August 6, 2012 Sheila Mackay

In National Day Laborer Organizing Network v. U.S. Immigration and Customs Enforcement Agency, No. 10 Civ. 3488 (SAS), 2012 U.S. Dist. LEXIS 97863 (S.D.N.Y., July 13, 2012), a Freedom of Information Act (FOIA) case, U.S. District Court Judge Shira Scheindlin not only offered kudos to technology-assisted review but also valuable advice for best practices in applying it in combination with other search methods.

Citing the recent decision in Da Silva Moore v. Publicis Groupe, where U.S. Magistrate Judge Andrew Peck noted, “In too many cases, however, the way lawyers choose keywords is the equivalent of the child’s game of ‘Go Fish,’” Judge Scheindlin agreed that in isolation, “[s]imple keyword searching is often not enough.” In this case, the defendants failed to offer the court detailed information about the keyword searches they ran—specifically, they did not provide the search terms or explain how they were combined—therefore, Judge Scheindlin could not determine the adequacy of their searches.

Noting that “the method in which [keywords] are combined and deployed is central to the inquiry,” the judge suggested that the FBI should have “run a few verification tests using sophisticated search techniques to ensure that the manual review was actually capturing the universe of responsive documents. Such tests would have given the Court significantly more confidence regarding the adequacy of these manual reviews.” Indeed, the judge suggested that there is a “need for careful thought, quality control, testing, and cooperation with opposing counsel in designing search terms or ‘keywords’ to be used to produce e-mails or other electronically stored information.” Judge Scheindlin urged litigants to adopt technology-assisted review methods, which “allow humans to teach computers what documents are and are not responsive” to increase the “effectiveness and efficiency” of searches.

Judge Scheindlin reinforced the potential value and efficacy of search – when executed appropriately – when she ordered that “the parties will need to work cooperatively to design and execute a small number of new, targeted searches.” She then added: “[t]he parties will need to agree on search terms and protocols – and, if necessary, testing to evaluate and refine those terms.”

Judge Scheindlin’s advocacy of technology-assisted review as a means to enhance keyword searching—as well as to compensate for its shortcomings—reaffirms the approach we have been espousing for a  long time.  That is, these e-discovery tools are not mutually exclusive: in fact, in our experience, using accurately crafted and vetted keyword searches and technology-assisted review together helps clients capitalize on the experience and knowledge of the lawyers working on a matter.

Sheila Mackay is Senior Director, E-Discovery Consulting at Conduent. She can be reached at

About the Author


Previous Article
High Noon in the Wild West of E-Discovery

There’s a new sheriff in town policing parties who flout their discovery obligations under the Federal Rule...

Next Article
How Dangerous Is Self-Collection in E-Discovery?

Just last week in another opinion United States District Court Judge Shira Scheindlin voiced skepticism reg...