The National Institute of Standards and Technology (NIST) is looking for public and private sector experts to take part in an exercise that will evaluate the digital forensics community’s ability to conduct mobile- and computer-based investigations.

In the study, to be conducted online, participants will examine simulated digital evidence, then answer questions that might arise in a real criminal investigation. The exercise should take about two hours, and participation is voluntary. The online test will be available for approximately three months.

“We want to understand the state of the practice,” said Barbara Guttman, leader of NIST’s digital forensics research program. “Can experts produce accurate and reliable information when examining data from a digital device?”

According to NIST, one challenge with digital evidence is that it can often be difficult to find key bits of evidence among large volumes of data. Also, technology changes so quickly that it can be difficult to keep up.

“Forensics experts can’t extract data perfectly in every possible scenario,” Guttman said. “Phones change. Apps change. The world just moves too fast.”

For the NIST black box study, participants will download simulated evidence from the NIST website in the form of one virtual mobile phone and one virtual computer. Such virtual devices, called “forensic images,” are commonly used in digital forensics, and study participants will be able to connect to them using the same software tools they use when working on real cases.

The study is open to all public and private sector digital examiners who conduct hard drive or mobile phone examinations as part of their official duties. NIST will not calculate the performance of any specific expert or laboratory. Instead, NIST will publish anonymized and aggregated results that show the overall performance for the expert community and different sectors within that community. 

According to NIST, the study will fulfill a critical need identified in a landmark 2009 report by the National Academy of Sciences. Titled Strengthening Forensic Science in the United States: A Path Forward, the report called for black box studies to measure the reliability of forensic methods that rely on human judgment. Courts and jurors can then consider the results of those studies when weighing evidence. The results of this study will also provide strategic direction for future research.

The black box study is part of a larger effort to evaluate the scientific foundations of digital forensic methods. NIST is also conducting scientific foundation reviews for DNA mixtures, firearms identification and bitemark analysis.

For more information and to enroll, visit the NIST Blackbox Study for Digital Examiners.