Why missing evidence is often worse than corrupted evidence

Contributed by Kris Carlson, COO, Former ICAC Commander, Digital Forensics Investigator, and Testifying Expert

Series Context. In Part 1, we examined the myth of the single source of truth and why forensic tools should never be treated as authoritative. The next question is equally important: What happens when the tool fails, but nobody realizes it? Silent failures are among the most significant risks in modern digital forensics because they create the illusion of completeness while concealing critical omissions. [1]

The Errors You Never See

Most investigators understand the danger of corrupted evidence.  A damaged hard drive, a failed acquisition, or a parser crash immediately signals that something has gone wrong. The problem is visible. The examiner knows there is an issue and can take corrective action, but silent failures are different.

A silent failure occurs when a forensic process appears to complete successfully, but critical information is omitted, misinterpreted, or never processed. There may be no obvious indication that anything is wrong.

• The software generates a report.
• The dashboard populates.
• The timeline appears complete.
• No error message appears.
• No warning is issued.

The examination then proceeds as though the tool successfully processed the evidence in full. In many cases, the investigator may never know that relevant information was missed.

That is what makes silent failures so dangerous. They do not announce themselves. They create the appearance of completeness while concealing gaps in the underlying analysis. As a result, investigative decisions, productions, expert opinions, and testimony may be based on information that appears reliable but is actually incomplete.

LCG Perspective.  Missing evidence can be just as dangerous as evidence that is wrong or misinterpreted. When a forensic tool fails openly, the examiner knows there is a problem. When it fails silently, the report may appear complete while the analysis is not. That gap can affect investigative decisions, productions, expert opinions, and testimony without anyone realizing that the underlying record is incomplete.

When “No Artifacts Found” Does Not Mean No Artifacts Exist

One of the most common misconceptions in digital forensics is treating a negative result as a definitive finding. Investigators often encounter tool reports stating:

• No messages found
• No location history found
• No deleted content identified
• No relevant application artifacts detected

At first glance, these statements appear clear. In reality, they require careful interpretation.

A critical distinction must be made between “no artifacts were found” and “the relevant artifacts were not processed, recognized, or supported by the tool.” The first suggests an absence of evidence. The second suggests a limitation in acquisition, parsing, or interpretation.

That distinction becomes especially important when working with new application versions, unsupported operating systems, recently modified database structures, cloud-hosted evidence, encrypted content, or partially acquired data sources. A forensic tool may fail to recognize a changed database schema. An acquisition platform may not fully support a newly released operating system. A cloud collection may authenticate successfully while still failing to capture certain data or categories of data; however, the absence of reported artifacts does not necessarily mean the artifacts do not currently or never existed. It may mean the tool did not acquire them, parse them, recognize them, or present them in the examiner-facing report.

This is why negative findings must be validated before they are treated as investigative conclusions.  Examiners should evaluate the scope and completeness of the acquisition, confirm which artifacts the tool could support, consider the relevant application and operating system versions, and account for encrypted, cloud-based, or partially acquired data.  When appropriate, negative findings should be tested through manual review, secondary tools, examination of source databases, or comparison with other available evidence.  Without that validation, investigators risk treating a tool limitation as proof that the evidence does not exist.

The Problem of Parser Drift

Digital forensic tools operate in a constantly changing environment. Applications evolve, operating systems update, cloud platforms modify APIs, developers change storage structures, and encryption models become more sophisticated. As a result, forensic vendors must continually update their parsers to keep pace with the data they are designed to interpret.

This problem is often described as parser drift. A parser that accurately interprets an application today may become less reliable after the next software update, and ineffective after future changes to the application, operating system, or cloud service.

Consider a messaging application that changes its database structure during a routine update. To the user, the application may continue to function normally. Messages sent, conversations displayed, attachments opened, and timestamps appear as expected. To the forensic tool, however, the underlying data may no longer match the structure the parser was designed to interpret.

The parser may continue searching for artifacts in expected tables, columns, file paths, or metadata fields that have been renamed, relocated, restructured, encrypted, or removed. The result may include:

• Missing conversations
• Incomplete message threads
• Incorrect participant attribution
• Timestamp inconsistencies
• Missing or unlinked attachments
• Misclassified artifacts
• Incomplete search results

These issues may not generate an obvious error. The parser may still process portions of the database, recover some artifacts, and produce a report that appears complete. Because some information is present, the missing information may be harder to detect.

From the examiner’s perspective, the tool output may appear normal. In reality, the tool may have silently failed to interpret portions of the evidence. That is the danger of parser drift: it can produce incomplete results without producing a clear warning that anything is wrong.

Case Vignette: The Missing Conversation

Imagine a corporate investigation involving allegations of insider misconduct.

Investigators acquire a company-issued mobile device and process it through a widely used forensic platform.  The resulting report contains several relevant communications but appears otherwise unremarkable.  Months later, during litigation, opposing experts independently review the same mobile device using a different forensic platform and a direct SQLite examination, and they discover an additional conversation thread involving key decision-makers.

• The messages were present within the source database.
• The original parser failed to process a recently modified application structure.
• No error message had been generated.
• No warnings had been displayed.
• The examiner reasonably believed the report was complete.

The issue was not incompetence; it was overreliance on a single layer of interpretation.  The litigation consequences, however, may be significant regardless of intent, damaging both the case at hand and the forensic expert’s reputation.

Timestamp Errors: Small Technical Problems, Major Investigative Consequences

Not every silent failure involves missing data. Sometimes the evidence is present, but the tool interprets it incorrectly. Timestamp normalization errors are a common example.

Digital systems store dates and times in many different formats, including Unix Epoch, Mac Absolute Time, Windows FileTime, application-specific formats, and cloud service timestamps. Some timestamps are stored in Coordinated Universal Time, some in local time, and some require a separate time zone or offset to be interpreted correctly.

Forensic tools must convert these values into readable dates and times that can be searched, filtered, reported, and placed into timelines. When that conversion is even slightly wrong, the investigative consequences can be significant.

A communication may appear to have occurred before a critical event instead of after it. A user action may appear outside the relevant timeline window. Location activity may appear inconsistent with other evidence. An alibi may appear stronger or weaker than it actually is. A sequence of events may be reversed, compressed, or expanded due to an incorrect interpretation of a timestamp.  The danger is that the underlying data may still be present. The report may appear complete. The timeline may appear organized and authoritative. Nothing may alert the examiner to the fact that the dates and times have been normalized incorrectly.

For that reason, timestamp evidence should not be accepted blindly. Examiners should understand the original timestamp format, confirm whether the value was stored in UTC or local time, consider time zone and daylight-saving issues, and, when necessary, validate critical events against the source database, file metadata, device settings, system logs, or another forensic tool.

A complete timeline is not necessarily an accurate timeline. If the timestamps are wrong, the conclusions built from them may be wrong as well.

Cloud Evidence Creates New Opportunities for Silent Failure

As evidence increasingly moves into cloud environments, the risk of omission errors continues to grow. Unlike traditional device-based evidence, cloud evidence is often collected through controlled access points that depend on the provider, the account, the available permissions, security settings/features, and the technical limitations of the collection method. We have witnessed this issue firsthand on numerous occasions.

Cloud acquisitions often depend on several variables, including:

• API access
• Account permissions
• Service provider limitations
• Authentication status
• Licensing level
• Administrative privileges
• Data retention settings
• Sync status between devices and cloud repositories

A cloud acquisition may complete “successfully” from a technical standpoint while still collecting only a subset of the available information. The tool may authenticate to the account, run the collection, generate an export, and produce a report, but that does not necessarily mean every relevant data category was captured.

Examples may include:

• Missing shared files
• Files skipped due to throttling, server disconnection, timeout events, or provider-side limitations
• Deleted content that was excluded or unavailable through the collection method
• Incomplete chat histories
• Partial audit logs
• Restricted account activity
• Files or messages unavailable due to permission limitations
• Cloud content referenced on a device but not actually collected from the cloud source

The challenge is that the collection process may not clearly communicate these limitations. A log may show that the acquisition completed, while also containing warnings, skipped items, partial failures, or category-specific limitations that are easy to overlook if the log is not easy to interpret, is proprietary, or very lengthy. In other situations, the tool may not provide a meaningful warning at all.

The resulting evidence set may therefore appear complete even when it is not. That risk is becoming increasingly important as organizations rely on Microsoft 365, Google Workspace, Slack, Teams, and other cloud native business applications for communications, document storage, collaboration, and user activity records.

Cloud evidence should not be evaluated solely by whether a collection was completed. It should be evaluated based on what was requested, what the tool was capable of collecting, what permissions were available, which data categories were included or excluded, what the logs reveal, and whether additional provider exports, administrative searches, or targeted preservation steps are necessary.

Legal and Methodological Consequences

Silent failures create problems that extend well beyond the technical examination. When evidence is omitted, misinterpreted, or never processed, the issue can affect regulatory inquiries, internal investigations, civil litigation, criminal prosecutions, and expert testimony.

The risk is not limited to the missing artifact itself. If an opposing expert later identifies omitted evidence, the challenge may shift from the specific finding to the reliability of the entire methodology. Questions may arise regarding how the evidence was validated, whether multiple tools were used, whether raw data were reviewed, whether parser limitations were considered, and what quality assurance procedures were in place.

Those questions matter because forensic conclusions are often judged not only by the result but also by the process used to reach it. Under FRE 702, experts must demonstrate that their opinions are based on reliable principles and methods and that those methods were reliably applied to the facts of the case. [5] A methodology that assumes software completeness without verification may therefore become difficult to defend.

Courts, regulators, and opposing experts are increasingly focused on the investigative process behind the output, not simply the existence of a polished report. Defensibility depends on methodology, documentation, and validation, not automation alone.

Building Defenses Against Silent Failures

No organization can eliminate every limitation on tools.  Organizations can, however, significantly reduce risk through disciplined practices.

Verify Critical Artifacts

• Review source databases directly when appropriate.
• Confirm high-impact findings independently.
• Validate timestamps against raw records.

Use Multiple Validation Methods

• Employ secondary forensic platforms.
• Compare parser outputs.
• Investigate discrepancies.

Understand Tool Limitations

• Review vendor support documentation.
• Track parser changes across updates.
• Document unsupported applications.

Maintain Validation Programs

• Use known datasets.
• Conduct regression testing.
• Revalidate after major software releases.

Encourage Methodological Skepticism

• Question negative findings.
• Distinguish between absence of evidence and absence of processing.
• Treat software output as a starting point, not a conclusion.

Quick Checklist

1. Never assume “no artifacts found” means no evidence exists.
2. Validate critical findings using independent methods.
3. Monitor parser support and software updates.
4. Use known datasets to evaluate forensic tools.
5. Document limitations and verification steps. [3][4]

Final Thought

The forensic community often focuses on visible failures because they are easy to identify.

Silent failures deserve equal attention because they often remain invisible until challenged by another examiner, opposing counsel, or a court.  The danger is not merely that evidence may be missed. The danger is that investigators, attorneys, and decision-makers may act with complete confidence in conclusions built upon incomplete information.

As digital ecosystems become increasingly complex, the possibility of omission errors will continue to grow. The solution is not to abandon forensic automation; rather, it is to recognize its limitations: Trust the tool/but verify the results.

References

[1] Carlson, K. Trust but Verify, Part 1: The Myth of the Single Source of Truth. LCG Discovery Experts.
https://lcgdiscovery.com/trust-but-verify-part-1-the-myth-of-the-single-source-of-truth/

[3] National Institute of Standards and Technology (NIST). Computer Forensics Tool Testing (CFTT) Project.
https://www.nist.gov/programs-projects/computer-forensics-tool-testing-cftt

[4] National Institute of Justice (NIJ). Electronic Crime Scene Investigation: A Guide for First Responders, Second Edition.
https://www.ojp.gov/pdffiles1/nij/219941.pdf

[5] Federal Rules of Evidence, Rule 702. Testimony by Expert Witnesses.
https://www.law.cornell.edu/rules/fre/rule_702

[6] Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993).
https://supreme.justia.com/cases/federal/us/509/579/

[7] National Institute of Standards and Technology (NIST). Guide to Integrating Forensic Techniques into Incident Response (NIST Special Publication 800-86).
https://csrc.nist.gov/pubs/sp/800/86/final

[8] Scientific Working Group on Digital Evidence (SWGDE). Minimum Requirements for Quality Assurance in the Processing of Digital and Multimedia Evidence.
https://www.swgde.org/documents/published-complete-listing/10-q-001-swgde-minimum-requirements-for-quality-assurance-in-the-processing-of-digital-and-multimedia-evidence/

This article is for general information and does not constitute legal advice.