Part 2: What Independent Audits Reveal That Internal Reviews Often Miss
Contributed by Jim Brigham, LCG VP of Risk Management, Former Operations Chief, State of Vermont, Office of Safety and Security
Series context. This article continues When Evidence Systems Break: Lessons from Independent Police Evidence Audits. Part 1 established that evidence failures are operational risk events driven by system drift rather than misconduct. Part 2 examines what independent reviewers consistently observe during evidence audits and why internal reviews, despite good intentions, often fail to surface cumulative risk early.
Independent Audits Look at Systems, Not Incidents
Internal evidence reviews are typically incident-driven. They focus on whether a specific discrepancy can be explained, corrected, or documented away.
Independent evidence audits start from a different premise. They assess whether the system as a whole can reliably produce defensible evidence outcomes under routine conditions, stress, and scrutiny. The question is not whether today’s evidence can be justified, but whether tomorrow’s evidence will withstand challenge. [1][2]
This distinction matters. Many agencies pass internal inspections while simultaneously carrying elevated operational risk. Independent reviewers routinely encounter systems that appear compliant on paper but fragile in execution. Policies exist. Logs are completed. Controls nominally function. Yet the system’s margin for error has quietly narrowed.
LCG perspective. Independent audits measure resilience, not intentions.
Drift Is Visible Only When Viewed Across Functions
One of the most consistent findings in independent audits is that no single unit sees the full risk picture.
Evidence intake may function well. Storage conditions may appear adequate. Supervisory checks may occur as scheduled. Each function, viewed in isolation, seems defensible. Risk emerges in the interfaces between them. [3]
Independent reviewers trace evidence across its full lifecycle, from seizure through disposition, and across organizational boundaries. This end-to-end view exposes friction points that internal reviews rarely test, including handoffs between patrol and evidence personnel, transitions between physical and digital systems, and exceptions handled outside standard workflows.
Over time, small inconsistencies at these seams compound into systemic vulnerability.
Informal Workarounds Are a Leading Risk Indicator
Internal reviewers are often aware of informal practices but normalize them as necessary adaptations.
Independent auditors treat workarounds differently. They ask why a workaround exists, how often it is used, and whether leadership is aware of its prevalence. In many audits, informal practices have effectively replaced formal procedures, even though policy remains unchanged. [4][5]
Common examples include temporary evidence storage that becomes routine, delayed documentation accepted as standard, and supervisory verifications completed retrospectively rather than contemporaneously.
These practices rarely violate policy outright. Instead, they erode evidentiary integrity incrementally while preserving the appearance of compliance.
Documentation Quality Reveals More Than Documentation Quantity
Most evidence systems generate extensive documentation. Internal reviews often focus on whether required fields are completed.
Independent audits assess documentation quality. Reviewers examine internal consistency, timing, narrative clarity, and alignment with actual practice. They look for patterns such as repeated corrections, generic language, unexplained gaps, or identical entries across multiple cases. [6]
These indicators often signal operational strain or the normalization of shortcuts. While each instance may be defensible individually, patterns suggest increasing exposure under external scrutiny.
Supervision Exists, but Verification Is Thin
Supervisory review is present in most agencies. Independent audits often reveal that supervision focuses on task completion rather than outcome validation.
Sign-offs confirm that a step occurred, not that it was effective. Supervisors may not independently verify environmental conditions, reconcile inventories against system data, or test access controls. [7]
Independent reviewers assess whether supervision meaningfully reduces risk or distributes accountability. When verification is thin, supervisory layers do not materially improve evidentiary resilience.
Technology Shifts Risk Without Governance
Evidence management technology frequently improves efficiency. Independent audits consistently show that it also redistributes risk.
Digital systems introduce dependencies on metadata integrity, role-based access, retention logic, system integration, and vendor controls. Internal reviews often assume that system configuration equates to compliance. Independent reviewers test whether the configuration aligns with policy, practice, and legal expectations. [8][9]
Common findings include overly broad access privileges, unclear audit trail ownership, undocumented system updates, and retention policies that conflict with prosecutorial needs. These gaps are rarely malicious. They reflect governance lag rather than technical failure.
Internal Reviews Are Structurally Constrained
Independent audits do not replace internal oversight. They complement it by addressing structural blind spots.
Internal reviewers operate within organizational hierarchies. They assess systems they helped design. Their scope is often limited by time, staffing, and competing priorities. Even well-executed internal reviews tend to validate known controls rather than challenge underlying assumptions. [10]
Independence allows auditors to apply external benchmarks, question long-standing practices, and test scenarios that internal teams may reasonably exclude.
Early Signals Are Usually Present, but Misread
Independent reviewers frequently identify early warning signs that were visible internally but not interpreted as risk.
These include unresolved minor discrepancies, repeated exception handling, growing reliance on individual expertise rather than on process, and increased prosecutorial questions that do not yet rise to the level of formal challenge. [11][12]
Because these signals do not immediately compromise cases, they are often deprioritized. Independent audits contextualize them as indicators of drift rather than isolated annoyances.
Credibility Is the Real Asset at Risk
The most important distinction independent audits surface is the difference between compliance and credibility.
Evidence systems may technically meet standards while losing confidence among prosecutors, defense counsel, or courts. Once credibility erodes, remediation becomes more complex and public narratives harder to control. [13]
Independent audits allow leadership to address vulnerabilities before credibility is tested externally, preserving both operational control and institutional trust.
Quick Checklist
- Assess evidence systems end-to-end, not function by function.
- Identify and document informal practices before they harden into norms.
- Use independent review to benchmark resilience, not just compliance. [14]
Final Thought
Independent evidence audits do not uncover hidden misconduct. They illuminate how capable systems slowly lose margin for error.
The value of independence lies in timing. Early review preserves leadership options and credibility. Delayed review narrows both.
The next article in this series examines how agencies can prepare for independent evidence audits proactively and use them as governance tools rather than crisis responses.
References (endnotes)
[1] ISO, ISO 31000:2018 Risk management – Guidelines (official ISO record): https://www.iso.org/standard/65694.html
[2] COSO, Internal Control – Integrated Framework (COSO landing page): https://www.coso.org/internal-control
COSO, Internal Control – Integrated Framework: Executive Summary (PDF copy): https://www.sechistorical.org/collection/papers/2010/2013_0501_COSOInternal.pdf
[3] Reason, James, Managing the Risks of Organizational Accidents (publisher page): https://www.routledge.com/Managing-the-Risks-of-Organizational-Accidents/Reason/p/book/9781840141054
[4] International Association of Chiefs of Police (IACP), Property & Evidence Control (PDF): https://www.theiacp.org/sites/default/files/2021-03/Evidence%20Control%20Formatted%2003.03.2021.pdf
[5] California POST, Evidence and Property Management (PDF): https://post.ca.gov/Portals/0/post_docs/publications/Evidence_and_Property_Management.pdf
[6] Federal Rules of Evidence, Rule 901 (Cornell LII): https://www.law.cornell.edu/rules/fre/rule_901
[7] U.S. GAO, Standards for Internal Control in the Federal Government (Green Book) (landing page): https://www.gao.gov/greenbook
GAO-14-704G (PDF): https://www.gao.gov/pdf/product/665712
[8] NIJ, Digital Evidence Policies and Procedures Manual (publication page): https://nij.ojp.gov/library/publications/digital-evidence-policies-and-procedures-manual
NIJ manual (PDF): https://www.ojp.gov/pdffiles1/nij/254661.pdf
[9] NIST, SP 800-53 Rev. 5, Security and Privacy Controls for Information Systems and Organizations (NIST CSRC record): https://csrc.nist.gov/pubs/sp/800/53/r5/upd1/final
NIST SP 800-53r5 (PDF): https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-53r5.pdf
[10] GAO, Standards for Internal Control in the Federal Government (Green Book) (same authority as [7], includes evaluation concepts and self-assessment limits): https://www.gao.gov/greenbook
[11] The Sedona Conference, Commentary on ESI Evidence & Admissibility (Second Edition, Oct. 2020) (PDF): https://www.thesedonaconference.org/sites/default/files/publications/ESI%20Evidence%20and%20Admissibility%20October%202020.pdf
[12] NIJ, Forensic Examination of Digital Evidence: A Guide for Law Enforcement (GovInfo preservation PDF): https://www.govinfo.gov/content/pkg/GOVPUB-J28-PURL-LPS49755/pdf/GOVPUB-J28-PURL-LPS49755.pdf
[13] United States v. Howard-Arias, 679 F.2d 363 (4th Cir. 1982) (Justia): https://law.justia.com/cases/federal/appellate-courts/F2/679/363/12720/
[14] U.S. GAO, Green Book (quick-checklist principles map cleanly to control environment, monitoring, and documentation): https://www.gao.gov/greenbook
This article is for general information and does not constitute legal advice.
