The 10-Check Automated Cleaning Pipeline
How deterministic rules combined with human verification and an audit trail turn raw spreadsheets into reproducible research tables.
In this lesson you will learn
- 1Trace how deterministic checks flag impossible physiological ranges, negative values, and duplicate participant rows.
- 2Understand why human review must sit between automated detection and final table modification.
- 3Explain why a timestamped Audit Trail Log is essential for peer-review reproducibility.
On this page▼
Interactive Cleaning Sandbox: Auditing & Logging in Action
This section walks through a structured 4-stage cleaning workflow: raw upload, deterministic cell-level detection, researcher approval, and audit-log generation.
Modifying raw spreadsheets manually without saving a change log makes it impossible to defend how outliers or duplicates were handled if questioned by an examiner.
Rigorous data preparation follows an evidence-first workflow: automated checks scan every row and cell to surface concrete anomalies, and the researcher decides which corrections to apply while preserving the original file intact.
In the interactive lab below, click "Audit Dataset" to highlight seven distinct data-entry faults across patient records, then click "1-Click Analysez Clean" to inspect the harmonised table and its complete audit log.
Interactive Clinical Data Cleaning Sandbox
Experience how Analysez's 10-point audit catches fatal flaws and sanitizes real research data
| ID | Age | Gender | Systolic BP | Fasting Glucose | Group |
|---|---|---|---|---|---|
| P001 | 142 | "Male" | 128 | 96 | Control |
| P002 | 45 | " Male " | 135 | 104 | Treatment |
| P003 | 52 | "female" | 120 mmHg | 110 | Treatment |
| P004 | 61 | "Female" | 142 | N/A | Control |
| P005 | 39 | "Male" | -15 | 88 | Control |
| P006 | 58 | "FEMALE" | 130 | 999 | Treatment |
| P002 | 45 | "Male" | 135 | 104 | Treatment |
Clinical Example
During pre-analysis screening of a 200-patient cardiology spreadsheet, row 14 shows Age = 142 years and row 88 repeats Patient ID P002 verbatim.
Interpretation: Flagging Age = 142 prompts source-chart verification (correcting a typo for 42), while removing the exact duplicate row restores independence of observations.
Common Pitfall
Overwriting the only copy of the raw spreadsheet during cleaning, leaving no way to undo a mistake or verify original values.
Key takeaways
- 1Deterministic cleaning rules inspect each column for physiological bounds, whitespace, embedded units, sentinel codes, and duplicate IDs before analysis.
- 2Every cleaning action should remain optional and human-reviewed so clinical domain knowledge guides whether an extreme value is a typo or a critical patient.
- 3Recording every cell transformation in an Audit Log allows co-authors, supervisors, and journal reviewers to reproduce the exact path from raw file to final table.
Check your understanding
One question per screen with immediate methodological rationale after each answer
In an adult cardiology dataset, a participant's age is recorded as '142'. How should a structured cleaning workflow handle this value?
Finished reading this lesson?
Mark as completed to track your progress through the handbook