Check a CSV file for data quality problems

--- name: check-a-csv-file-for-data-quality-problems description: "Profiles a pasted or described CSV or spreadsheet for missing values, duplicates, mixed types and outliers, then ranks fixes. Use when someone asks whether data is clean enough to analyze, import or chart." --- Apply this whenever someone shares a table of data (CSV, spreadsheet export or pasted rows) or asks whether data is clean, safe to import or ready for analysis. Steps 1. Confirm what you can see. If only a sample is shared, say so and state that rates you report are for the sample only. 2. For each column, work out the likely type (number, date, category, free text, identifier) and report: - How many values are missing, including blanks, "N/A", "null", "-" and similar placeholders. - Mixed types, such as numbers stored as text, dates in several formats, or currency symbols inside number columns. - Odd spacing, inconsistent capitalization and spelling variants of the same category. - Numeric values that look implausible for the field (negative ages, future birth dates, extreme outliers) and the rule you used to flag them. 3. Check rows: exact duplicates, near duplicates on the likely key (same email or order number with different details), and whether any column could be a unique identifier. 4. Check relationships you can verify, such as end date before start date, totals that do not match parts, or categories that should be mutually exclusive. 5. Note file-level issues if visible: strange characters from encoding problems, merged header rows, totals rows mixed into the data. Output shape - A one-paragraph verdict: ready, usable with fixes, or not trustworthy yet. - A table of findings with columns: issue, where, how many, why it matters, suggested fix. Order by impact on the stated purpose, highest first. - A short list of questions for the data owner that you could not settle from the file. Avoid - Fixing or deleting data without saying what changed; propose, do not silently apply. - Declaring an outlier an error when it could be real; label it "check this". - Inventing counts for data you cannot see. - Long statistical explanations; keep it practical.

Verdict: usable with fixes. About 6 percent of order dates are in a different format and one in forty rows is a likely duplicate. | Issue | Where | How many | Why it matters | Fix | | Duplicate orders | order_id | 31 rows (sample of 1,200) | Inflates revenue | Keep the latest row per order_id | | Dates as text | order_date | 74 rows use DD/MM/YYYY | Wrong sorting and totals | Convert to one format | | Placeholder blanks | region | 58 "N/A" | Hidden missing values | Treat as blank and decide a rule | | Possible outlier | amount | 1 row at 98,400 | Skews averages | Check this with the owner | Questions for the data owner: Is amount in one currency? Is region required?