Data & analytics · reviewed in July 2026
Data cleaning
Data cleaning is the process of detecting and fixing missing, duplicated, badly formatted, or inconsistent values in a dataset before analyzing it. It usually consumes most of the time in a real data analysis project.
Frequently asked questions
What do you do with missing values (NaN)?
It depends on the case: you can drop the affected rows or columns (dropna), fill them with a reasonable value like the mean or median (fillna), or leave them explicitly if the analysis handles them well.
How do you detect duplicate rows in pandas?
With df.duplicated(), which flags repeated rows, and df.drop_duplicates() to remove them.
Why does cleaning matter so much before analyzing?
Because dirty data quietly produces wrong conclusions — an average calculated over duplicates or mistyped values can look valid without being one, with no error signal at all.