Data Cleaning Spirals Into 89 Preprocessing Steps.

One small cleaning. Then another. Then another. Suddenly, 89 steps later and you're still not done.

Read time: 2.5 minutes

"The startup so-called cleaning may need cleaning." Another data scientist's cliché. 89 data preprocessing steps down the road, and we are still cleaning up the mess.

The scientist starts with one cleaning step. Eliminate missing values. Eliminate duplication. Then, correct the errors. After that, parse the date. Then, encode the entries. Then, normalize. Here comes step 89: "Oddly clean." The data is not ready.

3 shortcuts for avoiding inconvenience during preprocessing:

1. Define what the word "clean" means.
❌ Cleaning the image without a destination.
✅ Agreeing on the cleaning criteria beforehand.

2. Automate some stages if possible.
❌ Doing every step manually.
✅ Creating a pipeline that performs all steps automatically.
Automate everything wherever possible.

3. Go to analyzing as soon as you can.
❌ Pursuing perfect data.
✅ Always ask whether the data is good enough to answer business questions.
Perfection is the primary obstacle to progress.

💡Key Takeaway: 

Minor cleaning may lead to something far bigger, and before you know it, you would have gone through 89 steps of cleaning, only to realize you should have limited your efforts.

👉 LIKE this if you have ever cleaned data obsessively.

👉 SUBSCRIBE now to receive data science knowledge regular updates.

👉 Follow Glenda Carnate to deliver results rather than get lost in the world of data cleaning.

👉 COMMENT "89" if you have ever spent a long time cleaning data.

👉 SHARE this with a data scientist who needs to know when to stop cleaning.

Reply

or to participate.