- Daily Success Snacks
- Posts
- "The Dataset Is Clean." Said No Data Scientist Ever After Finding Their 47th NULL Value.
"The Dataset Is Clean." Said No Data Scientist Ever After Finding Their 47th NULL Value.
NULL values are like weeds. You think you've pulled them all. Then you find one more. Then another. Then another.

Read time: 2.5 minutes
Your buddy has told you this: the dataset is fine. But you know better. There's always that one lame NULL value hiding there, just waiting to screw you over.
He had trouble cleaning this up. The NULLs were deleted. The duplicates were taken care of. He fixed all the formats. And then his buddy said, "Let's call it a day." But his answer was, "One more NULL." However, he really knew there would never be a "one more."
3 Ways To Claim A Data Set Is Clean Without Losing Your Bling Bling.
1. Set a threshold on how many NULLs will be acceptable.
❌ Searching for all the NULLs to eliminate.
✅ Have a consensus on a threshold for acceptable NULLs (e.g., less than 1% NULLs).
You can't perfect something to move forward.
2. Build in automated checks to help eliminate the need for manually searching.
❌ Continuous manual search for missing values.
✅ Automated scripts that will send you a NULL flag and report.
Let the computer do it for you.
3. Know when to stop cleaning and start analyzing.
❌ Cleaning up until it is "perfect."
✅ Asking self: "Is this good enough to answer my business-related question?"
Only continue cleaning if the answer is NO; otherwise, deliver it as it's been cleaned up.
💡Key Takeaway:
There will always be at least one extra NULL. However, there's usually not enough time to adequately clean everything. Therefore, you need to clean intelligently. You may want to clean forever, but you cannot.
👉 LIKE this if you've ever said there was "one more NULL".
👉 SUBSCRIBE now for daily truths about data science that are painful but valuable.
👉 Follow Glenda Carnate so you can stop focusing on creating the perfect result and begin producing value.
Instagram: @glendacarnate
LinkedIn: Glenda Carnate on LinkedIn
X (Twitter): @glendacarnate
👉 COMMENT "NULL" on the biggest amount of time you ever wasted down a data-cleaning rabbit hole.
👉 SHARE this with a data scientist who spends way too much time cleaning data instead of analyzing it.
Reply