subtopic

Data quality issue log and impact

Keep a table of every issue: column, problem, rows affected, severity, treatment, decided by. Concepts: Issue log; impact on distributions and evaluation Project A: Log: renamed columns, decoded labels, 'unknown' handling, rare categories, any mapping that produced NaN (check after EVERY .map()). Project B: Log: dropped empty generation columns, DST duplicates, weather duplicates, pressure errors, missing load hours and how filled (time interpolation).