Skip to content

Add edaprep to Feature Engineering - #106

Open
bijay-odyssey wants to merge 1 commit into
krzjoa:masterfrom
bijay-odyssey:add-edaprep
Open

Add edaprep to Feature Engineering#106
bijay-odyssey wants to merge 1 commit into
krzjoa:masterfrom
bijay-odyssey:add-edaprep

Conversation

@bijay-odyssey

Copy link
Copy Markdown

Adding edaprep under Feature Engineering > General, at the last position per contribution rule 2.

edaprep profiles a dataset, works out which preprocessing operations actually apply to it, and prints the reasoning with the measurement attached before it acts:

income:
  + impute_median - 2.0% missing; median rather than mean because it is
                    unaffected by the skew (3.22) and by outliers
customer_id:
  x dropped - identifier: 100.0% of values are distinct, so it cannot
              generalise beyond the rows it was fitted on

Every statistic is fitted on the training frame alone, and that is enforced by tests rather than documented as a convention: one parses each transform method with ast and fails the build if it aggregates over its input, another requires that transforming a frame whole equals transforming it row by row.

It sits closest to Feature Engine in the existing list. Pandas-compatible, so I have added the pandas badge as the surrounding entries do.

  • MIT licensed, pure Python, no compiled code
  • Python 3.9-3.13, CI on Linux, macOS and Windows
  • On PyPI: https://pypi.org/project/edaprep/
  • 367 tests; benchmarks and method in docs/performance.md

Description starts with a capital and ends with a full stop, per rule 5. Checked it is not already listed, per rule 3.

Disclosure: I am the author. Happy to move it or reword if another section fits better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant