Affected module
Governance (Glossary / Classification / Domains)
Describe the bug
A French autoclassification agent will tag correctly as PII.Sensible a column containing the French first-names Geneviève, François, Michael... but won't be able to do so when the column contains uppercased first-names like SERGE, THÉODORE, HÉLÈNE, JEAN-MARC...
To Reproduce
Create a table with a column containing capitalized names and check that an auto-classification agent detects the column as sensible PII.
Create another table with the same column name, but now containing upper-cased names and run the same auto-classification agent: this columns is not detected as sensible PII.
Expected behavior
Casing of text should not degrade PII detection. This a real problem with databases of legacy systems where upper-case names are frequent.
OS
Kubernetes
Python version
3.10.19
OpenMetadata version
1.12.14
OpenMetadata Ingestion package version
1.12.14.2
Additional context
No response
Pre-submission checklist
Affected module
Governance (Glossary / Classification / Domains)
Describe the bug
A French autoclassification agent will tag correctly as
PII.Sensiblea column containing the French first-names Geneviève, François, Michael... but won't be able to do so when the column contains uppercased first-names like SERGE, THÉODORE, HÉLÈNE, JEAN-MARC...To Reproduce
Create a table with a column containing capitalized names and check that an auto-classification agent detects the column as sensible PII.
Create another table with the same column name, but now containing upper-cased names and run the same auto-classification agent: this columns is not detected as sensible PII.
Expected behavior
Casing of text should not degrade PII detection. This a real problem with databases of legacy systems where upper-case names are frequent.
OS
Kubernetes
Python version
3.10.19
OpenMetadata version
1.12.14
OpenMetadata Ingestion package version
1.12.14.2
Additional context
No response
Pre-submission checklist