Overview
Privacy-oriented named-entity recognition datasets use different sources, annotation schemes, and PII categories. ASC-PIE provides a shared evaluation framework for studying those differences without treating each dataset as an isolated experiment.
Challenge
Privacy datasets combine real and synthetic text and do not begin with one common label system. Meaningful comparisons require explicit preprocessing and label mapping before model results can be evaluated consistently.
My role
I designed and built the research pipeline, corpus-standardization workflow, experiments, and evaluation framework. The work became my M.A. thesis, “ASC-PIE: An Evaluation Framework for PII-Aware Named-Entity Recognition.”
What I built
I built individual preprocessing workflows for each source dataset, mapped their labels into a shared representation, and created a reproducible training and evaluation workflow using Python, PyTorch, Hugging Face Transformers and Datasets, scikit-learn, and seqeval.
Architecture
privacy datasets → preprocessing → shared PII labels → model training → NER evaluation
The experiment repository keeps dataset preparation, combined Hugging Face datasets, model training artifacts, and evaluation dependencies in one reproducible workflow.
Results
I completed the thesis and York University officially awarded my M.A. in Information Systems & Technology in 2026. The related SPRINT-PP research paper is submitted and under review; I do not present it as published or accepted.
Lessons
Comparable model evaluation starts with transparent data and label decisions. A shared schema makes differences between datasets visible instead of hiding them inside separate preprocessing notebooks.