Multilingual Ticket Preprocessing Under a Two-Week Deadline

The gap is driven by three interacting choices. First, lowercasing plus English stopword removal deletes negation and modal cues: 'cannot login' becomes 'login' and 'not able to pay' becomes 'able pay', inverting meaning. Keep a negation-aware tokenizer: do not remove not, no, cannot, can't, or Malay tak, and attach a NEG_ prefix to the next two tokens. Second, punctuation and emoji stripping removes urgency markers. Map repeated !, ???, and high-arousal emoji to explicit tokens such as URG_EXCLAIM, URG_QUESTION, and EMO_URGENT before cleaning. Third, applying an English stopword list to pinyin and Singlish deletes meaningful particles, while account numbers are split into spurious fragments. Use per-ticket language identification, a language-specific stopword list, and replace account numbers with ACCT_ID rather than tokenizing digits. Chinese needs segmentation with jieba or character n-grams, and pinyin/Han variants should normalize to a shared alias token. Ship these as deterministic preprocessors with unit tests on 500 known error cases. Do not adopt the multilingual BERT yet: the 8× cost is unjustified until a cheap ablation shows the remaining gap is semantic rather than tokenization-related.

Use a stratified 30,000-ticket holdout by language and urgency, and freeze the model type and hyperparameters. Run a factorial ablation: baseline, plus negation handling, plus emoji/punctuation tokens, plus language-specific stopwords, plus PII placeholder, plus Chinese segmentation and pinyin alias, and all fixes combined. Train each variant on the same 200,000-ticket training set with three seeds. Report macro-F1 and urgency recall overall and per language. The primary metric should be urgency recall at a fixed precision floor of 0.80 because missed urgent tickets are the costly error; macro-F1 is secondary. Guardrail: reject any fix that drops urgency recall by more than 1 point or macro-F1 by more than 0.5 point in any language, even if the aggregate improves. Use paired bootstrap confidence intervals over tickets and a McNemar test on discordant urgency predictions. Inspect confusion matrices by language to catch an English gain masking a Tamil or Malay loss. This design separates tokenization effects from model capacity and prevents aggregate improvements from hiding subgroup regressions.

Mask account numbers and personal names with typed placeholders rather than deleting them: ACCT_ID, PERSON, and PHONE, using deterministic regexes for account formats plus a small NER model for names. Deletion is risky because the presence of an account number and repeated contact attempts is itself an urgency signal; a placeholder preserves the fact that the ticket contains an account reference without exposing the value. Keep a separate boolean feature has_account_id and a count of prior mentions. Test three failure modes. First, regex over-masking order IDs that look like account numbers, which could remove product-specific urgency cues. Second, under-masking formatted or spaced account numbers. Third, placeholder drift if the regex changes between training and serving. Evaluate with a PII-recall test set of 1,000 tickets and a utility check: urgency recall with masked versus unmasked text should differ by less than 1 point. Verify that no account number appears in logs or TF-IDF vocabulary; hashing the placeholder is safer than storing raw IDs.