The increasing availability of digital text in diverse languages and scripts presents a significant challenge for natural language processing (NLP). Multilingual pre-trained language models (mPLMs) often struggle to handle transliterated data effectively, leading to performance degradation. Addressing this issue is crucial for improving cross-lingual transfer learning and ensuring accurate NLP applications across various languages and scripts, which is essential for global communication and information processing.