DatarrX MyWikisource Corpus

A high-quality, large-scale Burmese (Myanmar) language dataset extracted and refined from the official Burmese Wikisource. This dataset is specifically designed for Natural Language Processing (NLP) tasks, including language modeling, spell checking, and formal text analysis.

Dataset Summary

• Organization: DatarrX
• Maintainer: Khant Sint Heinn(Kalix Louis)
• Source: Burmese Wikisource (my.wikisource.org)
• Total Rows: 1,352,972
• Total Rows: 1,352,972
• Format: ID, Categories, Title, Context
• Language: Burmese (Myanmar) - 100% (Threshold Filtered)
• License: MIT

Data Origin (Source)

The data is sourced directly from the Wikimedia Foundation's Wikisource project for the Burmese language. Wikisource is a digital library of free-content textual sources, which includes historical documents, classical literature, legal texts, and poems that have been proofread by the community.

Data Processing Details

This dataset has undergone rigorous cleaning to ensure high utility for AI/ML applications:

1. Namespace Filtering: Only core encyclopedia/document entries (NS:0) were included, removing metadata and talk pages.
2. HTML/XML Cleaning: All residual tags (e.g., <p>, <div>, etc.) have been removed using regex patterns.
3. Language Threshold: A strict 70% Burmese character threshold was applied to every row to filter out heavy English or foreign language noise.
4. Deduplication: Repeated contexts and duplicate entries were removed to prevent model overfitting.
5. Sentence Refinement: - Sentences are split based on the Burmese period (။).
   • Extremely short fragments (less than 10 characters) were intelligently merged with adjacent sentences to maintain contextual meaning.
6. Noise Mitigation: Known noise patterns like "မေး။ ။" and "ဖြေ။ ။" have been stripped out.

Note: While significant efforts were made to clean the data, users might still encounter minor noise due to the nature of historical transcriptions in the source material.

Dataset Structure

Column	Description
id	Unique sequential identifier for each row.
categories	Categories assigned to the original Wikisource article.
title	The title of the original article/document.
context	The cleaned Burmese sentence or paragraph.

Licensing Information

This dataset is released under the MIT License. You are free to use, modify, and distribute it, provided that the original credit is given to the creators.

Contribution & Support

Developed by Kalix Louiis under the DatarrX organization. We aim to build robust open-source datasets for the Burmese NLP ecosystem.

• Organization Profile: DatarrX
• Author Profile: Khant Sint Heinn(Kalix Louis)

If you find this dataset useful, please consider giving it a ❤️ on the Hugging Face repository!

Citation

If you use this dataset, please cite it using the following formats:

APA: Khant Sint Heinn (Kalix Louis). (2026). MyWikisource: A Large-Scale High-Quality Burmese Corpus (Version 1.0) [Data set]. DatarrX. https://huggingface.co/datasets/DatarrX/mywikisource

@dataset{khant_sint_heinn_2026_mywikisource,
  author       = {Khant Sint Heinn (Kalix Louis)},
  title        = {MyWikisource: A Large-Scale High-Quality Burmese Corpus},
  year         = {2026},
  publisher    = {Hugging Face},
  organization = {DatarrX},
  version      = {1.0},
  url          = {https://huggingface.co/datasets/DatarrX/mywikisource}
}

About the Author

Khant Sint Heinn, working under the name Kalix Louis, is a Machine Learning Engineer focused on Natural Language Processing (NLP), data foundations, and open-source AI development. His work is centered on improving support for the Burmese (Myanmar) language in modern AI systems by building high-quality datasets, practical tools, and scalable infrastructure for language technology.

He is currently the Lead Developer at DatarrX, where he develops data pipelines, manages large-scale data collection workflows, and helps create open-source resources for researchers, developers, and organizations. His experience includes data engineering, web scripting, dataset curation, and building systems that support real-world machine learning applications.

Khant Sint Heinn is especially interested in advancing low-resource languages and making AI more accessible to underrepresented communities. Through his open-source contributions, he works to strengthen the Burmese (Myanmar) tech ecosystem and provide reliable building blocks for future language models, search systems, and intelligent applications.

His goal is simple: to turn limited language resources into practical opportunities through clean data, useful tools, and community-driven innovation.

Connect with the Author:
GitHub | Hugging Face | Kaggle