Meta info

Objective

Create a crowd-sourced repository for NLP sources for native languages of Sri Lanka.

Target languages

Sinhala and Tamil

What to add

• Datasets
• Research publications
• Open source implementations
• Useful articles, blog posts
• Other linguistic resources (e.g. Sinhala/Tamil phonology)
• Tools (or links to proposals) for crowdsourced data collection

How to contribute

• Send a pull request to lklangs.github.io with your changes or send an email to lklangs2019@gmail.com
• Use Browserling tool to generate HTML from README.md.

To do

• Resource templates
• Structuring
• Update website

Resources

Publications

• TICO-19: the Translation Initiative for COvid-19 2020
• The FLoRes Evaluation Datasets for Low-Resource Machine Translation: Nepali-English and Sinhala-English 2019
• Survey on Publicly Available Sinhala Natural Language Processing Tools and Research 2019
• Natural Language Processing for Government: Problems and Potential 2019
• Two New Evaluation Datasets for Low-Resource Machine Translation: Nepali-English and Sinhala-English
• Comparison Between Performance of Various Database Systems for Implementing a Language Corpus 2015
• Implementing a Corpus for Sinhala Language 2015
• Sinhala Text Classification: Observations from the Perspective of a Resource Poor Language 2015
• Sinhala Handwriting Recognition Mechanism Using Zone Based Feature Extraction 2015
• Sinhala-Tamil Machine Translation: Towards better Translation Quality 2014
• Building a WordNet for Sinhala 2014
• Corpus-based Sinhala Lexicon 2009

Datasets

• Translation Initiative for COVID-19 (English-Sinhala)
• Facebook Low Resource MT Benchmark (FLoRes)
• Large Sinhala ASR training data set
• Sinhala TTS
• SinMin Dataset
• Language Technology Research Lab University of Colombo School of Computing
• Small Sinhala newes corpus
• Ananya Sinhala NER Dataset
• Tamil-Sinhala-Emotion-Analysis
• Google Dakshina Dataset for 12 South Asian languages including Sinhala and Tamil
• Databricks Dolly 15k Sinhala Dataset a machine translated version of Databricks 15k dataset.
• Sinhala Question Answering Dataset 1k a Sinhala QA dataset with English translations.
• Sinhala Wikipedia 202306 Sinhala wikipedia according to 2023 June dump in huggingface datasets format.
• Fasttext bulit by Facebook using wikipedia.
  • Sinhala: bin, text
  • Tamil: bin, text
  • Fasttext bulit by CSE, UoM using wikipedia, News, and official government documents
  • Sinhala: bin
  • Tamil: bin
  • Sinhala-English parallel corpora
  • Subtitle pairs (600k+)
  • Sentences pairs (45k+): GNOME, KDE , and Ubuntu
  • Filtered Sinhala Wikipedia (155k+ sentences)
  • Sinhala common crawl (5m+ sentences)
  • Sinhala-POS-Data
  • Sinhala-Tamil-Aligned-Parallel-Corpus
  • Sinhala Lexicon
  • Sinhala TTS Textnorm grammer
  • Sinhala TTS Phonology
  • Sinhala G2P

Tools

• Machine Translation System for Sinhala -Tamil Language Pair
• MIDAS-NMT-English-Tamil
• Sentiment Analysis of Sinhala News Comments
• National Langauges Processing Centre (UoM) on Github
• Sinhala Sentence Similarity Measurement
• Tamil Emotion Tweet Scraper
• Morphological analyser and tokenizer for Sinhala nuons (SinLing)
• Sinhala-and-Tamil-NER
• Tamil-Tokeniser
• Sinhala-Tokeniser
• Sinhala skipgram model
• Sinhala TTS Recipe
• Sinhala ASR Recipe

Research Groups

• Local Languages Working Group by LK Domain Registry
• NLP gruop of UoM
• Language Technology Research Laboratory (LTRL) of UCSC