IsiZulu News (articles and headlines) and Siswati News (headlines) Corpora - za-isizulu-siswati-news-2022
Give Feedback 📑: DSFSI Resource Feedback Form
Dataset for both isiZulu news (articles and headlines) and Siswati news headlines. Process included scraping the data from internet, from Isolezwe news website http://www.isolezwe.co.za and public posts from the SABC news LigwalagwalaFM Facebook page https://www.facebook.com/ligwalagwalafm/ respectively.
The obtained datasets are isiZulu news articles, isiZulu news headlines, and Siswati news headlines.
Post data collection the datasets were then sent to annotators, and they were sent back after the annotation process. The datasets contain special characters, some English words and characters that are not ASCII encoded which must be removed prior to model training. The aim of these three datasets is to create a baseline news categorisation model for the two South African low resources languages i.e. isiZulu and Siswati.
For categorisation, we use high level IPTC NewsCodes as categories. You can view the news categories here data/news-categories-iptc-newscodes.csv
The datasets were found to have class categories with very few observations, hence the class categories which have less than 35 observations were removed for isiZulu and less 6 observations for Siswati.
The dataset has both full category data as well as reduced category data.
Please see the data-statement.md for full dataset information.
- Link to the DOI data repository - Zenodo Data Repository
- Andani Madodonga
- Vukosi Marivate - @vukosi
- Matthew Adendorff
See also the list of contributors who participated in this project.
Citation:
@article{Madodonga_Marivate_Adendorff_2023, title={Izindaba-Tindzaba: Machine learning news categorisation for Long and Short Text for isiZulu and Siswati}, volume={4}, url={https://upjournals.up.ac.za/index.php/dhasa/article/view/4449}, DOI={10.55492/dhasa.v4i01.4449}, author={Madodonga, Andani and Marivate, Vukosi and Adendorff, Matthew}, year={2023}, month={Jan.} }
Data is Licensed under CC 4.0 BY SA Code is Licences under MIT License.