TWB Voice Dataset v1.0

Dataset Summary

TWB Voice 1.0 is a multilingual speech corpus containing read speech data in three languages from Nigeria: Hausa, Shuwa Arabic, and Kanuri. This dataset was created as part of the TWB Voice project by CLEAR Global (formerly Translators without Borders) to support automatic speech recognition (ASR) development for underrepresented languages.

Languages

• Hausa (hau): Major language spoken in Nigeria, Niger, and neighboring regions. Also widely used as a lingua franca across West Africa
• Shuwa Arabic (shu): Variety of Arabic spoken by the Shuwa Arab people in the Lake Chad region
• Kanuri (kau): Language spoken around Lake Chad in Nigeria, Niger, Chad, and Cameroon

Supported Tasks

• Automatic Speech Recognition (ASR): Primary intended use case. Our baseline models can be accessed at:
  • Hausa Whisper-small fine-tuned
  • Hausa Wav2Vec-Bert-2.0 fine-tuned
  • Kanuri Whisper-small fine-tuned
  • Kanuri Wav2Vec-Bert-2.0 fine-tuned
• Speaker Recognition: Demographic metadata included for speaker analysis
• Language Identification: Multi-language corpus suitable for language ID tasks
• Speech Synthesis: Can also be used for TTS model training but you can also check our TTS-focused datasets for
  • Hausa
  • Kanuri

Dataset Structure

Data Instances

Each data instance contains:

• Audio file path and audio data
• Sentence text
• Speaker demographic information (age range, gender, education level, country of origin)
• Recording metadata

Data Fields

• id: Unique recording identifier
• path: Audio file path
• sentence: Read text prompt
• task_id: Task identifier for the recorded text prompt
• sentence_source: Source of the text content
• user_id: Speaker identifier
• age: Speaker age category (teens (18+), twenties, thirties, forties, fifties, sixties, seventies, over eighty)
• gender: Speaker gender
• duration: Audio duration in seconds
• locale: Language code
• variant: Language variant/dialect
• education_level: Speaker’s education level
• country_of_origin: Speaker’s country of origin
• created_at: Recording timestamp

Data Splits

Each language configuration contains:

• train: Training set (~80% of approved recordings)
• dev: Development/validation set (~10% of approved recordings)
• test: Test set (~10% of approved recordings)
• rejected: Recordings that failed quality review
• pending: Recordings awaiting quality review

Splits are created to ensure no speaker overlap between train/dev/test sets while maintaining approximately 80/10/10 duration ratios.

Dataset Statistics

Hours summary by language and split:

Hours approved by gender and language:

Usage

Loading the Dataset

from datasets import load_dataset

# Load specific language
hausa_dataset = load_dataset("CLEAR-Global/twb-voice-1.0", "hau")
shuwa_dataset = load_dataset("CLEAR-Global/twb-voice-1.0", "shu")
kanuri_dataset = load_dataset("CLEAR-Global/twb-voice-1.0", "kau")

# Load specific split
train_data = load_dataset("CLEAR-Global/twb-voice-1.0", "hau", split="train")

# Stream data for large datasets
dataset = load_dataset("CLEAR-Global/twb-voice-1.0", "hau", streaming=True)

Data Collection

The dataset was collected through TWB Voice platform coordinated by CLEAR Global. Native speakers were asked to read prompted text in their respective languages.

Quality Control

• All recordings, except for those in rejected and pending splits, underwent human review by native speakers.
• Only approved recordings are included in train/dev/test splits
• Rejected recordings and those pending for review are preserved in separate splits for analysis

Ethical Considerations

• All speakers consented to data collection and open publishing
• Speaker identities are anonymized using user IDs
• No attempts should be made to identify individual speakers
• Data should be used responsibly for technology development

Citation

@dataset{twb_voice_2024,
  title = {TWB Voice 1.0},
  author = {CLEAR Global},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/datasets/CLEAR-Global/twb-voice-1.0}
}

License

The dataset is released under CC BY-NC 4.0 after approval under the Dataset Access Terms. Access is gated for compliance with privacy and data protection laws (e.g., GDPR).

If your intended use of this dataset may be commercial in nature (such as building models into services that you monetize in any way), or you are unsure whether your use complies with the non-commercial restriction, especially for initiatives related to social good or public benefit, we encourage you to contact us to discuss potential licensing options or permissions. We are open to supporting impactful uses beyond the standard license terms. Contact us at tech@clearglobal.org, including a link to the dataset in question and a brief description of your intended use.

Acknowledgments

This dataset was created by CLEAR Global with support from the Patrick J. McGovern Foundation.

Contact

For questions about this dataset, please contact CLEAR Global or open an issue in the dataset repository.