Sign Language Translator Frontend

A web GUI of the sign-language-translator python package
and a tool to create sign-language-datasets
made with React.

Support Us ❤️

---

1. Sign Language Translator Frontend
   1. Pages and Features
      1. 1. Translator
      2. 2. Customize
         1. 2.1 Sign Dictionary Annotation
         2. 2.2 Sign Clip Extraction
         3. 2.3 Synthetic Sentences
      3. 3. Learn
         1. 3.1 Walkthrough
         2. 3.2 Courses
      4. 4. Documentation
   2. Local Setup
   3. Directory Tree
   4. License and citation
   5. ToDo / Contribution

Pages and Features

1. Translator

Translate bidirectionally between various text and sign languages using a variety of AI models.

Design	Current

2. Customize

Annotate sign language datasets and finetune AI models.

2.1 Sign Dictionary Annotation

Label video clips of individual words/signs with text gloss & text translation in various spoken languages and export the data as a mapping JSON.

Design	Current

2.2 Sign Clip Extraction

Specify sections of a long video which correspond to individual sentence, phrase or word, label them with text and export the data as mp4 clips a mapping JSON.

Design	Current

2.3 Synthetic Sentences

Arrange sign dictionary videos into sequences and label them with equivalent spoken language texts.

Design	Current

3. Learn

Train yourself to use this tool or teach hearing-impaired students quality lessons.

3.1 Walkthrough

Start a step by step walkthrough on which components to click or watch a video tutorial.

3.2 Courses

Interactive lessons in sign language videos, text & audio.

4. Documentation

Preview of the python library's documentation & research papers.

Local Setup

This react project was created using Vite. To run this project locally, follow these steps:

1. Clone and install the project

   git clone https://github.com/sign-language-translator/slt-frontend.git
   cd slt-frontend
   npm install

2. Start the development server

   npm run start

   Open http://localhost:3000/ to view it in the browser.

3. Run tests

   npm run test

4. Build app for production

   npm run build

Directory Tree

One line summary of each module.

SLT-Frontend

├── LICENSE
├── README.md
├── index.html
├── public
│   └── *
└── src
    ├── App.jsx  routes
    ├── main.jsx
    ├── components
    │   ├── index.jsx  export all components
    │   ├── Avatar
    │   │   └── index.jsx  three.js canvas with animated humanoid performing signs
    │   │
    │   └── TextArea
    │       └── index.jsx  write multilingual text with mic, virtual keyboad & speaker. tag supported & ambiguous tokens. Get synonyms & translation suggestions.
    │
    ├── pages
    │   ├── index.jsx  export all pages
    │   ├── Landing
    │   │   └── index.jsx  Welcome Page
    │   │
    │   └── Translator
    │       └── index.jsx  Bidirectional translation between signs & text
    │
    └── utils
        └── index.jsx  helpers

License and citation

@software{slt2024frontend,
  author       = {Mudassar Iqbal},
  title        = {Frontend for Sign Language Translator: Python Library and AI Framework},
  year         = {2024},
  publisher    = {GitHub},
  howpublished = {\url{https://github.com/sign-language-translator/slt-frontend}},
}

This project is licensed under the Apache 2.0 License. You are permitted to use the project, create modified versions, or incorporate pieces of the code into your own work. Your product or research, whether commercial or non-commercial, must provide appropriate credit to the original author(s) by citing this repository.

ToDo / Contribution

• Implement the features from the above GIFs
• Add React Helmet for SEO
• Add React Router for routing
• Add React Redux for state management
• Add React i18next for internationalization
• Add React Testing Library for testing
• Add React Loadable for code splitting