iSuara 守语

Resource QR Code
Mr. Ong looking through the side mirror
The Victim

Meet
Mr. Ong.

  • 4.9-Star Grab Driver.
  • He is Deaf.
  • On May 28, 2024,
    he was assaulted.
Problem Statement

Absence of two-way communication.

Education
85%

of deaf students experience severe academic isolation.

Healthcare
7x

more likely to face dangerous medical misdiagnoses.

The Workforce (MY)
Only 0.3%

of PwDs are in the civil services, compared to the supposed 1% national quota.

Sources: Utah Public Radio (2023), National Association of the Deaf, & The Star Malaysia (2024)

ı Suara

Built for Low Spec Phone

Zero extra hardware
Offline ML
Auto Translation
SignAvatar
Developed by sudo rm -rf /
Sign → Voice
Text → Sign
Gonka Network Inference Gateway

GonkaRouter Multi-Model Consensus Feel the emotion, not just a cold robot LLM

Input Layer

Multimodal Extraction

Sign → Voice
Sign + Emotion
Text → Sign
Malay Text
Routed via Gonka Gateway
Sign → Voice
Text → Sign
Cross-Model Verification

3-Way Consensus Architecture

Input Prompt
Sign + Emotion EfficientNet-B0 / Text
GonkaRouter Logo
Gonka Router Inference Gateway
Gonka Node A
DeepSeek Logo

DeepSeek-V4-Flash

Gonka Node B
MiniMax Logo

MiniMax-M2.7

Gonka Node C
Kimi Logo

Kimi-K2.6

Consensus Logic

Consensus Arbiter: DeepSeek-V4-Flash

→ 1 Natural Sentence → 1 Sign Gloss Sequence
Consensus Verified
Gonka Req ID: #req-dee-400951

System Architecture

SignAvatar 3D

OpenGL ES IK Solver 60 FPS LERP
  • Text-to-Sign 3D Engine
  • • Real-time anatomical IK
  • • Sub-frame fluid co-articulation
Machine Learning Model

Bidirectional LSTM
+ Dot-Attention

94.93% Accuracy
BiLSTM
92.4%
Transformer
91.6%
GRU
87.3%
CNN
86.8%
LSTM
86.1%

LiteRT100% Offline

  • 1.4x Faster than legacy TensorFlow
  • • with NPU & GPU Acceleration
  • • Tested 28ms Latency per Frame

Feature Engineered Velocity & Acceleration

  • Recognize 2x fast signs
  • • Adapts to both fast and slow signers
  • • By understanding motion dynamics instead of static frames

MediaPipe

75 Landmarks
  • • 33 Pose + 21 Hand + 21 Hand
  • • Hand-Cropping to improve speed

GonkaRouter

  • BIM has independent grammar rules and syntax
  • • Routed to 3 models for reasoning
  • • Contextual NLP to understand emotion
"Perut" + "Sakit"
"Tolong, saya sakit perut teruk."

EfficientNet-B0

  • • Able understand emotions based on facial expression
The Impact

AI For Society

44,000
Deaf Benefited Nationwide accessibility & independence
70M+
Worldwide Impact Over 70 million deaf around the whole world

Students

Students Can Ask Questions Anytime

Allows them to interrupt and participate in live lectures.

Workforce

Deaf Professionals Save on Costly Interpreters

Allows deaf professionals to communicate instantly for any meeting.

Healthcare

Direct Doctor-Patient Communication, Protecting Privacy

Direct communication between doctor and patient.

Competitive Analysis

Practical Innovation

Scroll table horizontally to compare
Feature Jari / MyBIM BrightSign Glove UKM Webcam Translator Research iSuara
Direction Text Avatar Gesture Text Sign Text
Sign Voice Text Avatar
Connectivity Online only Online only Online only Offline ML
ML Dataset - - 12 198
Hardware Phone only Expensive gloves Webcam Phone only
Language BIM ASL only BIM BIM (+ expand)
Semantic AI
User Validated Design

User Feedback & Iteration

Tested with UKM
Speech Therapy Undergraduates

specialized in Bahasa Isyarat Malaysia.

Latest Metric
94.93% (BiLSTM Attention)
35 FPS (Parallel GPU & NPU)
1.5m (Multi-Distance Trained)
Results
Precise detection even at 2x fast sign
Smoothly captures fast signs
Diverse range and angle detection
Interview with UKM Speech Therapy Undergraduate
UKM Speech Therapy Undergraduates
Sign Language Tutorial Class at UKM

ı Suara

Built for Low Spec Phone

Zero extra hardware
Offline ML
Auto Translation
SignAvatar
Developed by sudo rm -rf /
Sign → Voice
Text → Sign

How many million deaf people in the world?

430M

Worldwide with hearing loss (WHO estimate)

70M+

Belong to signing deaf communities (WFD)

There is no official count of the number of deaf people in the world. Estimates from the World Health Organisation mention that there are around 430 million deaf persons worldwide. This includes all people with a hearing loss, however small. The WFD estimates more than 70 million deaf people belong to signing deaf communities. We are working to develop reliable data on this issue.

Realities of Mainstream Schooling for Deaf Children

A schoolboy with a cochlear implant listens to his teacher during lessons at a school for the hearing impaired in Germany.
Eckehard Schulz / AP

A schoolboy with a cochlear implant listens to his teacher during lessons at a school for the hearing impaired in Germany.

85% of deaf children attend mainstream public schools

Grounds Problem Statement 1: Severe academic & communicative isolation without accessible real-time translation.

85% of deaf children attend mainstream public schools and many deaf advocates will say this is a good thing. Deafness is, after all, part of our society and deaf children need to be part of the world. But good intentions and good educational practices are two different things. And many deaf children don't receive the same quality of education as their hearing classmates.

Only 0.3% of Public Sector Workforce Are Disabled

The Star - Minister Dr Zaliha Mustafa addressing Dewan Rakyat
The Star News

Minister in the Prime Minister's Department Dr Zaliha Mustafa addressing the Dewan Rakyat with real-time sign language interpretation (BIM) in Parliament.

0.3% Actual PwD

Representation in Malaysia's public sector.

1.0% Policy Quota

Target civil service quota for OKU.

The public sector has yet to fill 1% of its positions with persons with disabilities (OKU), said Minister in the Prime Minister’s Department (Federal Territories) Dr Zaliha Mustafa. While there was an increasing trend in OKU civil service employment, the participation rate remains at approximately 0.3%, highlighting the urgent necessity for accessible assistive communication technologies like iSuara to break workplace barriers.

Accuracy and Loss

Model Accuracy and Loss Graph

Confusion Matrix

Confusion Matrix
The Future

Future Roadmap

Q3 2026

Core Expansions

Vocabulary Expansion Scaling Dataset
Video Call Integrations
Zoom Meet Teams
Q4 2028

Enterprising

Pay-As-You-Go API Metered Billing
Enterprise Targets
Grab Banks Clinics
Q1 2030

International Expansion

Malaysia BIM Indonesia BISINDO Singapore SgSL Philippines FSL Thailand TSL Vietnam VSL Cambodia CBDSL Laos LSL Myanmar MSL Timor-Leste LGT
India Pakistan
IPSL
Brunei Brunei

User Feedback

"My hand sign is not detected when I signed just a bit faster"

E

Emmanual

UKM Speech Therapy Undergraduate

"When I stand a bit further, the app unable to detect my hand sign"

L

Lim Pei Ying

UKM Speech Therapy Undergraduates

"This app have a great use of gemini api to rearrange the sentence format"

L

Lee Ming Dao

UM Computer Science(AI) undergraduate

"The real-time translation is quite fast... The sentence generation is very natural."

T

Tan Pei Pei

Hearing Individuals

"The latency is quiet high, the detection occurs after a few seconds the sign is done"

O

Ong Shi Jie

UKM Speech Therapy Undergraduates

"The app sometimes misintepret sign that looks similar like 'Kakak' to 'Abang'"

L

Low Rou En

UKM Speech Therapy Undergraduate

"This app definitely will benefit the DHH community but the dictionary is still strictly limited"

S

Saw Kaison

UPM EEE undergraduate

"Sometimes it cannot catch up when I sign a bit faster"

C

Ch’ng Wen Yi

UKM Speech therapy undergraduate