01
Motion Capture
The camera records gesture input at 60 fps, isolating hands from the background in real time.
Initializing the gesture gateway…
A Real-Time Hand Gesture-to-Text System for the Hearing and Speech Impaired
Live viewport
Skeletal tracking overlays every joint while text streams out the moment a sign resolves.
Transcription
Hello, my name is Aria. Nice to meet you!|
How it works
01
The camera records gesture input at 60 fps, isolating hands from the background in real time.
02
A neural network maps 21 hand keypoints per frame, tracking joint angles and motion trajectories.
03
Sequences are decoded into natural language and delivered as text and synthesized speech.
Capabilities
Sub-15 ms on-device prediction keeps conversation at human speed — no cloud round-trip.
ASL, BSL, ISL and regional variants, with a growing community-contributed sign corpus.
The full model runs locally, so translation works in classrooms, clinics and transit alike.
Speech is transcribed back into captions and animated signing avatars for true dialogue.
Mission & impact
Over 70 million people sign as their first language, yet most everyday counters, clinics and classrooms cannot answer back. GESTURELINK puts a translator in every pocket — so a hospital visit, a job interview or a conversation with a neighbour happens on equal footing, without a third person in the middle.