UnVoiced: Translating Indian Sign Language in Real-Time with OpenCV & Python

Breaking Communication Barriers

Communication between the deaf/mute community and non-signers remains a major accessibility gap. In UnVoiced and Sign-Language-Recognition, I built an automated Indian Sign Language (ISL) gesture translator.

Computer Vision Pipeline

The system processes video feeds through a dedicated multi-stage pipeline:

1. Skin Color Segmentation (HSV & YCrCb Color Spaces):

- Eliminates background clutter by isolating human skin tones in the Cr-Cb chrominance plane:

$$133 \le \text{Cr} \le 173 \quad \text{and} \quad 77 \le \text{Cb} \le 127$$

1. Morphological Filtering: Gaussian blur followed by dilation and erosion to fill holes in hand contours.
2. Convex Hull & Defect Extraction: Analyzing peak count between extended fingers to classify alphabetical characters ($A-Z$) and numeric digits ($0-9$).

import cv2
import numpy as np

def extract_hand_contour(frame):
    # Convert to HSV color space
    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
    lower_skin = np.array([0, 20, 70], dtype=np.uint8)
    upper_skin = np.array([20, 255, 255], dtype=np.uint8)
    
    mask = cv2.inRange(hsv, lower_skin, upper_skin)
    mask = cv2.GaussianBlur(mask, (5, 5), 100)
    
    contours, _ = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
    return max(contours, key=lambda c: cv2.contourArea(c)) if contours else None

Speech Synthesis (TTS)

Predicted letters and words are converted to real-time audio output using pyttsx3, enabling fluid two-way dialogue without human translators.