20251EIOT7: Dictowriter-Automated Scribe from speech for specially abled

Brief Description and Functionality
Individuals with impairments face increased challenges in both everyday activities and educational environments. Assistive technologies, encompassing both software and hardware, enable individuals to surmount challenges and attain improved results in educational settings as well as in their everyday lives. Dictowriter is a device designed for converting speech into text and write it on paper like an automated scribe.The technology works in the following step-by-step manner:
A. Listens to speech,
B. Converts it into text, and
C. Physically plots (writes or draws) that text using a plotter or robotic arm.
It's a fusion of speech recognition, text processing, and motion control technologies. The primary objective of this project is to facilitate the conversion of speech into text seamlessly and efficiently. The primary benefit of this system is its capability to transcribe spoken content onto paper using a pen. The pen writer equipped with sensors serves two primary functions. Initially, it captures the voice or speech of the user. Subsequently, upon receiving the instructions, the plotter or writing mechanism connected to it begins to transcribe the information onto the paper. The step-by-step guide to make this project has been listed below -
Speech Input & Recognition
Goal: Capture the user's voice and convert it to hand written documents on paper.
Technologies Used:
Microphone to capture audio.
Speech Recognition Engine:
Online: Google Speech API, Whisper, Azure Speech, etc.
Offline: Vosk, CMU Sphinx.
Process:
Microphone → Audio signal → Speech engine → Text output.
Text Processing
Purpose: Clean and format text for plotting.
Tasks:
Punctuation correction.
Word wrapping to fit page width.
Font styling (if supported).
Character vectorization (convert characters to paths).
Plotting Mechanism
Goal: Physically draw the recognized text.
Hardware Options:
XY Plotter
CNC Pen Plotter.
3D printer with pen attachment.
DIY robotic arm with servo motors.
Motion Control Logic:
Text is converted to G-code or SVG paths.
Controller sends step/direction signals to stepper motors.
Pen is moved along X and Y axes, and lifted/pressed via Z-axis or servo motor.
System Flow Diagram
User Speaks → Microphone → Speech Recognition Engine → Text Processor → Vectorization → G-code/SVG Generator → Plotter Controller → Motors Move → Pen Writes on Paper
Key Components
Component | Role |
---|---|
Microphone | Captures voice input |
Speech-to-Text Engine | Converts voice to text |
Text Formatter | Prepares text for plotting |
Font to Path Converter | Converts characters to draw paths |
Plotter Hardware | Moves pen to draw text |
Controller | Sends movement commands to motors |
Example Technologies to Build It
Layer | Tools |
---|---|
Speech Recognition | Whisper, Google API, Vosk |
Programming | Python, Node.js |
Plotting Hardware | AxiDraw, CNC Shield + Arduino, Custom Pen Plotter |
Text-to-Vector | Inkscape (for SVG), Hershey Fonts, FontForge |
Motion Control | GRBL firmware, gcode-sender, custom Python scripts |
Applications
Scribe, or to take notes for specially abled people
Hands-free note-taking
Smart journaling or printing spoken thoughts
Challenges
Real-time processing latency (presently limited to 10-13 words per minute)
Noise filtering for accurate speech recognition
Font complexity affecting plotting speed
Use of proper punctuation
Mechanical precision (pen lift/drop timing)
Market Survey


Useful Links
Skills Required
- CAD design
- 3D printing
- Machining on CNC lathe
- Wood Working
- ESP32 Programming
- PCB design
- Basic Coding in Python
- PCB milling
- SMT soldering/manual soldering
- Electronics testing