Voice-powered accessibility agent making the web accessible to everyone, especially for blind and visually impaired users.

• Overview
• Key Features
• Quick Start
• Tech Stack
• Project Structure
• Installation
• Development
• Architecture
• Configuration
• API Integration
• Roadmap
• Contributing
• License
• Support

Cupcake is a revolutionary accessibility application that empowers blind and visually impaired users to navigate and interact with any website using only their voice. By combining cutting-edge speech recognition, intelligent audio feedback, and smart content summarization, Cupcake transforms the web browsing experience into an intuitive, voice-controlled interface.

Mission: To break down digital barriers and create a truly inclusive web for everyone, regardless of visual ability.

• ✨ Complete Voice Control – No keyboard shortcuts to memorize, just speak naturally
• ✨ Intelligent Context Awareness – Understands page structure and content relevance
• ✨ Lightning Fast – Optimized for minimal latency and responsive interactions
• ✨ Privacy Focused – All API keys encrypted and stored locally
• ✨ Developer Friendly – Well-documented codebase with TypeScript strict mode

# 1. Clone the repository
git clone https://github.com/Manavarya09/cupcake.git
cd cupcake

# 2. Install dependencies
npm install

# 3. Set up environment
cp .env.example .env

# 4. Start development
npm run dev

That's it! Your Electron app with React frontend will launch automatically.

cupcake/
│
├── electron/                    # Desktop application logic
│   ├── main/
│   │   ├── index.ts            # Application entry point
│   │   ├── windowManager.ts     # Window lifecycle management
│   │   ├── hotkeyManager.ts     # Global hotkey handler
│   │   ├── ipcHandlers.ts       # Inter-process communication
│   │   ├── managers/            # Business logic managers
│   │   │   ├── sessionManager.ts
│   │   │   ├── apiKeyManager.ts
│   │   │   └── openclawManager.ts
│   │   ├── services/            # External service wrappers
│   │   │   ├── whisperService.ts
│   │   │   ├── ttsService.ts
│   │   │   ├── summarizerService.ts
│   │   │   └── openclawClient.ts
│   │   └── utils/               # Helper utilities
│   │       ├── constants.ts
│   │       ├── deviceIdentity.ts
│   │       └── store.ts
│   └── preload/                 # IPC security bridge
│
├── src/                         # React frontend application
│   ├── App.tsx                  # Root component
│   ├── main.tsx                 # DOM entry point
│   ├── windows/                 # Window components
│   │   ├── sightlineBar/       # Main control UI
│   │   ├── config/             # Settings window
│   │   └── borderOverlay/      # Visual overlay
│   └── lib/                     # Frontend utilities
│       └── ipc.ts              # IPC bridge client
│
├── cupcake-app/                    # Marketing website
├── shared/                      # Shared TypeScript types
├── build/                       # Build configuration
├── scripts/                     # Setup & build scripts
│
├── package.json                 # Project dependencies
├── tsconfig.json                # TypeScript config
├── vite.config.ts               # Build configuration
├── eslint.config.js             # Code quality
└── README.md                    # This file

• Node.js 18+ (LTS recommended)
• npm 9+ or yarn/pnpm
• macOS 10.13+ / Windows 10+ / Linux (Ubuntu 18.04+)

git clone https://github.com/Manavarya09/cupcake.git
cd cupcake

npm install

This installs all required packages including Electron, React, Vite, and service integrations.

# Create environment file from template
cp .env.example .env

# Edit with your API credentials
# Required: OpenAI API key for Whisper
# Optional: OpenClaw service endpoint

npm run build:electron

npm run bundle-openclaw

npm run dev

This command:

1. Compiles TypeScript for Electron main process
2. Starts Vite dev server (http://localhost:5173)
3. Launches Electron with live reload
4. Enables hot-module replacement (HMR)

Result: Code changes instantly reflect in the running application.

npm run build

Creates optimized bundles:

• dist-electron/ – Compiled main process
• dist/ – Optimized React bundle

npm run preview

Launch the production build locally for testing.

npm run dev              # Development with hot reload
npm run build            # Production build
npm run preview          # Preview production build
npm run build:electron   # Compile main process only
npm run electron:dev     # Start Electron in dev mode
npm run bundle-openclaw  # Bundle OpenClaw SDK

┌─────────────────────────────────────────────────────┐
│  Electron Main Process                              │
│  ┌──────────────────────────────────────────────┐   │
│  │ Window Manager                               │   │
│  │ - Creates/manages app windows                │   │
│  │ - Routes IPC messages                        │   │
│  └──────────────────────────────────────────────┘   │
│  ┌──────────────────────────────────────────────┐   │
│  │ Hotkey Manager                               │   │
│  │ - Global keyboard shortcuts                  │   │
│  │ - Voice activation triggers                  │   │
│  └──────────────────────────────────────────────┘   │
│  ┌──────────────────────────────────────────────┐   │
│  │ Services Layer                               │   │
│  │ - Whisper (speech recognition)               │   │
│  │ - TTS (text-to-speech)                       │   │
│  │ - Summarizer (content analysis)              │   │
│  │ - OpenClaw Client (backend communication)    │   │
│  └──────────────────────────────────────────────┘   │
│  ┌──────────────────────────────────────────────┐   │
│  │ Manager Layer                                │   │
│  │ - Session (user state)                       │   │
│  │ - API Keys (secure storage)                  │   │
│  │ - OpenClaw (service integration)             │   │
│  └──────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────┘
           ↕ IPC Bridge (Preload)
┌─────────────────────────────────────────────────────┐
│  React Frontend (Renderer Process)                  │
│  ┌──────────────────────────────────────────────┐   │
│  │ Cupcake Bar Window                           │   │
│  │ - Main control interface                     │   │
│  │ - Waveform visualization                     │   │
│  │ - Real-time feedback                         │   │
│  └──────────────────────────────────────────────┘   │
│  ┌──────────────────────────────────────────────┐   │
│  │ Config Window                                │   │
│  │ - Settings management                        │   │
│  │ - API key configuration                      │   │
│  └──────────────────────────────────────────────┘   │
│  ┌──────────────────────────────────────────────┐   │
│  │ Border Overlay                               │   │
│  │ - Visual accessibility indicators            │   │
│  └──────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────┘

• Process Isolation – Preload script enforces strict boundaries
• No Remote Code – Never load untrusted code as remote
• Encrypted Storage – API keys stored securely with Electron Store
• Type Safety – TypeScript strict mode prevents common vulnerabilities

# Required
VITE_DEV_SERVER_URL=http://localhost:5173

# API Keys (add your credentials)
OPENAI_API_KEY=sk-...
OPENCLAW_API_KEY=...
OPENCLAW_ENDPOINT=https://api.openclaw.io

# Optional Settings
LOG_LEVEL=info
ENABLE_METRICS=true

• Purpose: Speech-to-text recognition
• Features: Multi-language support, high accuracy, offline capable
• API Key Required: Yes

• Purpose: Audio output generation
• Features: Customizable rate, tone, language support
• Configuration: Embedded service

• Purpose: AI-powered backend services
• Features: Custom commands, advanced processing
• Configuration: Optional

• Purpose: Real-time bidirectional communication
• Features: Live updates, streaming data
• Configuration: Custom endpoint

• Voice recognition & control
• Basic navigation commands
• TTS output

• Advanced page understanding
• Custom command learning
• Context-aware assistance

• Browser extension support
• Mobile companion app
• Multi-language support

• Plugin system
• User feedback integration
• Open-source community building

// Use TypeScript strict mode
// Prefer interfaces over types
// Write descriptive variable names
// Keep functions focused and small

1. Type Safety – No any types; use proper generics
2. Error Handling – Use try-catch for async operations
3. Testing – Write unit tests for business logic
4. Documentation – Comment complex algorithms
5. Performance – Profile before optimizing

We follow conventional commits:

feat: add new feature
fix: resolve bug
chore: maintenance tasks
docs: documentation updates

# Find process using the port
lsof -i :5173

# Kill the process
kill -9 <PID>

# Clear build cache and rebuild
rm -rf dist-electron
npm run build:electron

# Reinstall dependencies
rm -rf node_modules package-lock.json
npm install

# Restart dev server
npm run dev

• Close unnecessary browser tabs
• Reduce audio buffer size in settings
• Disable auto-summarization if not needed

• Check microphone settings
• Ensure OpenAI API key is valid
• Verify internet connection
• Try speaking more clearly

• Check system audio settings
• Verify TTS service endpoint
• Try restarting the application

We welcome contributions from the community! Here's how to get started:

1. Check Issues for existing discussions
2. Fork the repository
3. Create a feature branch

git checkout -b feature/your-amazing-feature

1. Write clean, tested code
2. Follow our coding standards
3. Update relevant documentation
4. Create descriptive commit messages

git add .
git commit -m "feat: describe your amazing feature"
git push origin feature/your-amazing-feature

1. Provide a clear description of changes
2. Include screenshots/recordings if UI changes
3. Link related issues
4. Respond to review feedback

This project is licensed under the MIT License – see LICENSE file for details.

Users are free to use, modify, and distribute this software, provided they include the original license and copyright notice.

• Issues & Bugs – GitHub Issues
• Discussions – GitHub Discussions
• Documentation – Check the /docs folder for detailed guides

1. Check if the bug already exists
2. Create a new issue with:
   • Clear title and description
   • Steps to reproduce
   • Expected vs. actual behavior
   • Environment details (OS, Node version, etc.)

Create an issue with:

• Clear use case
• Expected behavior
• Suggested implementation (if any)

Cupcake stands on the shoulders of incredible open-source projects:

• Electron – Desktop magic
• React – UI excellence
• Vite – Lightning-fast builds
• TypeScript – Type safety
• OpenAI Whisper – Speech recognition
• Tailwind CSS – Beautiful styling

Cupcake is more than an app—it's a movement toward digital accessibility for all. By combining voice technology, AI, and thoughtful design, we're making the web a place where everyone can thrive, regardless of sight.

Made with care for accessibility. Built by developers. For everyone.