Zenvia: An Integrated System for Personalized Sizing, Color Recommendations, Virtual Wardrobe Management, and Fashion Assistance

Zenvia is a full-stack fashion intelligence platform that unifies computer vision, deep learning, web scraping, and generative AI into a cohesive style ecosystem. Built using a robust Flask backend and an interactive React frontend, Zenvia provides automated size estimation, skin-tone based seasonal color matching, live product search, a virtual wardrobe planner, and an AI-driven personal shopper.

Website: https://huggingface.co/spaces/G-Madhuri/Zenvia_project

Models: https://huggingface.co/G-Madhuri/zenvia-models

---

Core Modules & Features

1. Computer Vision-Based Size Estimation

• Technology: OpenCV, MediaPipe Pose tracking, and rule-based decision trees.
• Mechanism: Tracks critical body landmarks via a real-time webcam feed. Calculates pixel distance ratios (calibrated by inter-pupillary distance) to estimate torso height and shoulder width.
• Outcome: Maps body dimensions against standard sizing profiles (XS to XXXL) to output the user's best clothing fit without manual measurements.

2. Personal Color Analysis

• Technology: 5-fold cross-validated ResNet-50 Convolutional Neural Network (CNN) Ensemble.
• Dataset: Trained on Roboflow's "Eyes, Skin, and Hair Colors" dataset containing 6,770 curated images.
• Validation Accuracy: 94.18%
• Classification: Classifies the user's skin undertone into one of four seasons:
  • Invierno (Winter): Cool, clear undertones. Recommended palettes: Royal Blue, Emerald, Pure White, Fuchsia, Black.
  • Otoño (Autumn): Warm, muted undertones. Recommended palettes: Olive Green, Mustard, Terracotta, Rust, Forest Green.
  • Primavera (Spring): Warm, bright undertones. Recommended palettes: Coral, Peach, Butter Yellow, Aqua.
  • Verano (Summer): Cool, soft undertones. Recommended palettes: Lavender, Powder Blue, Dusty Rose, Soft Pink, Mauve.

3. Smart Product Discovery

• Technology: SerpAPI Google Shopping search engine.
• Mechanism: Converts sizing and color suggestions directly into parameterized search queries. Retrieves real-time results from top Indian retailers (Amazon, AJIO, Flipkart, Myntra), extracting price, rating, thumbnail, and purchase links.

4. Interactive Virtual Wardrobe Manager & Outfit Scheduler

• Technology: Cloudinary, LocalStorage, Chart.js, EmailJS SDK, Flask, Python Background Threads.
• Features:
  • Uploads: Direct image upload to Cloudinary with drag-and-drop interface support.
  • Outfit Builder: Drag-and-drop upper and lower clothing items onto a canvas to preview outfit combinations.
  • Weather Assistant: Leverages the OpenWeather API to display current conditions and recommend suitable outfits.
  • Outfit Scheduling: Users can schedule outfits for future dates with instant email confirmation sent using the EmailJS SDK.
  • Morning Reminder System: A Flask background thread periodically scans scheduled outfits and automatically sends morning reminder emails (6:00 AM–12:00 PM IST) via the EmailJS REST API on the scheduled day.
  • Statistics: Tracks wardrobe insights such as favorites, outfit usage frequency, and clothing distribution using interactive Chart.js visualizations.

5. Conversational AI Fashion Stylist (FashionBot)

• Technology: Mistral Large LLM (routed via HTTP).
• Mechanism: Context-aware styling advisor that responds to queries about seasonal fashion tips, trend guidelines, and customized clothing coordination.

---

Technology Stack

• Backend: Python 3.10+, Flask, Flask-CORS, PyTorch (TorchVision), MediaPipe, OpenCV, Pandas, Numpy, Requests, SerpAPI, Mistral API.
• Frontend: React 18, Vite, React Router v6, Chart.js, Tailwind CSS (CDN).
• Storage: Cloudinary (garment images), JSON file (scheduled outfits), LocalStorage (wardrobe state).
• Deployment: Hugging Face Spaces (Docker-based runtime).

---

Project Directory Structure

zenvia/
├── model.ipynb             # ResNet-50 training notebook (5-fold cross validation)
├── backend/
│   ├── app.py                  # Main Flask API — routing & size estimation
│   ├── fashionbot.py           # FashionBot blueprint integrating Mistral LLM
│   ├── virtual_wardrobe.py     # Virtual wardrobe blueprint + background reminder thread
│   ├── scheduled_outfits.json  # JSON database for scheduled garments & reminder states
│   ├── sizes.csv               # Shoulder & torso to apparel size mapping chart
│   └── requirements.txt        # Backend Python dependencies
│
└── frontend/
    ├── index.html              # Entry document (loads Tailwind CSS via CDN)
    ├── package.json            # Node.js project configuration
    ├── vite.config.js          # Vite build settings
    └── src/
        ├── main.jsx            # React root — router with 6 page routes
        ├── index.css           # Global stylesheet (animations, glassmorphism)
        ├── components/
        │   └── Modal.jsx       # Reusable alert/confirm/prompt modal
        └── pages/
            ├── Home.jsx        # Landing dashboard page with styled hero sections
            ├── Home.css        # Landing dashboard styles
            ├── Camera.jsx      # MediaPipe size estimation interface
            ├── ColorAnalysis.jsx # ResNet-50 skin-tone capture pipeline
            ├── Result.jsx      # Size result & Google Shopping product grid
            ├── Chatbot.jsx     # AI Stylist chat interface
            ├── Chatbot.css     # AI Stylist chat styles
            └── Wardrobe.jsx    # Drag-and-drop wardrobe, scheduler & charts
            └── Wardrobe.css    # Wardrobe manager styles

---

Environment Variables

All keys live in backend/.env. The frontend fetches them at runtime from /api/config — no frontend .env needed.

MISTRAL_API_KEY=
SERPAPI_API_KEY=
OPENWEATHER_API_KEY=
CLOUDINARY_CLOUD_NAME=
CLOUDINARY_UPLOAD_PRESET=
EMAILJS_SERVICE_ID=
EMAILJS_TEMPLATE_ID=
EMAILJS_PUBLIC_KEY=

---

Local Installation & Setup

Prerequisites

• Python 3.10 or higher
• Node.js 18 or higher
• A webcam for size estimation
• API keys for Mistral, SerpAPI, OpenWeatherMap, Cloudinary, and EmailJS

1. Backend

cd backend
python -m venv venv

# Windows
venv\Scripts\activate

# Mac / Linux
source venv/bin/activate

pip install -r requirements.txt

# Copy the template and fill in your keys
cp .env.example .env

python app.py

Flask runs on http://localhost:5000.

2. Frontend

cd frontend
npm install
npm run dev

React runs on http://localhost:5173.

Both terminals must remain open simultaneously during development.

---

Production Build & Single-Port Execution

Compile the React bundle and serve everything through Flask on a single port:

# Step 1 — Build the frontend
cd frontend
npm run build

# Step 2 — Start only the Flask server
cd backend
python app.py

Flask's catch-all route automatically serves frontend/dist/ for all client-side paths. Visit the full platform at http://localhost:5000.

---

Deploying to Hugging Face Spaces

Zenvia can be deployed directly to Hugging Face Spaces using the provided Dockerfile.

Build & Run Mechanism

The container is built as a multi-stage environment:

1. Node.js compiles the production React bundle.
2. A Python stage installs PyTorch (CPU edition), sets up system-level packages for OpenCV (libGL.so), and launches the Gunicorn production WSGI server.
3. PYTHONUNBUFFERED=1 outputs Flask logs to the Space build log stream in real time.

Setup Steps

1. Create a new Docker Space on Hugging Face.
2. Push your project code to the Space repository.
3. Add all required keys under Settings → Repository Secrets.
4. Hugging Face will automatically trigger compilation and launch the space.

---

API Reference

Method	Endpoint	Description
GET	/api	Health check — returns API status
GET	/api/config	Returns runtime config (Cloudinary, EmailJS, Weather keys)
POST	/api/color-analysis	Upload image, returns seasonal class & palette
POST	/api/process-frame	Send base64 webcam frame, returns pose measurements
POST	/api/reset-capture	Reset MediaPipe pose state between sessions
POST	/api/get-products	Product search by size, gender & category
POST	/api/get-color-products	Product search by color, gender & category
POST	/api/chat	FashionBot chat via Mistral LLM
POST	/api/schedule-outfit	Save scheduled outfit to JSON database

---

Model Details

• Architecture: ResNet-50 with custom fully connected head (Linear → ReLU → Dropout(0.5) → Linear)
• Training: 5-fold cross validation, ensemble prediction with accuracy-weighted voting
• Dataset: Roboflow "Eyes, Skin, and Hair Colors" — approximately 6,770 images
• Classes: Invierno (Winter), Otoño (Autumn), Primavera (Spring), Verano (Summer)
• Validation Accuracy: 94.18%

---

EmailJS Template Setup

Create a template in your EmailJS dashboard with the following field mappings:

• To Email: {{to_email}}
• Subject: {{subject}}
• Body (switch to HTML editor <>):

  {{{html_body}}}
  

  Triple braces {{{ }}} allow EmailJS to render raw HTML generated dynamically by the application.

---

Notes

• The serpapi package is installed as google-search-results — run pip install google-search-results, not pip install serpapi.
• Flask must run with threaded=True and use_reloader=False for the camera frame polling to work correctly. Both are set in app.py.
• The Cloudinary upload preset must be set to Unsigned in your Cloudinary dashboard.
• React StrictMode is intentionally not used — double-invocation of useEffect would cause the camera polling interval to conflict with itself.
• The frontend fetches all API keys at runtime from /api/config. No frontend .env file is required.

---

Publication & Conference Presentation

This research was presented at the 1st International Conference on AI Applications and Technological Innovations (ICAIATI-2025) held on 12–13 December 2025 at Chaitanya Bharathi Institute of Technology (CBIT), Hyderabad, India.

• Paper Title: Zenvia: An Integrated System for Personalized Sizing, Color Recommendations, Virtual Wardrobe Management and Fashion Assistance
• Status: Presented and currently under process for publication in partnership with Taylor & Francis Group.
• Conference Website: ICAIATI-2025

---

Authors

• Title: Zenvia: An Integrated System for Personalized Sizing, Color Recommendations, Virtual Wardrobe Management and Fashion Assistance
• Authors: Madhuri Gottumukkala · Peesari Sathvik Reddy
• Institution: Department of Artificial Intelligence & Machine Learning, Chaitanya Bharathi Institute of Technology (CBIT), Hyderabad, India.

---

License

Academic and research use only. All rights reserved by the authors.