Ashutosh4902 commited on
Commit
fa550e1
·
verified ·
1 Parent(s): 9f43e96

Upload 6 files

Browse files
Files changed (6) hide show
  1. .dockerignore +42 -0
  2. .gitignore +69 -0
  3. Dockerfile +41 -0
  4. README.md +135 -0
  5. app.py +909 -0
  6. requirements.txt +7 -0
.dockerignore ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ __pycache__
2
+ *.pyc
3
+ *.pyo
4
+ *.pyd
5
+ .Python
6
+ env/
7
+ venv/
8
+ .venv
9
+ pip-log.txt
10
+ pip-delete-this-directory.txt
11
+ .tox/
12
+ .coverage
13
+ .coverage.*
14
+ .cache
15
+ nosetests.xml
16
+ coverage.xml
17
+ *.cover
18
+ *.log
19
+ .git
20
+ .gitignore
21
+ .gitattributes
22
+ .dockerignore
23
+ Dockerfile
24
+ docker-compose*.yml
25
+ .DS_Store
26
+ *.swp
27
+ *.swo
28
+ *~
29
+ .idea/
30
+ .vscode/
31
+ *.egg-info/
32
+ dist/
33
+ build/
34
+ .env
35
+ .env.local
36
+ static/generated/*
37
+ *.md
38
+ !README.md
39
+ .github/
40
+ tests/
41
+ docs/
42
+ examples/
.gitignore ADDED
@@ -0,0 +1,69 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Python
2
+ __pycache__/
3
+ *.py[cod]
4
+ *$py.class
5
+ *.so
6
+ .Python
7
+ build/
8
+ develop-eggs/
9
+ dist/
10
+ downloads/
11
+ eggs/
12
+ .eggs/
13
+ lib/
14
+ lib64/
15
+ parts/
16
+ sdist/
17
+ var/
18
+ wheels/
19
+ pip-wheel-metadata/
20
+ share/python-wheels/
21
+ *.egg-info/
22
+ .installed.cfg
23
+ *.egg
24
+ MANIFEST
25
+
26
+ # Virtual environments
27
+ venv/
28
+ ENV/
29
+ env/
30
+ .venv
31
+ env.bak/
32
+ venv.bak/
33
+
34
+ # IDE
35
+ .vscode/
36
+ .idea/
37
+ *.swp
38
+ *.swo
39
+ *~
40
+ .DS_Store
41
+
42
+ # Environment
43
+ .env
44
+ .env.local
45
+ .env*.local
46
+
47
+ # Generated files
48
+ static/generated/*.png
49
+ static/generated/*.jpg
50
+ *.log
51
+
52
+ # Notebooks
53
+ .ipynb_checkpoints/
54
+ *.ipynb
55
+
56
+ # OS
57
+ .DS_Store
58
+ Thumbs.db
59
+
60
+ # Project specific
61
+ nn_ecommerce_outputs/
62
+ ECOMMERCE_PRODUCT_IMAGES/
63
+ flask_*.log
64
+ instance/
65
+
66
+ # Testing
67
+ .pytest_cache/
68
+ .coverage
69
+ htmlcov/
Dockerfile ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ FROM python:3.11-slim
2
+
3
+ # Set working directory
4
+ WORKDIR /app
5
+
6
+ # Install system dependencies for image processing and ML
7
+ RUN apt-get update && apt-get install -y --no-install-recommends \
8
+ libsm6 \
9
+ libxext6 \
10
+ libxrender-dev \
11
+ libgomp1 \
12
+ && rm -rf /var/lib/apt/lists/*
13
+
14
+ # Copy requirements
15
+ COPY requirements.txt .
16
+
17
+ # Install Python dependencies
18
+ RUN pip install --no-cache-dir -r requirements.txt
19
+
20
+ # Copy application
21
+ COPY . .
22
+
23
+ # Create directories
24
+ RUN mkdir -p static/generated templates
25
+
26
+ # Set environment variables
27
+ ENV FLASK_APP=app.py
28
+ ENV FLASK_ENV=production
29
+ ENV PYTHONUNBUFFERED=1
30
+ ENV HOST=0.0.0.0
31
+ ENV PORT=7860
32
+
33
+ # Expose port
34
+ EXPOSE 7860
35
+
36
+ # Health check
37
+ HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
38
+ CMD python -c "import requests; requests.get('http://localhost:7860/api/status', timeout=5)"
39
+
40
+ # Run application
41
+ CMD ["python", "app.py"]
README.md ADDED
@@ -0,0 +1,135 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: E-commerce Product Classifier
3
+ emoji: 🛍️
4
+ colorFrom: blue
5
+ colorTo: green
6
+ sdk: docker
7
+ app_file: app.py
8
+ pinned: false
9
+ ---
10
+
11
+ # E-commerce Product Classifier with Grad-CAM
12
+
13
+ **Live Product Image Classification using Deep Learning**
14
+
15
+ Production-ready Flask web application for e-commerce product classification using trained Custom CNN, MobileNetV2, and ResNet50 models with real-time Grad-CAM explainability visualization.
16
+
17
+ ## ✨ Features
18
+
19
+ - **3 Trained Models**: Custom CNN, MobileNetV2, ResNet50 (all optimized)
20
+ - **Grad-CAM Visualization**: See exactly which image regions influenced predictions
21
+ - **Real-time Predictions**: Upload any image and get instant results
22
+ - **9 Product Categories**: BABY_PRODUCTS, BEAUTY_HEALTH, CLOTHING_ACCESSORIES_JEWELLERY, ELECTRONICS, GROCERY, HOBBY_ARTS_STATIONERY, HOME_KITCHEN_TOOLS, PET_SUPPLIES, SPORTS_OUTDOOR
23
+ - **Model Comparison**: Side-by-side metrics of all 3 models
24
+ - **Production-Grade**: Thread-safe, auto-cleanup, error handling, logging
25
+ - **Responsive UI**: Works perfectly on desktop, tablet, mobile
26
+ - **Accessible**: WCAG 2.1 compliant (keyboard navigation, screen readers)
27
+
28
+ ## 🚀 How to Use
29
+
30
+ 1. **Upload an Image**: Drag-and-drop or click to select a product image
31
+ 2. **Select Models**: Choose which models to run (or run all 3)
32
+ 3. **Get Predictions**: See confidence scores and Grad-CAM heatmaps
33
+ 4. **Analyze Results**: View model comparisons and explanations
34
+
35
+ ## 📊 Model Performance
36
+
37
+ | Model | Accuracy | Precision | Recall | F1-Score | Size |
38
+ |-------|----------|-----------|--------|----------|------|
39
+ | Custom CNN | 45.47% | 41.17% | 45.47% | 0.3858 | 8.9 MB |
40
+ | MobileNetV2 | 71.76% | 71.13% | 71.76% | 0.7106 | 33 MB |
41
+ | ResNet50 | 76.93% | 77.39% | 76.93% | 0.7680 | 333 MB |
42
+
43
+ ## 🔧 Technical Details
44
+
45
+ - **Framework**: Flask (Python backend)
46
+ - **Models**: TensorFlow/Keras (.keras format)
47
+ - **Input Size**: 224×224 pixels
48
+ - **Classes**: 9 product categories
49
+ - **Explainability**: Grad-CAM overlay visualization
50
+ - **Deployment**: Docker on Hugging Face Spaces
51
+
52
+ ## 📁 Structure
53
+
54
+ ```
55
+ .
56
+ ├── app.py # Flask backend (production-optimized)
57
+ ├── requirements.txt # Python dependencies
58
+ ├── Dockerfile # Container configuration
59
+ ├── README.md # This file
60
+ ├── templates/
61
+ │ └── index.html # HTML template
62
+ ├── static/
63
+ │ ├── css/styles.css # Styling (responsive design)
64
+ │ └── js/app.js # Frontend (retry logic, state mgmt)
65
+ └── nn_ecommerce_outputs/
66
+ ├── models/
67
+ │ ├── custom_cnn.keras
68
+ │ ├── mobilenetv2.keras
69
+ │ └── resnet50.keras
70
+ ├── metadata/
71
+ │ ├── class_names.json
72
+ │ └── model_manifest.json
73
+ └── tables/
74
+ └── (CSV files with metrics)
75
+ ```
76
+
77
+ ## 🛠 Production Features
78
+
79
+ ✅ **Thread-safe model caching** - Safe for concurrent requests
80
+ ✅ **Automatic cleanup** - Old generated files cleaned up
81
+ ✅ **Retry logic** - Network failures handled gracefully
82
+ ✅ **Input validation** - File type & size checks
83
+ ✅ **Structured logging** - Debug everything
84
+ ✅ **Error handling** - User-friendly messages
85
+ ✅ **Progress tracking** - Real-time prediction progress
86
+ ✅ **Accessible UI** - WCAG 2.1 compliant
87
+
88
+ ## 📱 Browser Support
89
+
90
+ - Chrome/Chromium (latest)
91
+ - Firefox (latest)
92
+ - Safari (latest)
93
+ - Edge (latest)
94
+ - Mobile browsers (iOS Safari, Chrome Mobile)
95
+
96
+ ## 🚨 Troubleshooting
97
+
98
+ **Q: Models not loading?**
99
+ A: Check that `.keras` files exist in `nn_ecommerce_outputs/models/`
100
+
101
+ **Q: Predictions are slow?**
102
+ A: First prediction loads models (normal), subsequent are faster
103
+
104
+ **Q: Image upload fails?**
105
+ A: Check file size < 12 MB and format (JPG, PNG, WEBP, BMP)
106
+
107
+ **Q: Want to see logs?**
108
+ A: Check Space Settings → Logs tab for detailed information
109
+
110
+ ## 📚 Resources
111
+
112
+ - [Flask Documentation](https://flask.palletsprojects.com)
113
+ - [TensorFlow/Keras](https://tensorflow.org)
114
+ - [Grad-CAM Paper](https://arxiv.org/abs/1610.02055)
115
+ - [HF Spaces Docs](https://huggingface.co/docs/hub/spaces)
116
+
117
+ ## 📄 License
118
+
119
+ MIT License - Free for academic and commercial use
120
+
121
+ ## 🤝 About This Project
122
+
123
+ This is a comprehensive deep learning project for e-commerce product classification featuring:
124
+ - Custom CNN trained from scratch
125
+ - Transfer learning with MobileNetV2 and ResNet50
126
+ - Extensive explainability analysis with Grad-CAM
127
+ - Production-ready web deployment
128
+
129
+ **Author**: Ashutosh Rajendra Patil
130
+ **Institution**: University of Europe for Applied Sciences
131
+ **Dataset**: Kaggle ecommerce_product_images_18K (18,175 images, 9 categories)
132
+
133
+ ---
134
+
135
+ **Status**: ✅ Production Ready | **Python**: 3.9+ | **TensorFlow**: 2.15.0 | **Updated**: June 2026
app.py ADDED
@@ -0,0 +1,909 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import csv
4
+ import json
5
+ import logging
6
+ import os
7
+ import uuid
8
+ from datetime import datetime, timedelta
9
+ from pathlib import Path
10
+ from functools import lru_cache
11
+ from threading import RLock
12
+
13
+ import numpy as np
14
+ from PIL import Image, ImageOps
15
+ from flask import Flask, jsonify, render_template, request, send_from_directory, url_for
16
+ from werkzeug.utils import secure_filename
17
+
18
+ # ============================================================================
19
+ # CONFIGURATION
20
+ # ============================================================================
21
+
22
+ logging.basicConfig(
23
+ level=logging.INFO,
24
+ format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
25
+ )
26
+ logger = logging.getLogger(__name__)
27
+
28
+ class AppConfig:
29
+ """Production-grade configuration"""
30
+ MAX_UPLOAD_SIZE = 30 * 1024 * 1024 # 30 MB
31
+ PREDICTION_TIMEOUT = 120 # 2 minutes
32
+ GENERATED_FILES_RETENTION_DAYS = 7
33
+ GRADCAM_OVERLAY_BASE_WEIGHT = 0.58
34
+ GRADCAM_OVERLAY_HEATMAP_WEIGHT = 0.42
35
+ MODEL_CACHE_SIZE = 10
36
+
37
+ # ============================================================================
38
+ # PATHS & INITIALIZATION
39
+ # ============================================================================
40
+
41
+ BASE_DIR = Path(__file__).resolve().parent
42
+ PROJECT_DIR = BASE_DIR.parent
43
+ KAGGLE_OUTPUT_DIR = Path("/kaggle/working/nn_ecommerce_outputs")
44
+
45
+ # Look for the artifact folder next to app.py first (the normal layout),
46
+ # then one level up, then the Kaggle path. First match wins.
47
+ def _find_default_artifact_dir():
48
+ candidates = [
49
+ BASE_DIR / "nn_ecommerce_outputs",
50
+ PROJECT_DIR / "nn_ecommerce_outputs",
51
+ KAGGLE_OUTPUT_DIR,
52
+ ]
53
+ for candidate in candidates:
54
+ if candidate.exists():
55
+ return candidate
56
+ return BASE_DIR / "nn_ecommerce_outputs"
57
+
58
+ DEFAULT_ARTIFACT_DIR = _find_default_artifact_dir()
59
+
60
+ ARTIFACT_DIR = Path(os.getenv("NN_ARTIFACT_DIR", DEFAULT_ARTIFACT_DIR)).resolve()
61
+ MODEL_DIR = Path(os.getenv("NN_MODEL_DIR", ARTIFACT_DIR / "models")).resolve()
62
+ METADATA_DIR = Path(os.getenv("NN_METADATA_DIR", ARTIFACT_DIR / "metadata")).resolve()
63
+ TABLE_DIR = Path(os.getenv("NN_TABLE_DIR", ARTIFACT_DIR / "tables")).resolve()
64
+ FIGURE_DIR = Path(os.getenv("NN_FIGURE_DIR", ARTIFACT_DIR / "figures")).resolve()
65
+ GENERATED_DIR = BASE_DIR / "static" / "generated"
66
+ DATASET_IMAGE_DIR = ARTIFACT_DIR / "dataset_images"
67
+ DATASET_IMAGE_MANIFEST_PATH = TABLE_DIR / "dataset_images_manifest.csv"
68
+ LOCAL_DATASET_DIR = PROJECT_DIR / "ECOMMERCE_PRODUCT_IMAGES"
69
+
70
+ GENERATED_DIR.mkdir(parents=True, exist_ok=True)
71
+
72
+ ALLOWED_EXTENSIONS = {".jpg", ".jpeg", ".png", ".webp", ".bmp"}
73
+
74
+ # ============================================================================
75
+ # THREAD-SAFE CACHING
76
+ # ============================================================================
77
+
78
+ _cache_lock = RLock()
79
+ _model_cache = {}
80
+ _dataset_cache = None
81
+ _dataset_cache_time = None
82
+ _tf_module = None
83
+ _keras_module = None
84
+
85
+ CACHE_TTL_SECONDS = 3600
86
+
87
+ # ============================================================================
88
+ # FLASK APP
89
+ # ============================================================================
90
+
91
+ app = Flask(__name__)
92
+ app.config["MAX_CONTENT_LENGTH"] = AppConfig.MAX_UPLOAD_SIZE
93
+ app.config["JSON_SORT_KEYS"] = False
94
+
95
+ # ============================================================================
96
+ # UTILITIES
97
+ # ============================================================================
98
+
99
+ def validate_file_extension(filename, allowed=ALLOWED_EXTENSIONS):
100
+ """Validate file extension"""
101
+ if not filename:
102
+ return False
103
+ ext = Path(secure_filename(filename)).suffix.lower()
104
+ return ext in allowed
105
+
106
+ def cleanup_old_generated_files():
107
+ """Remove generated files older than retention period"""
108
+ if not GENERATED_DIR.exists():
109
+ return 0
110
+
111
+ cutoff = datetime.now() - timedelta(days=AppConfig.GENERATED_FILES_RETENTION_DAYS)
112
+ removed = 0
113
+
114
+ try:
115
+ for file in GENERATED_DIR.glob("*.png"):
116
+ try:
117
+ if datetime.fromtimestamp(file.stat().st_mtime) < cutoff:
118
+ file.unlink()
119
+ removed += 1
120
+ except Exception as e:
121
+ logger.warning(f"Failed to delete {file}: {e}")
122
+ except Exception as e:
123
+ logger.error(f"Cleanup failed: {e}")
124
+
125
+ return removed
126
+
127
+ def load_json(path, fallback):
128
+ """Safely load JSON file"""
129
+ try:
130
+ if path.exists():
131
+ with path.open("r", encoding="utf-8") as f:
132
+ return json.load(f)
133
+ except Exception as e:
134
+ logger.warning(f"Failed to load {path}: {e}")
135
+ return fallback
136
+
137
+ def normalize_artifact_path(value, allowed_extensions=None):
138
+ """Normalize and validate artifact path (security)"""
139
+ artifact_path = str(value or "").replace("\\", "/").lstrip("/")
140
+ parts = Path(artifact_path).parts
141
+
142
+ # Block dangerous patterns
143
+ if not artifact_path or ".." in parts or artifact_path.startswith("/"):
144
+ return None
145
+
146
+ # Validate extension if provided
147
+ if allowed_extensions:
148
+ ext = Path(artifact_path).suffix.lower()
149
+ if ext not in allowed_extensions:
150
+ return None
151
+
152
+ # Resolve and check symlinks
153
+ try:
154
+ file_path = (ARTIFACT_DIR / artifact_path).resolve()
155
+ artifact_root = ARTIFACT_DIR.resolve()
156
+ if not str(file_path).startswith(str(artifact_root)):
157
+ return None
158
+ except Exception:
159
+ return None
160
+
161
+ return artifact_path
162
+
163
+ def artifact_path_to_file(artifact_path):
164
+ """Convert normalized path to file with security checks"""
165
+ normalized = normalize_artifact_path(artifact_path)
166
+ if not normalized:
167
+ raise ValueError("Invalid artifact image path.")
168
+
169
+ file_path = (ARTIFACT_DIR / normalized).resolve()
170
+ artifact_root = ARTIFACT_DIR.resolve()
171
+ if file_path != artifact_root and artifact_root not in file_path.parents:
172
+ raise ValueError("Artifact image path is outside the output folder.")
173
+ return file_path
174
+
175
+ # ============================================================================
176
+ # METADATA LOADERS
177
+ # ============================================================================
178
+
179
+ def class_names():
180
+ """Load class names with caching"""
181
+ names = load_json(METADATA_DIR / "class_names.json", [])
182
+ return names if isinstance(names, list) else []
183
+
184
+ def model_manifest():
185
+ """Load model manifest with fallback"""
186
+ manifest = load_json(METADATA_DIR / "model_manifest.json", {})
187
+ models = manifest.get("models", {}) if isinstance(manifest, dict) else {}
188
+
189
+ if not models and MODEL_DIR.exists():
190
+ logger.info("Building model manifest from filesystem")
191
+ for path in sorted(MODEL_DIR.glob("*.keras")):
192
+ key = path.stem.lower().replace(" ", "_")
193
+ models[key] = {
194
+ "file": path.name,
195
+ "safe_name": key,
196
+ "display_name": path.stem.replace("_", " ").title(),
197
+ "last_conv_layer": None,
198
+ }
199
+
200
+ return {
201
+ "input_size": manifest.get("input_size", [224, 224]) if isinstance(manifest, dict) else [224, 224],
202
+ "models": models,
203
+ }
204
+
205
+ def read_csv_table(path, max_rows=12):
206
+ """Read CSV table with error handling"""
207
+ if not path.exists():
208
+ return None
209
+
210
+ try:
211
+ with path.open("r", encoding="utf-8", newline="") as handle:
212
+ reader = csv.DictReader(handle)
213
+ rows = list(next(iter([reader]), []))[:max_rows]
214
+ return {
215
+ "columns": reader.fieldnames or [],
216
+ "rows": rows,
217
+ "url": url_for("output_file", filename=f"tables/{path.name}"),
218
+ }
219
+ except Exception as e:
220
+ logger.warning(f"Failed to read table {path}: {e}")
221
+ return None
222
+
223
+ # ============================================================================
224
+ # DATASET IMAGE MANAGEMENT
225
+ # ============================================================================
226
+
227
+ def dataset_images_manifest():
228
+ """Load dataset images with caching"""
229
+ global _dataset_cache, _dataset_cache_time
230
+
231
+ # Return cached if still valid
232
+ if _dataset_cache is not None and _dataset_cache_time is not None:
233
+ if (datetime.now() - _dataset_cache_time).total_seconds() < CACHE_TTL_SECONDS:
234
+ return _dataset_cache
235
+
236
+ rows = []
237
+
238
+ # Try manifest file first
239
+ if DATASET_IMAGE_MANIFEST_PATH.exists():
240
+ try:
241
+ with DATASET_IMAGE_MANIFEST_PATH.open("r", encoding="utf-8", newline="") as handle:
242
+ reader = csv.DictReader(handle)
243
+ for index, row in enumerate(reader):
244
+ artifact_path = normalize_artifact_path(row.get("artifact_path"))
245
+ if not artifact_path:
246
+ continue
247
+ file_path = ARTIFACT_DIR / artifact_path
248
+ if not file_path.exists():
249
+ continue
250
+ rows.append({
251
+ "id": str(row.get("id") or index),
252
+ "source": "artifact",
253
+ "label": row.get("label") or file_path.parent.name,
254
+ "label_id": row.get("label_id"),
255
+ "filename": row.get("filename") or file_path.name,
256
+ "artifact_path": artifact_path,
257
+ })
258
+ except Exception as e:
259
+ logger.warning(f"Failed to read manifest: {e}")
260
+
261
+ # Fallback to artifact directory
262
+ if not rows and DATASET_IMAGE_DIR.exists():
263
+ try:
264
+ image_files = sorted(
265
+ path for path in DATASET_IMAGE_DIR.rglob("*")
266
+ if path.suffix.lower() in ALLOWED_EXTENSIONS
267
+ )
268
+ for index, file_path in enumerate(image_files):
269
+ rows.append({
270
+ "id": str(index),
271
+ "source": "artifact",
272
+ "label": file_path.parent.name,
273
+ "label_id": None,
274
+ "filename": file_path.name,
275
+ "artifact_path": file_path.relative_to(ARTIFACT_DIR).as_posix(),
276
+ })
277
+ except Exception as e:
278
+ logger.warning(f"Failed to scan artifact directory: {e}")
279
+
280
+ # Fallback to local directory
281
+ if not rows and LOCAL_DATASET_DIR.exists():
282
+ try:
283
+ image_files = sorted(
284
+ path for path in LOCAL_DATASET_DIR.rglob("*")
285
+ if path.suffix.lower() in ALLOWED_EXTENSIONS
286
+ )
287
+ for index, file_path in enumerate(image_files):
288
+ rows.append({
289
+ "id": str(index),
290
+ "source": "local",
291
+ "label": file_path.parent.name,
292
+ "label_id": None,
293
+ "filename": file_path.name,
294
+ "local_path": file_path.relative_to(LOCAL_DATASET_DIR).as_posix(),
295
+ })
296
+ except Exception as e:
297
+ logger.warning(f"Failed to scan local directory: {e}")
298
+
299
+ _dataset_cache = rows
300
+ _dataset_cache_time = datetime.now()
301
+ logger.info(f"Loaded {len(rows)} dataset images")
302
+ return rows
303
+
304
+ def dataset_image_by_id(image_id):
305
+ """Find dataset image by ID"""
306
+ image_id = str(image_id)
307
+ for row in dataset_images_manifest():
308
+ if row["id"] == image_id:
309
+ return row
310
+ return None
311
+
312
+ def dataset_image_url(row):
313
+ """Get URL for dataset image"""
314
+ if row.get("source") == "local":
315
+ return url_for("dataset_image_file", image_id=row["id"])
316
+ return url_for("output_file", filename=row["artifact_path"])
317
+
318
+ def dataset_image_file_path(row):
319
+ """Get file path for dataset image with validation"""
320
+ if row.get("source") == "local":
321
+ local_path = normalize_artifact_path(row.get("local_path"))
322
+ if not local_path:
323
+ raise ValueError("Invalid local dataset image path.")
324
+
325
+ file_path = (LOCAL_DATASET_DIR / local_path).resolve()
326
+ dataset_root = LOCAL_DATASET_DIR.resolve()
327
+ if file_path != dataset_root and dataset_root not in file_path.parents:
328
+ raise ValueError("Local dataset image path outside dataset folder.")
329
+ return file_path
330
+
331
+ return artifact_path_to_file(row["artifact_path"])
332
+
333
+ # ============================================================================
334
+ # TENSORFLOW & MODEL LOADING (THREAD-SAFE)
335
+ # ============================================================================
336
+
337
+ def tensorflow_modules():
338
+ """Load TensorFlow with thread safety"""
339
+ global _tf_module, _keras_module
340
+
341
+ with _cache_lock:
342
+ if _tf_module is not None and _keras_module is not None:
343
+ return _tf_module, _keras_module
344
+
345
+ try:
346
+ logger.info("Loading TensorFlow...")
347
+ import tensorflow as tf
348
+ # The models were saved with standalone Keras 3 (their config refers
349
+ # to keras.src.models.functional). Loading them through tensorflow.keras
350
+ # fails with "Could not deserialize class 'Functional'". So prefer the
351
+ # standalone keras package and fall back to tf.keras only if absent.
352
+ try:
353
+ import keras
354
+ logger.info(f"Using standalone Keras {keras.__version__}")
355
+ except ImportError:
356
+ from tensorflow import keras
357
+ logger.info("Using tensorflow.keras (standalone keras not found)")
358
+ logger.info(f"TensorFlow {tf.__version__} loaded")
359
+ # Models were trained with mixed_float16 precision (see Kaggle notebook).
360
+ try:
361
+ keras.mixed_precision.set_global_policy("mixed_float16")
362
+ logger.info("Mixed precision policy set: mixed_float16")
363
+ except Exception as policy_exc:
364
+ logger.warning(f"Could not set mixed_float16 policy: {policy_exc}")
365
+ except ImportError:
366
+ logger.error("TensorFlow not installed")
367
+ raise RuntimeError("TensorFlow not installed. Run: pip install tensorflow")
368
+ except Exception as exc:
369
+ logger.error(f"TensorFlow initialization failed: {exc}")
370
+ raise RuntimeError(f"TensorFlow init failed: {exc}") from exc
371
+
372
+ _tf_module = tf
373
+ _keras_module = keras
374
+ return _tf_module, _keras_module
375
+
376
+ def model_path_for(model_info):
377
+ """Get model file path"""
378
+ return MODEL_DIR / model_info["file"]
379
+
380
+ def load_model(model_key):
381
+ """Load model with thread-safe caching"""
382
+ with _cache_lock:
383
+ if model_key in _model_cache:
384
+ logger.debug(f"Using cached model: {model_key}")
385
+ return _model_cache[model_key]
386
+
387
+ manifest = model_manifest()
388
+ models = manifest["models"]
389
+
390
+ if model_key not in models:
391
+ logger.error(f"Unknown model: {model_key}")
392
+ raise KeyError(f"Unknown model: {model_key}")
393
+
394
+ model_info = models[model_key]
395
+ model_path = model_path_for(model_info)
396
+
397
+ if not model_path.exists():
398
+ logger.error(f"Model file missing: {model_path}")
399
+ raise FileNotFoundError(f"Model not found: {model_path}")
400
+
401
+ try:
402
+ logger.info(f"Loading model: {model_key} from {model_path}")
403
+ tf, keras = tensorflow_modules()
404
+ # The transfer models (MobileNetV2, ResNet50) contain a
405
+ # Lambda(preprocess_input) layer that was saved by name only, so the
406
+ # actual function must be supplied via custom_objects. The Custom CNN
407
+ # has no Lambda and needs nothing extra.
408
+ custom_objects = {}
409
+ try:
410
+ if model_key == "mobilenetv2":
411
+ from keras.applications.mobilenet_v2 import preprocess_input as _pre
412
+ custom_objects["preprocess_input"] = _pre
413
+ elif model_key == "resnet50":
414
+ from keras.applications.resnet50 import preprocess_input as _pre
415
+ custom_objects["preprocess_input"] = _pre
416
+ except Exception as pre_exc:
417
+ logger.warning(f"Could not import preprocess_input for {model_key}: {pre_exc}")
418
+ # safe_mode=False is required for the Lambda layers; compile=False
419
+ # skips the optimizer state we don't need for inference.
420
+ try:
421
+ model = keras.models.load_model(
422
+ str(model_path), safe_mode=False, compile=False,
423
+ custom_objects=custom_objects or None,
424
+ )
425
+ except TypeError:
426
+ model = keras.models.load_model(
427
+ str(model_path), custom_objects=custom_objects or None,
428
+ )
429
+ logger.info(f"Model loaded: {model_key}")
430
+
431
+ _model_cache[model_key] = (model, model_info)
432
+
433
+ # Limit cache size
434
+ if len(_model_cache) > AppConfig.MODEL_CACHE_SIZE:
435
+ oldest = next(iter(_model_cache))
436
+ del _model_cache[oldest]
437
+ logger.debug(f"Removed oldest cached model: {oldest}")
438
+
439
+ return model, model_info
440
+ except Exception as e:
441
+ logger.error(f"Failed to load model {model_key}: {e}")
442
+ raise
443
+
444
+ # ============================================================================
445
+ # IMAGE PROCESSING
446
+ # ============================================================================
447
+
448
+ def prepare_image(file, input_size):
449
+ """Prepare uploaded image for prediction"""
450
+ try:
451
+ image = Image.open(file).convert("RGB")
452
+ original_size = image.size
453
+
454
+ image = ImageOps.fit(image, input_size, Image.Resampling.LANCZOS)
455
+ # Models contain their own Rescaling / preprocess_input layers,
456
+ # so they expect raw 0-255 float pixels (matches the Kaggle notebook).
457
+ image_array = np.array(image, dtype="float32")
458
+
459
+ # Save temp preview
460
+ preview_name = f"preview_{uuid.uuid4().hex}.png"
461
+ preview_path = GENERATED_DIR / preview_name
462
+ image.save(preview_path)
463
+
464
+ logger.info(f"Image processed: {original_size} -> {input_size}")
465
+ return image_array, url_for("static", filename=f"generated/{preview_name}")
466
+ except Exception as e:
467
+ logger.error(f"Image preparation failed: {e}")
468
+ raise ValueError(f"Invalid image file: {e}")
469
+
470
+ def prepare_artifact_image(image_path, input_size, image_url):
471
+ """Prepare artifact image for prediction"""
472
+ try:
473
+ image = Image.open(image_path).convert("RGB")
474
+ image = ImageOps.fit(image, input_size, Image.Resampling.LANCZOS)
475
+ image_array = np.array(image, dtype="float32")
476
+ return image_array, image_url
477
+ except Exception as e:
478
+ logger.error(f"Artifact image preparation failed: {e}")
479
+ raise ValueError(f"Failed to load image: {e}")
480
+
481
+ # ============================================================================
482
+ # GRAD-CAM & PREDICTIONS
483
+ # ============================================================================
484
+
485
+ def colorize_heatmap(heatmap):
486
+ """Convert grayscale heatmap to color"""
487
+ try:
488
+ import matplotlib
489
+ try:
490
+ cmap = matplotlib.colormaps["jet"] # matplotlib >= 3.7
491
+ except AttributeError:
492
+ import matplotlib.cm as cm
493
+ cmap = cm.get_cmap("jet") # older matplotlib
494
+ return cmap(heatmap)[:, :, :3]
495
+ except Exception:
496
+ # Fallback: manual red colorization
497
+ colored = np.zeros((*heatmap.shape, 3))
498
+ colored[:, :, 0] = heatmap # Red channel
499
+ return colored
500
+
501
+ def _find_last_4d_layer_name(model):
502
+ """Last layer whose output is 4D (B,H,W,C) — matches the notebook."""
503
+ for layer in reversed(model.layers):
504
+ try:
505
+ if len(layer.output.shape) == 4:
506
+ return layer.name
507
+ except Exception:
508
+ continue
509
+ return None
510
+
511
+ def make_gradcam_heatmap(model, img_array, pred_index, preferred_layer_name=None):
512
+ """Generate Grad-CAM heatmap"""
513
+ try:
514
+ tf, keras = tensorflow_modules()
515
+
516
+ # Prefer the manifest's layer if it actually exists in this model,
517
+ # otherwise auto-detect the last 4D feature layer (notebook behavior).
518
+ last_conv_layer_name = None
519
+ if preferred_layer_name:
520
+ try:
521
+ model.get_layer(preferred_layer_name)
522
+ last_conv_layer_name = preferred_layer_name
523
+ except Exception:
524
+ logger.info(f"Manifest layer '{preferred_layer_name}' not found; auto-detecting.")
525
+ if not last_conv_layer_name:
526
+ last_conv_layer_name = _find_last_4d_layer_name(model)
527
+
528
+ if not last_conv_layer_name:
529
+ logger.warning("No 4D feature layer found for Grad-CAM")
530
+ return None, None
531
+
532
+ last_conv_layer = model.get_layer(last_conv_layer_name)
533
+ grad_model = keras.models.Model(
534
+ model.inputs, [last_conv_layer.output, model.output]
535
+ )
536
+
537
+ with tf.GradientTape() as tape:
538
+ conv_outputs, predictions = grad_model(np.expand_dims(img_array, axis=0), training=False)
539
+ predictions = tf.cast(predictions, tf.float32)
540
+ loss = predictions[:, pred_index]
541
+
542
+ grads = tape.gradient(loss, conv_outputs)
543
+ # Under mixed_float16 these can be float16; cast for stable math.
544
+ conv_outputs = tf.cast(conv_outputs, tf.float32)
545
+ grads = tf.cast(grads, tf.float32)
546
+ pooled_grads = tf.reduce_mean(grads, axis=(0, 1, 2))
547
+
548
+ conv_outputs = conv_outputs[0]
549
+ heatmap = conv_outputs @ pooled_grads[..., tf.newaxis]
550
+ heatmap = tf.squeeze(heatmap)
551
+ heatmap = tf.nn.relu(heatmap)
552
+ heatmap /= tf.math.reduce_max(heatmap) + 1e-10
553
+
554
+ return heatmap.numpy(), last_conv_layer_name
555
+ except Exception as e:
556
+ logger.warning(f"Grad-CAM generation failed: {e}")
557
+ return None, None
558
+
559
+ def save_gradcam_overlay(image_array, heatmap, model_key):
560
+ """Save Grad-CAM overlay visualization"""
561
+ try:
562
+ heatmap_image = Image.fromarray(np.uint8(heatmap * 255)).resize(
563
+ (image_array.shape[1], image_array.shape[0]),
564
+ Image.Resampling.BILINEAR,
565
+ )
566
+ heatmap_resized = np.asarray(heatmap_image).astype("float32") / 255.0
567
+ colored_heatmap = colorize_heatmap(heatmap_resized)
568
+ # image_array is raw 0-255; scale to 0-1 only for display (matches notebook)
569
+ base = np.clip(image_array / 255.0, 0, 1)
570
+ overlay = np.clip(
571
+ (AppConfig.GRADCAM_OVERLAY_BASE_WEIGHT * base) +
572
+ (AppConfig.GRADCAM_OVERLAY_HEATMAP_WEIGHT * colored_heatmap),
573
+ 0, 1
574
+ )
575
+
576
+ overlay_name = f"gradcam_{model_key}_{uuid.uuid4().hex}.png"
577
+ overlay_path = GENERATED_DIR / overlay_name
578
+ Image.fromarray(np.uint8(overlay * 255)).save(overlay_path)
579
+
580
+ logger.debug(f"Grad-CAM saved: {overlay_name}")
581
+ return url_for("static", filename=f"generated/{overlay_name}")
582
+ except Exception as e:
583
+ logger.error(f"Grad-CAM save failed: {e}")
584
+ return None
585
+
586
+ def top_predictions(predictions, labels, top_n=5):
587
+ """Get top N predictions"""
588
+ top_indices = np.argsort(predictions)[-top_n:][::-1]
589
+ return [
590
+ {
591
+ "label": labels[idx] if idx < len(labels) else f"Unknown {idx}",
592
+ "confidence": float(predictions[idx]),
593
+ "confidence_pct": int(predictions[idx] * 100),
594
+ }
595
+ for idx in top_indices if predictions[idx] > 0
596
+ ]
597
+
598
+ def shap_figure_url(model_key):
599
+ """Return URL for the precomputed SHAP reference figure, if present."""
600
+ candidate = FIGURE_DIR / f"shap_{model_key}.png"
601
+ if candidate.exists():
602
+ return url_for("figure_file", filename=candidate.name)
603
+ return None
604
+
605
+ def predict_with_models(image_array, image_url, selected_models):
606
+ """Run predictions with all selected models"""
607
+ labels = class_names()
608
+ manifest = model_manifest()
609
+
610
+ if not selected_models:
611
+ selected_models = list(manifest["models"].keys())
612
+
613
+ results = []
614
+ for model_key in selected_models:
615
+ try:
616
+ logger.info(f"Running inference: {model_key}")
617
+ model, model_info = load_model(model_key)
618
+ predictions = model.predict(np.expand_dims(image_array, axis=0), verbose=0)[0]
619
+ top_rows = top_predictions(predictions, labels)
620
+ best_index = int(np.argmax(predictions))
621
+
622
+ heatmap, layer_name = make_gradcam_heatmap(
623
+ model, image_array, best_index,
624
+ preferred_layer_name=model_info.get("last_conv_layer"),
625
+ )
626
+ heatmap_url = save_gradcam_overlay(image_array, heatmap, model_key) if heatmap is not None else None
627
+
628
+ results.append({
629
+ "model_key": model_key,
630
+ "display_name": model_info.get("display_name", model_key),
631
+ "top_label": top_rows[0]["label"],
632
+ "confidence": top_rows[0]["confidence"],
633
+ "confidence_pct": top_rows[0]["confidence_pct"],
634
+ "top_predictions": top_rows,
635
+ "heatmap_url": heatmap_url,
636
+ "gradcam_layer": layer_name,
637
+ "model_file": model_info.get("file"),
638
+ "error": None,
639
+ })
640
+ logger.info(f"Inference complete: {model_key} -> {top_rows[0]['label']}")
641
+
642
+ except KeyError as e:
643
+ logger.warning(f"Model config error {model_key}: {e}")
644
+ results.append({
645
+ "model_key": model_key,
646
+ "display_name": manifest["models"].get(model_key, {}).get("display_name", model_key),
647
+ "error": f"Model not found: {model_key}",
648
+ })
649
+ except FileNotFoundError as e:
650
+ logger.warning(f"Model file missing {model_key}: {e}")
651
+ results.append({
652
+ "model_key": model_key,
653
+ "display_name": manifest["models"].get(model_key, {}).get("display_name", model_key),
654
+ "error": "Model file missing. Check server configuration.",
655
+ })
656
+ except Exception as e:
657
+ logger.exception(f"Unexpected error in {model_key}")
658
+ results.append({
659
+ "model_key": model_key,
660
+ "display_name": manifest["models"].get(model_key, {}).get("display_name", model_key),
661
+ "error": f"Inference failed: {type(e).__name__}: {e}",
662
+ })
663
+
664
+ return {"image_url": image_url, "results": results}
665
+
666
+ # ============================================================================
667
+ # APP STATUS
668
+ # ============================================================================
669
+
670
+ def app_status():
671
+ """Get application status"""
672
+ labels = class_names()
673
+ manifest = model_manifest()
674
+ dataset_images = dataset_images_manifest()
675
+ models = []
676
+
677
+ for key, info in manifest["models"].items():
678
+ path = model_path_for(info)
679
+ models.append({
680
+ "key": key,
681
+ "display_name": info.get("display_name", key),
682
+ "file": info.get("file"),
683
+ "exists": path.exists(),
684
+ "last_conv_layer": info.get("last_conv_layer"),
685
+ "total_parameters": info.get("total_parameters"),
686
+ })
687
+
688
+ tensorflow_available = True
689
+ tensorflow_message = "Ready"
690
+ try:
691
+ tensorflow_modules()
692
+ except Exception as exc:
693
+ tensorflow_available = False
694
+ tensorflow_message = str(exc)
695
+ logger.warning(f"TensorFlow unavailable: {exc}")
696
+
697
+ comparison_path = TABLE_DIR / "final_comparison_table.csv"
698
+ metrics_path = TABLE_DIR / "model_metrics.csv"
699
+ report_path = ARTIFACT_DIR / "ecommerce_nn_explainability_report.pdf"
700
+
701
+ ready = bool(labels) and any(model["exists"] for model in models) and tensorflow_available
702
+
703
+ return {
704
+ "ready": ready,
705
+ "artifact_dir": str(ARTIFACT_DIR),
706
+ "model_dir": str(MODEL_DIR),
707
+ "metadata_dir": str(METADATA_DIR),
708
+ "class_count": len(labels),
709
+ "models": models,
710
+ "tensorflow_available": tensorflow_available,
711
+ "tensorflow_message": tensorflow_message,
712
+ "tables": {
713
+ "final_comparison": read_csv_table(comparison_path),
714
+ "metrics": read_csv_table(metrics_path),
715
+ },
716
+ "dataset_images": {
717
+ "count": len(dataset_images),
718
+ "classes": sorted({row["label"] for row in dataset_images}),
719
+ "dir": str(DATASET_IMAGE_DIR),
720
+ "local_dir": str(LOCAL_DATASET_DIR),
721
+ "manifest": str(DATASET_IMAGE_MANIFEST_PATH),
722
+ },
723
+ "report_url": url_for("output_file", filename="ecommerce_nn_explainability_report.pdf")
724
+ if report_path.exists() else None,
725
+ }
726
+
727
+ # ============================================================================
728
+ # ROUTES
729
+ # ============================================================================
730
+
731
+ @app.route("/")
732
+ def index():
733
+ """Serve main HTML"""
734
+ return render_template("index.html")
735
+
736
+ @app.route("/api/status")
737
+ def status():
738
+ """Get app status"""
739
+ return jsonify(app_status())
740
+
741
+ @app.route("/api/dataset-images")
742
+ def dataset_images():
743
+ """Get paginated dataset images"""
744
+ try:
745
+ limit = max(1, min(int(request.args.get("limit", 24)), 96))
746
+ offset = max(0, int(request.args.get("offset", 0)))
747
+ except ValueError:
748
+ limit, offset = 24, 0
749
+
750
+ label_filter = (request.args.get("label") or "").strip()
751
+ all_images = dataset_images_manifest()
752
+ labels = sorted({row["label"] for row in all_images})
753
+ filtered = [row for row in all_images if not label_filter or row["label"] == label_filter]
754
+ page_rows = filtered[offset : offset + limit]
755
+
756
+ items = []
757
+ for row in page_rows:
758
+ item = dict(row)
759
+ item["url"] = dataset_image_url(row)
760
+ items.append(item)
761
+
762
+ next_offset = offset + len(items) if offset + len(items) < len(filtered) else None
763
+
764
+ return jsonify({
765
+ "items": items,
766
+ "total": len(filtered),
767
+ "offset": offset,
768
+ "limit": limit,
769
+ "next_offset": next_offset,
770
+ "classes": labels,
771
+ })
772
+
773
+ @app.route("/dataset-images/<image_id>")
774
+ def dataset_image_file(image_id):
775
+ """Serve dataset image"""
776
+ image_info = dataset_image_by_id(image_id)
777
+ if image_info is None:
778
+ return jsonify({"error": "Image not found"}), 404
779
+
780
+ try:
781
+ image_path = dataset_image_file_path(image_info)
782
+ except Exception as exc:
783
+ logger.error(f"Dataset image error: {exc}")
784
+ return jsonify({"error": str(exc)}), 400
785
+
786
+ return send_from_directory(image_path.parent, image_path.name)
787
+
788
+ @app.route("/api/predict", methods=["POST"])
789
+ def predict():
790
+ """Predict on uploaded image"""
791
+ labels = class_names()
792
+ if not labels:
793
+ logger.error("class_names.json not found")
794
+ return jsonify({"error": "Classifier not configured. Run Kaggle notebook first."}), 400
795
+
796
+ image_file = request.files.get("image")
797
+ if not image_file or not image_file.filename:
798
+ return jsonify({"error": "No image uploaded."}), 400
799
+
800
+ if not validate_file_extension(image_file.filename):
801
+ return jsonify({"error": "Invalid image format. Use JPG, PNG, WEBP, or BMP."}), 400
802
+
803
+ try:
804
+ manifest = model_manifest()
805
+ selected_models = request.form.getlist("models") or list(manifest["models"].keys())
806
+
807
+ logger.info(f"Prediction request: {image_file.filename} with models {selected_models}")
808
+ image_array, image_url = prepare_image(image_file, manifest["input_size"])
809
+ return jsonify(predict_with_models(image_array, image_url, selected_models))
810
+ except ValueError as e:
811
+ logger.warning(f"Invalid image: {e}")
812
+ return jsonify({"error": str(e)}), 400
813
+ except Exception as e:
814
+ logger.exception("Prediction error")
815
+ return jsonify({"error": "Prediction failed. Check server logs."}), 500
816
+
817
+ @app.route("/api/predict-dataset", methods=["POST"])
818
+ def predict_dataset():
819
+ """Predict on dataset image"""
820
+ labels = class_names()
821
+ if not labels:
822
+ return jsonify({"error": "Classifier not configured."}), 400
823
+
824
+ payload = request.get_json(silent=True) or {}
825
+ selected_models = payload.get("models") or []
826
+ if isinstance(selected_models, str):
827
+ selected_models = [selected_models]
828
+
829
+ image_info = dataset_image_by_id(payload.get("image_id"))
830
+ if image_info is None:
831
+ return jsonify({"error": "Dataset image not found."}), 404
832
+
833
+ try:
834
+ manifest = model_manifest()
835
+ image_url = dataset_image_url(image_info)
836
+ image_path = dataset_image_file_path(image_info)
837
+
838
+ logger.info(f"Dataset prediction: {image_info.get('filename')} with models {selected_models}")
839
+ image_array, image_url = prepare_artifact_image(image_path, manifest["input_size"], image_url)
840
+ return jsonify(predict_with_models(image_array, image_url, selected_models))
841
+ except Exception as e:
842
+ logger.exception("Dataset prediction error")
843
+ return jsonify({"error": "Dataset prediction failed."}), 500
844
+
845
+ @app.route("/figures/<path:filename>")
846
+ def figure_file(filename):
847
+ """Serve precomputed reference figures (SHAP, confusion matrices, etc.)"""
848
+ safe = os.path.basename(filename)
849
+ target = (FIGURE_DIR / safe).resolve()
850
+ if not str(target).startswith(str(FIGURE_DIR.resolve())) or not target.exists():
851
+ return jsonify({"error": "Figure not found"}), 404
852
+ return send_from_directory(FIGURE_DIR, safe)
853
+
854
+ @app.route("/outputs/<path:filename>")
855
+ def output_file(filename):
856
+ """Serve output files"""
857
+ try:
858
+ # Validate path to prevent traversal
859
+ if not normalize_artifact_path(filename):
860
+ return jsonify({"error": "Invalid file path"}), 400
861
+ return send_from_directory(ARTIFACT_DIR, filename)
862
+ except Exception as e:
863
+ logger.warning(f"Output file error: {e}")
864
+ return jsonify({"error": "File not found"}), 404
865
+
866
+ @app.before_request
867
+ def periodic_cleanup():
868
+ """Clean old generated files (every 10 requests)"""
869
+ if not hasattr(app, "request_count"):
870
+ app.request_count = 0
871
+
872
+ app.request_count += 1
873
+ if app.request_count % 10 == 0:
874
+ removed = cleanup_old_generated_files()
875
+ if removed > 0:
876
+ logger.info(f"Cleaned {removed} old files")
877
+
878
+ @app.errorhandler(413)
879
+ def request_entity_too_large(error):
880
+ """Handle file too large"""
881
+ return jsonify({"error": "File too large. Max 30 MB."}), 413
882
+
883
+ @app.errorhandler(500)
884
+ def internal_error(error):
885
+ """Handle server errors"""
886
+ logger.exception("Internal server error")
887
+ return jsonify({"error": "Server error. Check logs."}), 500
888
+
889
+ # ============================================================================
890
+ # STARTUP
891
+ # ============================================================================
892
+
893
+ if __name__ == "__main__":
894
+ logger.info("=== Starting Ecommerce Product Classifier ===")
895
+ logger.info(f"Artifact dir: {ARTIFACT_DIR}")
896
+ logger.info(f"Model dir: {MODEL_DIR}")
897
+
898
+ # Check TensorFlow at startup
899
+ try:
900
+ tensorflow_modules()
901
+ except RuntimeError as e:
902
+ logger.warning(f"TensorFlow not available at startup: {e}")
903
+
904
+ debug = os.getenv("FLASK_DEBUG", "0") == "1"
905
+ port = int(os.getenv("PORT", "5000"))
906
+ host = os.getenv("HOST", "0.0.0.0")
907
+
908
+ logger.info(f"Starting server on {host}:{port} (debug={debug})")
909
+ app.run(host=host, port=port, debug=debug, use_reloader=False)
requirements.txt ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ flask==3.0.3
2
+ werkzeug==3.0.1
3
+ numpy==1.26.4
4
+ pillow==10.4.0
5
+ tensorflow==2.19.0
6
+ keras==3.13.2
7
+ matplotlib==3.9.2