Description

This project aims to develop an intelligent spam detection system using deep learning techniques. The objective is to accurately classify emails as either "spam" or "not spam" by building and deploying a robust model. The project involves several stages, starting with dataset acquisition, followed by data preprocessing, model training, and evaluation. The best-performing model and preprocessing pipeline are then saved and integrated into an interactive web application using Django and Streamlit.

Problem

Email and SMS platforms are plagued with spam, threatening user security and productivity. Traditional rule-based or statistical filters often fail to adapt to new spam tactics, especially when faced with obfuscated language or minimal messages.

Objective

• Develop a deep learning-based binary text classifier
• Accurately distinguish between spam and ham messages

• Capable of handling multiple datasets for training and evaluation

• Integrated custom preprocessing pipeline tailored for spam detection
• Included steps like lemmatization, stopword removal, and tokenization

• Generated spam-specific features (e.g., keyword presence, punctuation patterns)

• Enabled real-time message classification through:
• Streamlit – interactive demo UI
• FastAPI – API-first architecture
• Django – full-stack integration for web apps

Solution

• Developed an end-to-end pipeline for spam message classification

• Combined 4 datasets (SMS Spam Collection, NUS SMS Corpus, SpamAssassin, others)
• Total size: 19,314 messages

• Custom preprocessing pipeline included:
• Stopword removal
• Stemming
• Punctuation stripping
• Spam-word scoring (based on keyword frequency)

• spam_word_score – Custom metric for spam keyword density
• len_message – Length of the message

• Used CNN and LSTM architectures with:
• Keras Tokenizer and padded sequences
• Concatenated custom features (e.g., spam_word_score)
• Keras Tuner for hyperparameter tuning

• Streamlit frontend for interactive spam classification
• Django backend app deployed on DigitalOcean

Technologies Used

• NLTK – for text processing
• Regular expressions – for pattern-based text cleaning
• Stemming – to reduce words to their root form
• Stopword filtering – to remove non-informative words

• TensorFlow – backend engine for model training
• Keras – used to build CNN and LSTM models

• Scikit-learn – for evaluation metrics and preprocessing
• Keras Tuner – for hyperparameter optimization
• Matplotlib & Seaborn – for data visualization

• Streamlit – interactive frontend
• Django – backend framework
• DigitalOcean – cloud hosting and deployment

• Combined SMS and Email datasets from open-source repositories

Challenges Faced

• Ham messages significantly outnumbered spam messages
• Addressed using oversampling, stratified train-test split, and metrics beyond accuracy

• CNN and LSTM models overfit quickly due to limited data
• Used Dropout, L2 regularization, and early stopping to prevent overfitting

• Tokenizer had to be saved and reused consistently across training and inference
• Ensured no mismatch between training and deployment inputs

• Integrated engineered features (e.g., spam score, message length)
• Required careful reshaping and concatenation with embedded sequences

• Streamlit was lightweight and easy to set up
• Django + model + tokenizer + webcam consumed significant memory during deployment

Methodology

• Embedding layer – 64 dimensions
• 1D Convolution – 128 filters
• Global MaxPooling
• Dropout – 0.5
• Dense layers
• Output: Sigmoid activation

• Classification Report – precision, recall, F1-score
• Confusion Matrix
• Learning Curves – Train vs Validation Accuracy & Loss
• Tokenizer, padded sequences, and engineered features

• Streamlit App: User enters message → outputs label, confidence, and matched spam keywords
• Django App: Production-ready backend hosted at:
  🔗 sunilmanirudhan.linustech.in/spam/spam

• Custom spam_word_score – Enhanced interpretability of model predictions
• Hybrid NLP + Deep Learning – Combined classical techniques with modern architectures
• Dual Deployment – Accessible via both Streamlit and Django platforms
• Keras Tuner – Automated architecture and hyperparameter tuning

Result

<div class="mb-16 fw-bold">🏆 Best Performing Model</div> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px;"> <li><strong>Model:</strong> Convolutional Neural Network (CNN)</li> <li><strong>Test Accuracy:</strong> 97.83%</li> </ul> <div class="mb-16 fw-bold">📈 Evaluation Metrics (Spam Class)</div> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px;"> <li><strong>Precision:</strong> 94%</li> <li><strong>Recall:</strong> 98%</li> <li><strong>F1-Score:</strong> 96%</li> </ul> <div class="mb-16 fw-bold">🧮 Confusion Matrix</div> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px;"> <li><strong>True Positives:</strong> 560</li> <li><strong>False Positives:</strong> 33</li> <li><strong>False Negatives:</strong> 9</li> <li><strong>True Negatives:</strong> 1330</li> </ul>

Description

We decided to analyze the players' data from the game FIFA 19. There is a lot of several players from different countries, playing in different competitions. Their abilities in the game should reflect their real-world skills. The game's creators are about its efforts by creating game attributes, such as sprint speed, force bullets, endings or headers that we can express numerically. In the project, we will try to classify players into a game room based on these attributes, and positions and predict their market value in the game. Since the game does not always show players the market value of the player, but only its attributes, our model can be useful in determining the amount offered for the transfer of the player when negotiating in the game. When in the real world the abilities of the players (e.g., ending) do not bear any numerical. We will try to use the fact that in the game of expression, we can find out which attributes are important for certain gaming positions.

Problem

Football clubs face the challenge of selecting the right players from a global pool with varying skill levels, prices, and growth potential. Relying solely on scouts and intuition can lead to overpriced or underperforming signings. Clubs like Manchester United need a data-driven approach to:

• Predict accurate player market values
• Identify high-potential, affordable players
• Support recruitment with concrete metrics

Objective

• Analyze the FIFA 19 dataset to identify patterns in player performance and valuation.
• Build a machine learning model to predict market value of players.
• Use the model to assist Manchester United in selecting 5 new signings based on specific criteria: young age, high potential, and affordability.

Solution

• Cleaned and transformed the FIFA dataset (89 columns, 18k+ players)
• Engineered new features like Potential Gap
• Transformed monetary columns: Value, Wage, Release Clause
Modeling Approaches
• Linear Regression (with and without PCA)
• Random Forest
• Gradient Boosting
• XGBoost (multiple variants)
• Support Vector Machines
Player Filtering Criteria
• Age less than 27
• Overall rating greater than 80
• Positive Potential Gap
• Value and Release Clause under €80M
Final Output
• Matched top players to Manchester United’s position needs: RW, LB, RM, CM, CB

Technologies Used

Machine Learning Models
• Linear Regression
• Gradient Boosting
• XGBoost
• Random Forest
• Support Vector Machines
Techniques Applied
• Principal Component Analysis (PCA)
• Box-Cox Transformation
• Cross-validation
• Feature Selection
Dataset
• FIFA 19 player dataset
• 18,207 rows
• 89 features

Challenges Faced

• Complex formatting in monetary features (symbols like €, M, K)
• Data sparsity in categorical fields like position and club
• Need for dummification and scaling before applying models
• High-dimensional correlation and multicollinearity management
• XGBoost and SVM were computationally intensive on the full dataset

Methodology

Data Cleaning
• Removed redundant/low-variance columns
• Eliminated irrelevant text and image-related data
Feature Engineering
• Converted "Value" and "Wage" from string to numeric format
• Created Potential Gap = Potential - Overall
Dimensionality Reduction
• Performed PCA to reduce features from 89 to 47
• Applied correlation filtering to remove highly collinear variables
Modeling
• Used caret package for model training with 10-fold cross-validation
• Compared models using RMSE as the performance metric
Final Deployment
• Shortlisted 5 ideal signings based on predicted value, availability, and performance fit

Result

<div class="mb-16 fw-bold">Best Performing Model</div> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px;"> <li><strong>XGBoost (Model 4)</strong> achieved the lowest RMSE: <strong>0.7217</strong></li> </ul> <div class="mb-16 fw-bold">Top 5 Suggested Player Signings</div> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px;"> <li><strong>T. Werner</strong> – RW</li> <li><strong>F. Thauvin</strong> – RM</li> <li><strong>Jorginho</strong> – CM</li> <li><strong>D. Alaba</strong> – LB</li> <li><strong>N. Süle</strong> – CB</li> </ul> <div class="mb-16 fw-bold">Transfer Budget</div> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px;"> <li><strong>Total Budget:</strong> €186M</li> </ul> <div class="mb-16 fw-bold">Business Impact</div> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px;"> <li>Enabled data-driven recruitment decisions</li> <li>Helped avoid high-fee, high-risk player signings</li> </ul>

Description

Usage

The dataset can be used for:

• Building a Recommendation System based on user input or preference.
• Classification purposes based on audio features and available genres.
• Any other application that you can think of. Feel free to discuss!

Column Description

• track_id: The Spotify ID for the track.
• artists: The names of the artists who performed the track. If there is more than one artist, they are separated by a semicolon (;).
• album_name: The album name in which the track appears.
• track_name: The name of the track.
• popularity: A value between 0 and 100 indicating the popularity of a track. It is calculated based on the number of plays and recency of plays.
• duration_ms: The length of the track in milliseconds.
• explicit: Whether the track has explicit lyrics (true = yes, false = no or unknown).
• danceability: A measure (0.0 to 1.0) of how suitable a track is for dancing, based on musical elements like tempo and beat strength.
• energy: A measure (0.0 to 1.0) of intensity and activity in a track. Higher values indicate fast, loud, and noisy tracks.
• key: The musical key of the track, mapped to standard Pitch Class notation (e.g., 0 = C, 1 = C♯/D♭, 2 = D, etc.).
• loudness: The overall loudness of a track in decibels (dB).
• mode: Indicates the modality of a track (1 = Major, 0 = Minor).
• speechiness: Detects the presence of spoken words in a track. Values above 0.66 indicate mostly speech-based tracks.
• acousticness: A confidence measure (0.0 to 1.0) of whether the track is acoustic. Higher values indicate acoustic tracks.
• instrumentalness: Predicts whether a track contains no vocals. Values closer to 1.0 indicate instrumental tracks.
• liveness: Measures the presence of an audience in the recording. Values above 0.8 suggest a live performance.
• valence: A measure (0.0 to 1.0) of the musical positiveness of a track. Higher values indicate happier and more cheerful tracks.
• tempo: The estimated tempo of a track in beats per minute (BPM).
• time_signature: An estimated time signature, ranging from 3 to 7 (e.g., 3/4 to 7/4).
• track_genre: The genre of the track.

Problem

Not specified.

Objective

Not specified.

Solution

Not specified.

Technologies Used

Not specified.

Challenges Faced

Not specified.

Methodology

Not specified.

Result

Not specified.

Description

The HR salary dataset from Kaggle is a valuable resource for analyzing employee compensation trends. It contains information on employee demographics, job-related details, and compensation figures. This data can be used to identify factors that influence salary, such as experience, education, and job title, and assess organisational pay equity.

Problem

• Predict salaries of new hires or internal transfers
• Justify salary revisions based on experience, skills, and performance
• Ensure consistent and fair compensation using analytics

Objective

• Create a machine learning model that predicts employee salaries
• Build an interactive web dashboard that HR managers can access
• Improve prediction accuracy through feature engineering and model tuning

Solution

• Used an HR dataset with 311 entries and 38 features
• Cleaned and reduced the dataset to 8 key features (e.g., age, experience, department, race, project count, performance score)

• Trained and compared multiple models:
• Linear Regression
• Random Forest
• Gradient Boosting (with and without hyperparameter tuning)

• Built and hosted a Flask-based interactive web dashboard on PythonAnywhere

Technologies Used

• Python

• Pandas
• Scikit-learn
• GridSearchCV
• Matplotlib

• Flask
• HTML (for dashboard)

• PythonAnywhere

• Google Colab
• Spyder

Challenges Faced

• Small dataset size made overfitting a concern
• Target variable (salary) was skewed, requiring transformation
• Feature selection was non-trivial due to missing or redundant fields
• Needed to extract derived features like age and experience from DOB, hire/termination dates

Methodology

• Identify salary influencers such as experience, age, and performance

• Filled missing values (e.g., termination date) using the current date
• Removed uninformative columns (e.g., Employee ID, Manager Name)

• Extracted Age from DOB and Experience from Date of Hire
• Encoded categorical variables

• Used correlation heatmaps and scatter plots
• Confirmed project count and performance score had the highest influence on salary

• Trained models using train/test split
• Applied target transformation to handle salary skew
• Tuned Random Forest and Gradient Boosting using GridSearchCV

• Created HTML forms and Flask routes
• Hosted the app on PythonAnywhere

Result

<div class="mb-16 fw-bold">Best Model</div> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px;"> <li><strong>Tuned Gradient Boosting</strong></li> </ul> <div class="mb-16 fw-bold">Performance</div> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px;"> <li><strong>R² Score:</strong> 0.76 (acceptable range for business use)</li> </ul> <div class="mb-16 fw-bold">Business Impact</div> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px;"> <li>TCS HR can now predict salaries with improved accuracy and transparency</li> </ul>

Description

.

Problem

Not specified.

Objective

Not specified.

Solution

Not specified.

Technologies Used

Not specified.

Challenges Faced

Not specified.

Methodology

Not specified.

Result

Not specified.

Description

Stock market forecasting often suffers from inconsistency due to noisy data, emotional trading, and the isolated use of indicators or sentiment. Traders lack a unified system that combines technical signals, sentiment analysis, and AI models to generate high-confidence trade decisions. Traditional methods based on single indicators (like MACD or RSI alone) often fail in volatile or uncertain markets.

Problem

Traditional stock prediction methods fail to combine all key market signals — technical patterns, news sentiment, financial fundamentals, and global macroeconomic factors. This fragmented view results in missed opportunities, false signals, and poor portfolio decisions.

Retail investors and analysts need a single, AI-powered platform that:

• Predicts price direction
• Validates trades through sentiment and indicator signals
• Considers global index trends and sector-specific insights
• Provides a visual dashboard for decisions and backtesting

Objective

To develop a comprehensive trading and portfolio decision support system that integrates:

• LSTM-based trend prediction
• FinBERT sentiment analysis
• Technical indicators (MACD, RSI, support/resistance, EMA)
• Fundamental analysis (PE, EPS, financial ratios)
• Sector-wise analysis (e.g., Tech, Pharma, Finance)
• Global index comparison (S&P 500, NASDAQ, FTSE, Nifty 50)
• Dashboard for visualization, filtering, and simulation

Solution

• Custom-built rule-based system combining MACD, RSI, support/resistance, and EMA
• Logic engine accepts trades only when multiple indicators align
• Files: MACD.ipynb, intraday rsi+macd.ipynb, resistance and support.ipynb

• Trained LSTM model on historical stock prices
• Forecasts short-term direction and momentum
• Integrated as a filter layer in trading logic

• Scraped financial headlines, tweets, and news
• Applied FinBERT to extract sentiment polarity score
• Used sentiment to confirm or reject trade direction

• Parsed financial ratios: P/E, ROE, EPS, market cap from APIs and earnings reports
• Used for stock ranking and to filter out risky trades

• Grouped stocks by sector (e.g., Tech, Pharma, Finance)
• Analyzed sector volatility, news flow, and index correlation (e.g., Bank Nifty vs. SBI)

• Correlated global indices (S&P 500, NASDAQ, FTSE) with domestic indices
• Detected global sentiment impact on Indian stock market

• Upload or select a stock for analysis
• View LSTM-based trend prediction
• Check technical indicator alignment
• See sentiment polarity scores
• Compare sector trends
• Get portfolio risk suggestions
• Built with Streamlit or Flask (depending on version)

Technologies Used

• Python

• Pandas
• NumPy
• TA-Lib

• TensorFlow / Keras – LSTM modeling

• HuggingFace Transformers – FinBERT model

• Matplotlib
• Plotly

• yfinance
• Alpha Vantage
• Screener APIs (for financial data)

• Django / FastAPI (backend)
• HTML / CSS / JavaScript (frontend)

• SQL
• CSV files

• Financial statements APIs
• Sector-wise and global index data APIs

Challenges Faced

• Aligning and synchronizing data across different timeframes (e.g., daily RSI vs. intraday LSTM)
• Fine-tuning the sentiment model with domain-specific financial vocabulary
• Combining qualitative and quantitative data (e.g., news sentiment + PE ratio + MACD) for holistic decision-making
• Preventing overfitting in LSTM due to limited historical data windows
• Designing a clean, low-latency dashboard capable of real-time model output
• Handling ambiguous or overlapping sector classifications (e.g., hybrid companies)

Methodology

• Fetched market prices, global indices, sentiment data, and financials using APIs

• Handled null values and scaled numerical data
• Computed log returns
• Encoded categorical variables

• Created composite technical features (e.g., MACD cross + RSI < 30 + proximity to support)
• Derived sentiment scores and fundamentals ranking

• LSTM for trend prediction
• FinBERT for sentiment analysis
• Custom rule engine for technical signal interpretation
• Ensemble logic to combine AI, sentiment, and technical signals

• Ran simulated trades over historical periods
• Evaluated results using Profit/Loss, Accuracy, and Drawdown metrics

• Designed an interactive dashboard interface
• Enabled filtering by stock, sector, or sentiment

Result

<ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px;"> <li>Achieved <strong>over 78% accuracy</strong> in directional prediction when sentiment and technical signals aligned</li> <li>Provided <strong>5–20% improved ROI</strong> compared to basic indicator or LSTM-only models</li> <li>Dashboard allowed users to simulate strategies, visualize trades, and rank sectors effectively</li> </ul> <div class="mb-16 fw-bold">Key Capabilities Enabled</div> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px;"> <li>Predict market direction using AI (LSTM + FinBERT)</li> <li>Confirm trades with technical indicators (MACD, RSI, EMA, support/resistance)</li> <li>Filter stocks based on fundamentals (e.g., P/E, EPS, ROE)</li> <li>Adapt strategies based on sector and global index trends</li> </ul>

Description

Waste classification and plant disease identification are critical for sustainability and agriculture. Manual sorting is time-consuming, error-prone, and inconsistent. Automating these tasks requires accurate object detection and segmentation, even in noisy, real-world environments with class imbalance and complex backgrounds.

Problem

Waste classification and plant disease identification are critical for sustainability and agriculture. Manual sorting is time-consuming, error-prone, and inconsistent. Automating these tasks requires accurate object detection and segmentation, even in noisy, real-world environments with class imbalance and complex backgrounds.

Objective

To develop and deploy a robust deep learning system using YOLOv8 for:

• Trash Object Detection: Detect and classify cardboard, glass, metal, and plastic waste
• Plant Disease Instance Segmentation: Identify diseased regions in plant leaves with pixel-level precision
• Live Prediction: Real-time object detection using webcam input
• Deployment: Integrated with Django and FastAPI for full-stack deployment

Solution

• Trained YOLOv8n on a trash dataset with 1,783 training images
• Classes: cardboard (0), glass (1), metal (2)
• Used pretrained weights and AdamW optimizer
• Evaluation Metrics:
• mAP@0.5: 98.9%
• mAP@0.5–0.95: 88.4%
• Precision: 97.3%
• Recall: 96.3%

Technologies Used

• Ultralytics YOLOv8
• PyTorch
• OpenCV

• ONNX Export for model deployment
• FastAPI + Django for live inference with webcam input

• Google Colab (Tesla T4 GPU)

• Python
• YAML
• JSON

Challenges Faced

• CPU-based training on local system led to slow and interrupted runs
• Initial plant disease segmentation model showed low generalization on unseen data
• Webcam-based live deployment faced performance drops (<40% accuracy)
• Heavy model load time; skipped deeper tuning due to time constraints

Methodology

• Verified image-label pairs for consistency
• Converted annotations to YOLO format
• Ensured uniform image size:
  • 640×640 for trash detection
  • 256×256 for plant segmentation

• Used modular Python functions for dataset loading and training configuration
• Monitored key training metrics:
  • Precision
  • Recall
  • mAP
  • Box/Classification Loss
• Trained both detection and segmentation variants of YOLOv8

• Plotted confidence score histograms
• Analyzed confusion matrices and false positive rates
• Tracked mAP evolution across epochs and different augmentation strategies

• Integrated YOLOv8 model with FastAPI for backend inference
• Connected to a Django frontend that captures webcam images and displays results
• Exported the final trained model in ONNX format for cross-platform deployment

Result

<div class="mb-16 fw-bold">Results Summary</div> <table class="table text-secondary-light" style="width: 100%; border-collapse: collapse;"> <thead> <tr> <th style="text-align: left; padding: 8px; border-bottom: 1px solid #ccc;">Task</th> <th style="text-align: left; padding: 8px; border-bottom: 1px solid #ccc;">mAP@50</th> <th style="text-align: left; padding: 8px; border-bottom: 1px solid #ccc;">mAP@50–95</th> <th style="text-align: left; padding: 8px; border-bottom: 1px solid #ccc;">Precision</th> <th style="text-align: left; padding: 8px; border-bottom: 1px solid #ccc;">Recall</th> </tr> </thead> <tbody> <tr> <td style="padding: 8px; border-bottom: 1px solid #eee;">Trash Detection</td> <td style="padding: 8px;">0.989</td> <td style="padding: 8px;">0.884</td> <td style="padding: 8px;">0.973</td> <td style="padding: 8px;">0.963</td> </tr> <tr> <td style="padding: 8px; border-bottom: 1px solid #eee;">Plant Disease Segmentation</td> <td style="padding: 8px;">0.504</td> <td style="padding: 8px;">0.176</td> <td style="padding: 8px;">0.593</td> <td style="padding: 8px;">0.368</td> </tr> </tbody> </table> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px; margin-top: 16px;"> <li><strong>Mosaic augmentation</strong> significantly improved detection accuracy</li> <li>Trash detection model achieved <strong>real-time performance</strong> (2.2 ms/image)</li> <li>Segmentation model underperformed due to limited training scope</li> </ul>

Description

Generating realistic handwritten digit images requires a model capable of learning complex data distributions from limited training samples. Traditional image generation methods often lack diversity and tend to overfit the dataset.

The key challenge is to train a GAN (Generative Adversarial Network) that can:

• Generate sharp and visually diverse handwritten digits
• Maintain training stability throughout epochs
• Avoid common pitfalls such as mode collapse and vanishing gradients

Problem

Generating realistic handwritten digit images requires a model capable of learning complex data distributions from limited training samples. Traditional image generation methods often lack diversity and tend to overfit the dataset.

The key challenge is to train a GAN (Generative Adversarial Network) that can:

• Generate sharp and visually diverse handwritten digits
• Maintain training stability throughout epochs
• Avoid common pitfalls such as mode collapse and vanishing gradients

Objective

To implement and train a Generative Adversarial Network (GAN) on the MNIST dataset that learns to generate synthetic 28×28 grayscale handwritten digit images.

• Monitor training progression using generated image samples and loss plots
• Perform hyperparameter tuning to identify optimal training configurations
• Justify all architectural and design decisions for both generator and discriminator
• Explore advanced GAN variants such as:
  • DCGAN – Deep Convolutional GAN
  • WGAN – Wasserstein GAN for better stability
  • StyleGAN – For high-quality and stylized image synthesis

Solution

• Generator: Upsamples random noise into 28×28 grayscale images using Dense and Conv2DTranspose layers for feature expansion and image shaping.
• Discriminator: Classifies real vs. fake images using Conv2D layers with Dropout for regularization and overfitting control.
• Custom GAN Class: Encapsulates the full training loop with a custom train_step() method for adversarial updates of both generator and discriminator.
• LossPlotCallback: Custom Keras callback to visualize and save generated image samples and loss trends after each epoch, aiding in training diagnostics.

Technologies Used

• TensorFlow / Keras – For building and training GAN models
• Matplotlib / PIL – For saving generated images and plotting loss curves

• MNIST – Grayscale handwritten digit dataset (28×28)

• Adam Optimizer – With learning rate and β₁ tuning for stability
• Binary Cross-Entropy – Used as the loss function for both generator and discriminator

• Jupyter Notebook / Python – For implementation and experimentation

Challenges Faced

• Generator instability observed at higher epochs, requiring monitoring and early stopping
• Mode collapse risk was mitigated using dropout in the discriminator and label smoothing (0.9 for real labels)
• Spiking generator loss indicated increasing difficulty in fooling a strong discriminator
• Small image resolution (28×28) limited the expressive power of the generator

Methodology

• Input: 100-dimensional latent vector
• Dense layer: Output shape 7×7×256
• Conv2DTranspose layers:
  • 128 filters → 7×7
  • 64 filters → 14×14
  • 1 filter → 28×28 (final output)
• Activations: LeakyReLU for intermediate layers, Tanh for output
• Output: 28×28 grayscale image

• Input: 28×28×1 image
• Conv2D layers:
  • 64 filters + Dropout
  • 128 filters + Dropout
• Output: Flatten → Dense → Sigmoid (probability real/fake)

• For each batch:

Stable Diffusion-Based Image Generation and Prompt Engineering

×



Description

🧩 Problem Statement

Generating high-quality, realistic, and stylistically accurate images using text prompts is a complex task. Models like Stable Diffusion can produce widely varied outputs depending on multiple generation parameters.

Key Factors Influencing Output

• Prompt wording: Subtle changes can drastically affect composition and detail
• Negative prompts: Used to suppress unwanted elements
• Scheduler selection: Influences the image generation process (e.g., DDIM, Euler)
• CFG Scale: Controls the strength of prompt adherence (higher = more literal)
• Inference steps: Affects image quality, detail, and noise reduction

Understanding how each parameter impacts the output is critical for controlling quality, style, and realism in AI art and computer vision applications.

Problem

🧩 Problem Statement

Generating high-quality, realistic, and stylistically accurate images using text prompts is a complex task. Models like Stable Diffusion can produce widely varied outputs depending on multiple generation parameters.

Key Factors Influencing Output

• Prompt wording: Subtle changes can drastically affect composition and detail
• Negative prompts: Used to suppress unwanted elements
• Scheduler selection: Influences the image generation process (e.g., DDIM, Euler)
• CFG Scale: Controls the strength of prompt adherence (higher = more literal)
• Inference steps: Affects image quality, detail, and noise reduction

Understanding how each parameter impacts the output is critical for controlling quality, style, and realism in AI art and computer vision applications.

Objective

To explore and evaluate how prompt design, negative prompts, diffusion schedulers, CFG scale, and inference steps impact the output quality of a Stable Diffusion model.

🔍 Goal

Identify optimal parameter combinations that strike a balance between:

• Realism – Photographic quality and coherence of the output
• Creativity – Diversity and uniqueness of generated content
• Style Control – Consistent aesthetic output aligned with artistic intent

Solution

🧪 Experimental Breakdown

This project was executed in 6 experimental parts, each focusing on evaluating a specific parameter of the Stable Diffusion model.

✅ Part A: Negative Prompt Experiment
• Main Prompt: "A cozy cottage in the forest at sunset, highly detailed"
• Negative Prompt: "daylight, bright, sunny, day time"
• Insight: Negative prompt effectively transformed the output into a night-time setting by suppressing bright/daytime features.
✅ Part B: Scheduler Comparison
• Schedulers Tested:
  • DPMSolverMultistepScheduler – Clearer, sharper, photorealistic
  • EulerAncestralDiscreteScheduler – Softer, more artistic but less sharp
✅ Part C: CFG Scale Tuning
• CFG Values Tested: 1.0, 3.0, 7.5, 12.0, 15.0, 19.0, 25.0
• Observations:
  • Low CFG → More artistic, vague
  • Mid-range CFG (7.5–12.0) → Best realism and fidelity
  • High CFG → Overfitting, unwanted artifacts
• Conclusion: Optimal CFG range: 7.5–12.0
✅ Part D: Inference Step Variation
• Steps Tested: 10, 40, 80
• Findings:
  • 10 steps → Fast but blurry with many artifacts
  • 40 steps → Best balance between quality and speed
  • 80 steps → Slight improvement, but not time-efficient
• Conclusion: 40 steps is the sweet spot
✅ Part E: Prompt Engineering + Optimization
• Prompt: “A futuristic glass building glowing at night, cinematic lighting, ultra-realistic”
• Parameter Combos Tested:
  • CFG=7.5, Steps=30 → Acceptable but low clarity
  • CFG=12.0, Steps=40 → Best result: realistic, clean, glowing effects
  • CFG=15.0, Steps=60 → Over-sharpened with lighting artifacts

Technologies Used

Model & Libraries
• Stable Diffusion v1.5 – Pretrained text-to-image generation model
• Diffusers Library (Hugging Face) – Simplified pipeline for loading models and schedulers
Schedulers Tested
• DPMSolverMultistepScheduler
• EulerAncestralDiscreteScheduler
Hardware
• CUDA GPU – Used for efficient, fast image generation
Environment
• Python – Core programming language
• Jupyter Notebook – Interactive development and visualization

Challenges Faced

• CFG Balance: Too low leads to vague, unfocused results; too high produces overfitted, rigid textures
• Inference Time: Significantly longer at higher steps (e.g., 80+), affecting real-time usability
• Scheduler Tradeoff: DPMSolver offers realism and sharpness; EulerA provides softness with artistic style — choice depends on use case
• Manual Prompt Tuning: Required for each new prompt to find optimal configuration (no one-size-fits-all)
• GPU Memory Management: Generating multiple images concurrently can exhaust GPU resources; requires careful batching or scaling

Methodology

🧪 Pipeline Setup
• Used StableDiffusionPipeline from Hugging Face diffusers
• Deployed on GPU for fast inference
• Schedulers were set dynamically based on experiment
Prompt Control
• Tested base prompts, negative prompts, and parameter combinations
Parameter Sweeps
• Generated and saved images by varying one key parameter per experiment
Analysis Criteria
• Visual sharpness
• Prompt accuracy
• Artifact presence
• Realism vs artistic expression
Experiment Results

Experiment	Best Configuration	Insight/Output
Negative Prompting	Added: “daylight, sunny…”	Shifted sunset to night effectively
Scheduler Comparison	DPMSolver	Clearer and more realistic than EulerA
CFG Scale	7.5 – 12.0	Best balance between accuracy and style
Inference Steps	40	Fast + high-quality rendering
Prompt Optimization	CFG=12.0, Steps=40	Best result for futuristic architecture scene

Conclusion
• Prompt design and tuning of CFG scale & inference steps are critical to realistic generation
• Best Configuration:
• Scheduler: DPMSolverMultistepScheduler
• CFG Scale: 12.0
• Inference Steps: 40
• Negative prompts and scheduler selection

  Result

  <div class="mb-16 fw-bold">📷 Visual Output</div> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px;"> <li><strong>Epoch 1:</strong> Random noise</li> <li><strong>Epoch 10:</strong> Shapes of digits begin to appear</li> <li><strong>Epoch 30:</strong> Recognizable, sharp digits generated</li> </ul> <div class="mb-16 fw-bold">📉 Loss Behavior</div> <table class="text-secondary-light" border="1" cellpadding="8" cellspacing="0" style="border-collapse: collapse;"> <thead> <tr> <th>Epoch</th> <th>d_loss (↓)</th> <th>g_loss (↑)</th> </tr> </thead> <tbody> <tr> <td>1</td> <td>0.45</td> <td>0.44</td> </tr> <tr> <td>10</td> <td>0.65</td> <td>0.91</td> </tr> <tr> <td>30</td> <td>0.64</td> <td>0.95</td> </tr> </tbody> </table> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px; margin-top: 10px;"> <li>Discriminator loss remained stable</li> <li>Generator loss increased as expected (indicates adversarial progress)</li> </ul> <div class="mb-16 fw-bold">🔁 Hyperparameter Tuning</div> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px;"> <li>Learning rates tested: <code>0.0002</code>, <code>0.0001</code></li> <li>Adam β₁ values: <code>0.5</code>, <code>0.4</code></li> <li>Latent dimensions: <code>100</code>, <code>128</code></li> </ul> <p class="text-secondary-light"><strong>Best Configuration:</strong></p> <ul class="text-secondary-light" style="list-style-type: disc; padding-left: 20px;"> <li><code>lr = 0.0002</code></li> <li><code>β₁ = 0.4</code></li> <li><code>latent_dim = 100</code></li> <li><strong>Generator loss:</strong> 0.80</li> <li><strong>Discriminator loss:</strong> 0.63</li> </ul>