Abstract

In the age of computers, individuals used to look for computers through every avenue or ask for such assistance to find the ideal equipment based on the necessity. It is therefore important to have all the information under one roof. Hence, this novel approach have proposed a chatbot recommendation system for computers, accessories and repair centers which provides those information at ease. This particular method uses Large Language Models to understand the user’s preferences and requirements through a user-friendly Chat - Bot interface. Furthermore, this platform utilized a state of the art ASR model to gather audio inputs. This study tends to give the best solutions towards users by collecting reviews of devices and repair centers by web-scraping certain websites to gather relevant data. In addition, image processing has been introduced in order to identify computer accessories without any difficulties. All these features will be available through the proposed chatbot system such that any individual with varying knowledge of electronic devices can get a solution without any hesitation.

Methodology

Image Classification

A dataset consisting of around 10000 images of computer hardware was created.
Using VGG16 pre-trained model features of the images were extracted.
Using OCR most common keywords were extracted from the images in the dataset.
When a new image is uploaded to the system, the features of the image are extracted and compared with the features of the images in the dataset.
The most similar image is retrieved and the keywords of the image are extracted using OCR.
Image is classified by combining the feature similarity and OCR results.

Chat bot with NER and Intent Classifier

Building a State Based Chat - Bot as the major interface where users can interact.
Intent Classifier Model to identify users' intents and identify the state based on the intent.
Name Entity Recognition Model (NER) to identify the entities within the queries and pass the specifications towards the recommendation model.

Automatic Speech Recognition

ASR technology in Sri Lankan websites enhances product searches by converting speech to text, integrating YouTube video reviews, and enabling visual product searches.
Two primary ASR development approaches were considered: CTC Architecture using the LJ Speech dataset and the Seq2Seq Approach with the "Common Accent" dataset, ultimately opting for Seq2Seq for its regional accent recognition capabilities.
The chosen Seq2Seq method employs Mel-spectrogram for acoustic analysis and 30-second audio chunks for model training, ensuring accurate transcription of audio into text utilizing an encoder and decoder with cross-attention mechanisms, significantly improves ASR system accuracy and enhances the user experience in Sri Lanka.

Laptop & Repair Center Recommendation

Employs collaborative and content-based recommendation models.
Generates TFIDF vector from user's keywords, compares it with the laptop data set using cosine similarity, and suggests similar laptops.
Integrates endorsements from users with similar typologies to enhance recommendations.
Identifies repair centers within a 10-kilometer radius based on the user's location.
Utilizes Google reviews for evaluation, enabling users to choose repair centers with higher ratings and credibility.
Provides essential details such as contact numbers, locations, and ratings for selected repair centers.

Research Milestones

Project Proposal - 12% MARCH-MAY 2023

Comprehensive explanation and documentation of research gaps , research problem and proposed solution with individual development plan to be achieved within the year to reach the research objective.

◉ Proposal Presentation : Marks - 6% | Date : 30th March 2023

◉ Proposal Report : Marks - 6% | Date : 5th May 2023
Progress Presentation I - 16% MAY 2023

Live demonstration of individual components to justify 50% completion of each component hence 50% completion of the proposed system.

◉ Status Document I : Marks - 1% | Date : 22nd May 2023

◉ Progress Presentation : Marks - 15% | Date : 26th May 2023
Research Paper - 10% AUGUST 2023

Presenting the findings in a conference

◉ Research Paper : Marks - 10% | Date : 30th August 2023
Progress Presentation II - 19% SEPTEMBER 2023

Live demonstration of individual components to justify 90% completion of each component as well as 90% completion of the intergrated proposed system.

◉ Research Poster : Marks - 0% | Date : 4th September 2023

◉ Progress Presentation : Marks - 12% | Date : 4th September 2023

◉ Code Evaluation : Marks - 6% | Date : 4th September 2023

◉ Status Document II : Marks - 1% | Date : 7th September 2023
Final Assessment - 43% OCTOBER-NOVEMBER 2023

Final demonstration of the completed product developed throughout the year.

◉ Logbook : Marks - 2% | Date : 30th October 2023

◉ Final Presentation & VIVA: Marks - 20% | Date : 30th October 2023

◉ Final Report : Marks - 19% | Date : 27th November 2023

◉ Website Assessment : Marks - 2% | Date : 7th November 2023

SWAM CHAT

Research Scope

Abstract

Research Gap

Research Problem

Objectives

Methodology

Image Classification

Chat bot with NER and Intent Classifier

Automatic Speech Recognition

Laptop & Repair Center Recommendation

Technologies

Research Milestones

documents

Project Charter

Project Proposal

Status Document - I

Status Document - II

Final Report

Presentations

Proposal presentation

progress presentation i

progress presentation ii

final presentation

about Us

Dr. Lakmini Abeywardhana

Supervisor

Shanila Thirimanne

Amanullah Unais

Wishwa Rathnaweera

Minindu Senadeera

Contact Us