Voice assistant for visually impaired using LLM and VLM
The global population living with visual impairment is growing. There are limited accessibility technologies for improving their quality of life. In this project, building upon the recent development in AI technology, especially Large Language Models (LLMs) and Vision Language Models (VLMs), we design and implement a user-friendly and privacy-safe voice assistant for visually impaired individuals. The assistant can be deployed on off-line computing devices to provide guidance through low-latency speech-to-speech interactions.
Network security anomaly detection with machine learning solutions
In the modern hyper-connected digital world, cyber threats and vulnerabilities are not only increasing in number but also growing more complex and sophisticated. Conventional rule-based Detection Systems (IDS) are inadequate in identifying the advanced and evolving threats. We implement a two-layer machine learning-based detection model, where the first layer performs binary classification to differentiate benign from malicious traffic and the secondary multi-class classification layer identifies specific attack types, to facilitate targeted countermeasures. Among all evaluated machine learning models, LightGBM achieved superior performance with 99% accuracy, 98.1% F1-score, and minimal resource usage, outperforming traditional methods like SVM, KNN, random forest, and decision trees.
Skin cancer detection using transfer learning: from system design to mobile deployment
Skin cancer is the most common type of cancer. Recognizing the poor availability and reliability of existing solutions for skin cancer detection, we leverage the recent advancements in machine learning models and their on-device capabilities to provide an efficient and handy solution for the early detection of skin cancer. Specifically, we designed a lightweight solution using transfer learning with compact pretrained models to aid in the detection of skin cancer. Our solution can accurately detect malignant skin cancer and identify the five major types of skin cancers from images captured with a handheld dermoscope. We also deploy the solution on an Android mobile device. With our mobile application, general practitioners and remote healthcare workers can make guided referrals to a dermatologist for patients.