Hi! I’m Puru Soni
Undergraduate Computational Physics Student at University at Buffalo
Research Assistant at Embedded Sensing and Computing (ESC) group, University at Buffalo
MY PROJECTS
AutoTA
We implemented a Django-based LLM teaching assistant chatbot using GPT-4 and a custom-built Retrieval Augmented Generation (RAG) pipeline that provides relevant course materials as context to tailor academic guidance while upholding academic integrity and problem-solving prowess.
We won the second prize at UB Hacking 2023! University at Buffalo’s official hackathon. (269 total participants)
OneStop
I developed a Multiplatform app integrating useful functionalities including Pomodoro timer, Password Manager, Reminders, Tasks, Alarms, and Timer with captcha protected and encrypted secure login system.
I implemented the backend with Python and SQL and user-friendly GUI utilizing Kivy framework.
MY EXPERIENCE
Click image to go to experience page
MindVoice Research Project, ESC Group
I’m currently collaborating on the MindVoice research project, mentored by Wenyao Xu, to improve Automatic Speech Recognition (ASR) for people with speech impairments by adopting a combination of Audio and Brainwave (EEG) data.
I took a Multimodal approach and utilized TensorFlow to implement a GRU encoder for Audio MFCCs and a Convolutional Neural Network encoder for EEG Frequency Domain Cross Covariance Matrices.
Our Paper got Accepted to IEEE/ACM Chase’ 24 and Published in the Smart Health Journal!
Anarghya Das, Puru Soni, Ming-Chun Huang, Feng Lin, Wenyao Xu,
Multimodal speech recognition using EEG and audio signals: A novel approach for enhancing ASR systems, Elsevier Smart Health, Volume 32, 2024, 100477, ISSN 2352-6483, https://authors.elsevier.com/c/1iwFi8MxYVeYfD
Technology
Python, TensorFlow, Git, GitHub, Weights & Biases
Timeline
May 2023 – Present
Role
Research Assistant
Location
ESC Group, University at Buffalo, Buffalo, New York, USA
Last Mile Delivery Drone Research Project, ADAMS Lab
Technology
Python, Kivy Library, UI
Timeline
Summer 2023 (May 2023 – August 2023)
Role
Research Assistant (and occasional drone pilot 🙂)
Location
ADAMS Lab, University at Buffalo, Buffalo, New York, USA
The Last Mile Delivery Drone research project, mentored by Souma Chowdhury, aims to enhance urban delivery efficiency using drones, integrating simulation and real-flight data to solve logistical challenges.
I led a three-person to develop initial version of the robust Python app for FPV drone flight, using the Kivy library for the GUI and OpenCV for camera feed processing.
I also led implementation of multiprocessing to handle real-time drone data via Pymavlink without lag and collaborated on experiments at UB’s SOAR facility.
Crazyflie Swarm Research Project, ADAMS Lab
As a Research Assistant at ADAMS Lab, mentored by Souma Chowdhury, I engaged with a swarm of nano-quadcopters (Crazyflies), learning ROS and adapting to Ubuntu VM and VICON Motion Capture Lab at UB to manage and experiment with autonomous (and manual) flight of multiple Craryflies simultaneously using the Crazyswarm 2 library.
Click here to go to experience page (and see video of it in-action!)
Technology
ROS, Python, Linux, VICON Motion Capture, Crazyswarm 2 Library
Timeline
Summer 23 (May 2023 – August 2023)
Role
Research Assistant
(and occasional nano-quadcopter crasher 🙁. They’re very hard to fly manually)
Location
ADAMS Lab, University at Buffalo, Buffalo, New York, USA
Human Vision Inference from Brainwaves Research Project, ESC Group
Technology
Python, PyTorch, MNE Library
Timeline
September 2022 – May 2023
Role
Research Assistant
Location
ESC Group, University at Buffalo, Buffalo, New York, USA
In my research, mentored by Wenyao Xu, I investigated using brainwaves to infer visual information through machine learning, aiming to advance brain-computer interfaces. My focus was on classifying visual data based on brainwave analysis.
I developed a VGG architecture based Convolutional Neural Network (CNN) with PyTorch, achieving 95% classification accuracy and an F1 score of 0.97.
I presented my findings at UB’s Celebration of Student Academic Excellence Showcase in 2023!
MY CERTIFICATIONS
Deep Learning Specialization
Machine Learning Course
TensorFlow Course
ABOUT ME
I focus on integrating software engineering and research to solve real-world challenges. My current endeavors in the Embedded Sensing and Computing (ESC) Group Projects involve enhancing the accuracy of Automatic Speech Recognition models for those with speech disorders, leveraging the potency of both audio and EEG data.
In the past, I’ve also led efforts in ADAMS Lab in crafting a Python-based FPV flight application for drone control utilizing Multiprocessing, incorporating Kivy and OpenCV. Concurrently, I have hands-on experience with the Crazyswarm 2 library, controlling nano-quadcopter swarms, and have employed advanced neural network architectures using PyTorch to harness EEG data’s spatial features for Vision Inference with the ESC Group.
I continually seek opportunities to learn and grow. I hold certifications in Deep Learning, Generative Adversarial Networks, and Machine Learning from DeepLearning.ai, underscoring my theoretical understanding and practical application of these advanced concepts. Additionally, I possess extensive Python software engineering skills, which I continuously refine and apply in my projects.
Puru Soni's Portfolio 