Puru Soni's Portfolio

AutoTA

2023 / Web App

Scroll all the way down for the video demo! (or click here)

Overview

AutoTA utilizes OpenAI’s GPT-4 to offer a chatbot that assists users with their queries in a manner consistent with academic integrity. It prioritizes educational guidance over direct code solutions, leveraging course materials to inform its responses.

We won the second prize at UB Hacking 2023! University at Buffalo’s official student-run hackathon. (269 total participants)

AutoTA can be tailored to any course by simply switching the provided course materials, making it easy for use in multiple courses. We have been actively engaging with university faculty with plans to test AutoTA for introductory CSE courses in near future.

My Role

I developed the custom Retrieval Augmented Generation (RAG) pipeline, including the vector search functionality for course documents utilizing Natural Language Processing (NLP) and the cosine similarity function to enable efficient information retrieval.

Additionally, I programmed the integration of the Django backend with the OpenAI API, and used object oriented programming in Python to ensure the continuity of dialogues by maintaining the context of previous user messages for coherent follow-up interactions.

Tech Stack

• Python: For backend development and AI model integration.
• JavaScript & CSS: For creating a dynamic and user-friendly interface.
• Django: To connect the frontend with the backend API seamlessly.
• Bootstrap: For styling and responsive design.

After winning the hackathon, we have significantly improved AutoTA, and we’re currently engaging with university faculty for use of AutoTA in introductory CSE courses.

Better RAG Pipeline: I improved upon the RAG Pipeline by shifting to ChromaDB, an Open Source production grade vector database, for efficient context retrieval. Additionally, I increased the context granularity from “per lecture” to “per slide” (see figure).

Beautiful UI: I shifted AutoTA from custom HTML and Bootstrap to Gradio, which provides a beautiful and easy-to-use interface for machine learning applications.

Open Source LLM: I moved AutoTA from GPT-4, a proprietary LLM by OpenAI (ironic, isn’t it?), to Mixtral 8x7B Instruct, an Open Source LLM by Mistral.ai. This gives us more control over AuoTA and we don’t have to rely on OpenAI.

Prompt Engineering: I did extensive changes and improvements to AutoTA’s prompt engineering to bring Mistral 8x7B Instruct’s performance up to par with Academic Integrity standards. I also increased the citation of course materials in AutoTA’s response.

Dockerization: I successfully containerized and deployed AutoTA with Docker, increasing scalability and dependability.

Deployment: I set up robust Nginx reverse proxy on server to provide secure, efficient access via a dedicated subdomain, dramatically enhancing user access speed and security.

Hackathon Version (tailored to CSE 220: Systems Programming)

Updated Version (tailored to CSE 220: Systems Programming)