Projects | Vedaant Jain

Quacer-C- Quantitative Certification of Knowledge Comprehension in LLMs

Tue, 10 Sep 2024 00:00:00 +0000

QuaCer-C is a framework for quantitatively certifying the knowledge comprehension capabilities of Large Language Models (LLMs). Using the structured nature of knowledge graphs, we are able to derive specifications for reasoning over unstructured data like text providing a way to formally understanding reasoning using exact confidence intervals.

Authors: Isha Chaudhary, Vedaant Jain, Gagandeep Singh

HumorDB

Wed, 12 Jun 2024 00:00:00 +0000

HumorDB is a novel image-only dataset designed to advance visual humor understanding in AI systems. It consists of carefully curated image pairs with contrasting humor ratings, emphasizing subtle visual cues that trigger humor while mitigating potential biases. The dataset enables evaluation through binary classification, range regression, and pairwise comparison tasks.

Authors: Vedaant Jain, Felipe Feitosa, Gabriel Kreiman

Parkinson's Disease Progression

Sun, 12 May 2024 00:00:00 +0000

This project addresses limitations in Parkinson’s Disease (PD) research by creating synthetic data that aims to simulate changes in facial features associated with PD progression. Using diffusion models and inpainting techniques, we developed a pipeline to generate realistic image pairs representing the transition from healthy to PD-affected facial states. Additionally, we utilized evaluations based on training classification models on the synthetic data. We also explored Model generalization using subset of FFHQ dataset and saw improvement by 5% over previous model baseline.

Curriculum Learning for Embodied Planning with LLMs

Fri, 10 May 2024 00:00:00 +0000

This project explores the application of Curriculum Learning to improve the performance of GPT-2 models in Embodied Natural Language Processing tasks using the ALFWorld dataset. We developed curricula for both Action Modeling and Reinforcement Learning stages, demonstrating significant improvements in task success rates and action efficiency.

Authors: Bohan Liu, Vedaant Jain, Aarohi Gupta

Key aspects of this research include:

Developing difficulty scoring mechanisms for task demonstrations
Creating “Easy” and “Hard” curriculum sets to structure model training
Investigating the impact of curricula on model generalization across task types
Exploring the potential of few-shot learning with large language models
Demonstrating the effectiveness of a two-stage “easy-then-hard” curriculum in Reinforcement Learning

Our results show that carefully designed curricula can enhance model performance, improve generalization to unseen tasks, and increase learning efficiency in embodied AI environments.

LLMs Mimic Reddit

Fri, 10 May 2024 00:00:00 +0000

This project explores the potential of Large Language Models (LLMs) to accurately simulate user behavior in Reddit communities. We investigate if LLMs can effectively mimic the communication patterns of specific users when provided with their comment history as context, focusing on the r/science subreddit.

Authors: Vedaant Jain*, Yoshee Jain∗, Ishq Gupta, Aditi Shrivastava, Koustuv Saha, Eshwar Chandrasekharan

Key aspects of this research include:

Developing prompting strategies for comment prediction and masked fill-in-the-blank tasks
Evaluating LLM performance on style similarity (formality, syntax) and content similarity (semantics, emotions)
Analyzing the accuracy of LLMs in replicating user-specific communication nuances
Exploring the potential applications in automated moderation and prosocial behavior promotion

Multi-Modal Information Extraction from Academic Resumes

Wed, 10 May 2023 00:00:00 +0000

This project addresses the challenge of extracting structured information from academic resumes, which often span multiple pages and contain complex, domain-specific content. We developed a novel approach combining document layout analysis and sequence tagging to accurately segment and extract key information from various resume sections.

Key aspects of this research include:

Utilizing Document-Image-Transformer (DiT) for title detection and resume sectioning
Implementing BERT-based sequence tagging models for information extraction from specific sections (education, employment, publications)
Creating a labeled dataset of 30+ academic resumes (250+ pages) for model training and evaluation

Neural Style Transfer with Rust and PyTorch

Sat, 10 Dec 2022 00:00:00 +0000

This project implements artistic style transfer using Convolutional Neural Networks (CNNs) in Rust. It combines the content of one image with the artistic style of another, creating unique visual outputs. The system is deployed as a web application, allowing users to easily interact with the model through a REST API.

Key features of this project include:

Implementation of neural style transfer algorithms using Rust bindings for PyTorch
GPU-accelerated model training for improved performance
Development of a REST API for seamless integration between the user interface and the server hosting the model
Web-based interface for users to upload content and style images and receive stylized outputs