Build a Medical RAG Q&A App with Next.js & Gemini!

In this video, we’ll build a powerful medical Q&A application using cutting-edge AI technologies like Google’s Gemini, Next.js, Pinecone, and Shadcn/ui. Learn how to leverage Retrieval Augmented Generation (RAG) to create an app that allows users to:
– Upload medical reports (PDFs and images): Extract key information using Gemini’s vision capabilities.
– Ask questions about the report: Get personalized insights and recommendations by querying a knowledge base powered by Pinecone.

Technologies Used:
– Google Gemini
– Next.js
– Pinecone
– Shadcn/ui
– Vercel AI SDK
– Hugging Face Inference API

Source Code:
https://github.com/KoushikJit/almost-md

Tutorial on creating the knowledge-base:
https://youtu.be/rb9jXHto_RU

Live website:
https://almost-md.vercel.app/

Gemini Developers’ competition:
https://discuss.ai.google.dev/c/ai-studio/8

Chapters:
00:00:00 Introduction
00:00:52 Final App Demo
00:03:16 Setting Up the NextJS Project
00:05:12 UI Sketch Overview
00:06:40 Designing the Landing Page Layout
00:08:03 Dark Mode Implementation with ShadcnUI
00:11:05 Mobile Layout with Shadcn Drawer
00:14:15 Building the Report Component UI (File Upload, Text, Buttons)
00:19:09 Handling Report Selection & File Validation
00:23:58 Converting Reports to Base64 with Image Compression
00:27:26 Sending API Requests for Report Data Extraction
00:30:02 Using Google Generative AI in Next.js
00:35:40 Displaying Extracted Data in the Report Component UI
00:36:57 Adding Loading Indicators
00:38:15 Human Verification & Report Data Confirmation
00:41:44 Desktop View Layout Design
00:44:47 Building the Chat Component
00:47:52 Integrating Vercel AI SDK & useChat Hook
00:50:01 Creating the Message Boxes
00:53:17 Building Form for User Queries
00:58:21 Creating the chat API for Streaming AI Responses
01:00:31 Building Search Query for Similarity Search
01:01:48 Pinecone Client Initialization
01:05:45 Convert Search Query into Embeddings with Hugging Face API
01:09:43 Performing Similarity Search on Pinecone database
01:15:24 Augmenting Gemini Prompt with RAG retrievals
01:17:13 Streaming AI Responses with Vercel AI SDK
01:19:50 Displaying Streaming Responses in Chat
01:20:26 Handling Markdown Formatting in Chat Messages

Don’t forget to like, subscribe, and hit the notification bell to stay updated on future AI tutorials!