Title [LI] Outline for CS6303 - Large Language Models: Applications, Systems and Impacts ✅

Lahore University of Management Sciences CS6303: Topics in Large Language Models: Systems, Applications and Impacts (Fall 2024) Instructors Dr. Ihsan Ayyub Qazi (https://web.lums.edu.pk/~ihsan/) Dr. Zafar Ayyub Qazi (https://web.lums.edu.pk/~zafar/) Course Description Since their introduction in 2017, Transformers have revolutionized Deep Learning, becoming a cornerstone of modern Artificial Intelligence (AI). These powerful architectures have fueled the development of groundbreaking large language models (LLMs) such as GPT, BERT, and T5, which in turn have enabled the creation of AI systems like ChatGPT, Gemini, Claude, and Llama. Beyond natural language processing, Transformers have pushed the boundaries of AI in multimodal domains, powering innovative text-to-image and video generation tools like DALL-E, Midjourney, and Sora. This graduate-level course offers an in-depth exploration of designing LLM based systems and their applications and impacts across different verticals. The course will cover critical aspects of system design, such as optimization algorithms and scalability, and dives into the specifics of deploying LLMs for real-world applications. Practical components include building and managing LLM-based systems, from simple chatbots to complex multi-agent frameworks, highlighting the integration of tools like OpenAI Function Calling and LangChain Expression Language (LCEL). As the course progresses, it delves into the use of LLMs across various sectors, including healthcare, education, and productivity, illustrating their transformative impact and addressing ethical considerations. Advanced topics include strategies for enhancing robustness, domain adaptation, and mitigation of biases and hallucinations in LLM outputs. Through a combination of theoretical insights and hands-on assignment(s) and a project, students will gain a comprehensive understanding of how LLMs function and learn to harness their potential responsibly in diverse settings, preparing them for advanced roles in technology and AI development. Course Objectives ● Develop an in-depth understanding of approaches for building LLM based systems and applications ● To explore emerging evidence of the application of LLMs in various sectors ● Analyze and address ethical and societal implications of LLMs Course Learning Outcomes (CLOs) CLO1: CLO2: CLO3: CLO4: CLO5: CLO6: CLO7: Articulate the key components of LLM system architecture, including tokenization, model architecture, and contextual embeddings. Develop an understanding of system design considerations for optimizing training and inference phases of LLM based systems Formulate strategies for scaling LLM applications efficiently, focusing on aspects such as vector data storage, fault tolerance, and latency reduction. Gain practical skills in building simple to complex LLM applications, from initial design through deployment (including considerations of data sourcing, model selection, safety considerations, and deploying LLMs as web APIs). To master topics such as prompt engineering, retrieval augmented generation, and the utilization of embeddings and vector databases in LLM applications. Assess the application and impact of LLMs in different sectors such as healthcare, education, and productivity. Identify and mitigate common issues in LLMs such as hallucinations, social biases, and security risks. Develop strategies for robustness, domain adaptation, and legal compliance in LLM deployments. Grading Breakup and Policy Assessment Weight (%) Related CLOs Class Participation & Attendance 10% CLO1 – CLO7