Deep Learning & Generative AI

Generative AI & Large Language Models (LLMs) Interview Questions

Interview questions on Generative AI & Large Language Models (LLMs) Interview Questions.

18 questions

All subtopics LLM Architectures & Foundations Tokenization & Embeddings Attention, KV Cache & Efficient Inference LLM Training & Pre-training Decoding Strategies & Prompt Engineering Fine-Tuning & Model Adaptation RAG & Vector Databases Alignment & Preference Optimization LLM Evaluation & Benchmarking Agents, Tool Use & Memory Multimodal Generative AI LLM Safety & Security LLMOps & Production

LLM Training & Pre-training

Q1. How are Large Language Models (LLMs) trained?

LLM Training & Pre-training

Q2. What is masked language modeling, and how does it aid pretraining?

LLM Training & Pre-training

Q3. What is the BERT model, and how was it trained?

LLM Training & Pre-training

Q4. How does RoBERTa differ from the BERT model?

LLM Training & Pre-training

Q5. How do autoregressive and masked models differ in LLM training?

LLM Training & Pre-training

Q6. What is the next sentence prediction, and how does it enhance LLMs?

LLM Training & Pre-training

Q7. What is overfitting, and how can it be mitigated in LLMs?

LLM Training & Pre-training

Q8. Why is cross-entropy loss used in language modeling?

LLM Training & Pre-training

Q9. How are gradients computed for embeddings in LLMs?

LLM Training & Pre-training

Q10. What is the Jacobian matrix in transformer backpropagation?

LLM Training & Pre-training

Q11. How do eigenvalues and eigenvectors relate to dimensionality reduction?

LLM Training & Pre-training

Q12. What is KL divergence, and how is it used in LLMs?

LLM Training & Pre-training

Q13. What is the derivative of the ReLU function, and why is it significant?

LLM Training & Pre-training

Q14. How does the chain rule apply to gradient descent in LLMs?

LLM Training & Pre-training

Q15. What is hyper hyperparameter, and why is it important?

LLM Training & Pre-training

Q16. What is “continued pre training” with Masked Language Modeling, and why is it useful?

LLM Training & Pre-training

Q17. What’s the difference between token masking and whole-word masking in MLM?

LLM Training & Pre-training

Generative AI & Large Language Models (LLMs) Interview Questions

Q1. How are Large Language Models (LLMs) trained?

Q2. What is masked language modeling, and how does it aid pretraining?

Q3. What is the BERT model, and how was it trained?

Q4. How does RoBERTa differ from the BERT model?

Q5. How do autoregressive and masked models differ in LLM training?

Q6. What is the next sentence prediction, and how does it enhance LLMs?

Q7. What is overfitting, and how can it be mitigated in LLMs?

Q8. Why is cross-entropy loss used in language modeling?

Q9. How are gradients computed for embeddings in LLMs?

Q10. What is the Jacobian matrix in transformer backpropagation?

Q11. How do eigenvalues and eigenvectors relate to dimensionality reduction?

Q12. What is KL divergence, and how is it used in LLMs?

Q13. What is the derivative of the ReLU function, and why is it significant?

Q14. How does the chain rule apply to gradient descent in LLMs?

Q15. What is hyper hyperparameter, and why is it important?

Q16. What is “continued pre training” with Masked Language Modeling, and why is it useful?

Q17. What’s the difference between token masking and whole-word masking in MLM?

Q18. What are the three main steps involved in LLM training?