Deep Learning & Generative AI

Generative AI & Large Language Models (LLMs) Interview Questions

Interview questions on Generative AI & Large Language Models (LLMs) Interview Questions.

23 questions

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Tokenization & Embeddings

Generative AI & Large Language Models (LLMs) Interview Questions

Q1. What does tokenization entail, and why is it critical for LLMs?

Q2. What are the different parameters of a tokenizer?

Q3. What are the key considerations in designing an effective tokenization strategy for language models?

Q4. Can you explain the Bag-of-Words (BoW) model and discuss its limitations?

Q5. What are word2vec models and how are they trained?

Q6. What are the main limitations of the word2vec model?

Q7. What are different types of embeddings?

Q8. Which tokenizer is used in the BERT model, and what are the special tokens used in the BERT Base Uncased model?

Q9. What are the embeddings, and how are they initialized in LLMs?

Q10. How do LLMs manage out-of-vocabulary (OOV) words?

Q11. What is Byte‑Pair Encoding (BPE) and how does it work?

Q12. Illustrate with a simple example, how does BPE work?

Q13. Which tokenizer is used in the GPT-3/4 model, and what are the special tokens used in the GPT-3/4 model?

Q14. What are positional encodings, and why are they used?

Q15. What are different types of positional embeddings?

Q16. How does fine-tuning embedding models improve dense retrieval?

Q17. How does Adaptive Softmax optimize LLMs?

Q18. What is contrastive learning?

Q19. What makes TSDAE a suitable method for learning sentence embeddings in a completely unsupervised setting, and how does it differ from masked language modeling?

Q20. What is cosine-similarity loss, and how is it used to train embedding models?

Q21. Suppose you have a limited labeled dataset and want to train a sentence embedding model. How would you approach this effectively without compromising performance?

Q22. What is a few-shot classification, and why is it useful?

Q23. What are the three main steps of the SetFit framework for few-shot text classification?