CareerGuide 15 of 36
Career guide · Guide 15 of 36
Top 50 Deep Learning Interview Questions and Answers
Top 50 deep learning interview questions with clear answers for freshers and experienced: neural networks, training, CNNs, RNNs, transformers and deployment.
On this page
These 50 questions cover what deep learning interviews ask, from freshers to experienced candidates: how neural networks work and learn, training problems and fixes, CNNs, RNNs and transformers, regularisation, PyTorch practice, and running models in production. Each answer is written in plain words and gets straight to the point. To learn the topics step by step, start with the Deep Learning course; for classic algorithms see the machine learning interview questions, and for language models the LLM interview questions.
Question list
Show all 50 questions
- What is deep learning?
- What is a neuron in a neural network?
- What are activation functions, and why are they needed?
- What is the difference between a shallow and a deep neural network?
- What is a loss function?
- What is gradient descent?
- What is backpropagation?
- What is the learning rate, and what happens if it is too high or too low?
- What is an epoch, a batch and an iteration?
- What is overfitting in deep learning, and how do you prevent it?
- What is dropout?
- What is a convolutional neural network (CNN)?
- What is a recurrent neural network (RNN)?
- What is transfer learning?
- What is the difference between deep learning and classic machine learning?
- What is the vanishing gradient problem?
- What is batch normalisation?
- What optimisers do you know, and how does Adam differ from SGD?
- What is weight initialisation, and why does it matter?
- What is the softmax function?
- What is data augmentation?
- What is pooling in a CNN?
- What are LSTM and GRU, and why were they introduced?
- What is attention, and why did transformers replace RNNs?
- What is fine-tuning, and how is it different from feature extraction?
- How do you handle class imbalance in a deep learning model?
- What is early stopping?
- What is weight decay (L2 regularisation)?
- How do you choose the batch size?
- What is the difference between model.train() and model.eval() in PyTorch?
- How would you debug a neural network that is not learning?
- What are embeddings in deep learning?
- What is the difference between a GPU and a CPU for deep learning?
- What is a learning rate schedule, and what is warm-up?
- What is an autoencoder?
- What are residual connections, and why do they help?
- How does a transformer block work?
- What is mixed-precision training?
- How do you train a model that does not fit on one GPU?
- What is knowledge distillation?
- What is quantisation, and when would you use it?
- How do you explain the predictions of a deep learning model?
- What is the difference between generative and discriminative models?
- What are diffusion models?
- How do you deploy a deep learning model efficiently?
- What would you do if your model performs well on validation but poorly in production?
- What is self-supervised learning?
- How do you decide between building a deep learning model and using a pre-trained API?
- What is a generative adversarial network (GAN)?
- Tell me about a deep learning project you worked on.
Beginner deep learning interview questions (freshers)
1. What is deep learning?
Deep learning is a type of machine learning that uses neural networks with many layers to learn patterns directly from data. Each layer transforms its input into a slightly more useful representation, so early layers in an image model might detect edges, and later ones detect shapes and whole objects. Because it learns features by itself, deep learning works especially well on unstructured data such as images, audio and text, where hand-designing features is hard. It usually needs more data and computing power than classic machine learning, and it is the foundation of modern AI, including large language models.
Learn more: Neural Networks Explained Simply, with a Network You Can Run
2. What is a neuron in a neural network?
A neuron is the basic unit of a neural network. It takes several numbers as input, multiplies each by a weight, adds them together with a bias, and passes the result through an activation function. The weights and bias are learned during training; the activation function adds non-linearity. A single neuron with a sigmoid activation is essentially logistic regression. The power of neural networks comes from connecting many neurons in layers, so that their simple calculations combine into complex functions.
Learn more: Neural Networks Explained Simply, with a Network You Can Run
3. What are activation functions, and why are they needed?
An activation function is applied to each neuron’s weighted sum to add non-linearity. Without it, stacking any number of layers would still produce a straight-line (linear) model, because a combination of linear functions is linear. Common choices:
- ReLU: zero for negative inputs, the input itself otherwise. The default in hidden layers, because it is simple and trains well.
- Sigmoid: squeezes values between 0 and 1; used for yes/no outputs.
- Tanh: squeezes values between -1 and 1.
- Softmax: turns a vector into probabilities that add up to 1; used for multi-class outputs.
- GELU: a smooth variant of ReLU, common in transformers.
Learn more: Neural Networks Explained Simply, with a Network You Can Run
4. What is the difference between a shallow and a deep neural network?
A shallow network has one hidden layer between input and output; a deep network has many. In theory, a single wide hidden layer can approximate many functions, but in practice depth is far more efficient: each layer can build on the features found by the layer before, so complex patterns are represented with fewer neurons overall. Depth also brings challenges, such as vanishing gradients and longer training, which techniques like ReLU activations, normalisation and residual connections were developed to solve.
Learn more: Neural Networks Explained Simply, with a Network You Can Run
5. What is a loss function?
A loss function measures how wrong a model’s predictions are, as a single number that training tries to make as small as possible. The choice depends on the task: binary cross-entropy for yes/no classification, categorical cross-entropy for choosing among several classes, mean squared error or mean absolute error for predicting numbers. Cross-entropy punishes confident wrong answers heavily, which pushes the model towards well-calibrated probabilities. The loss must be differentiable so that gradients can be calculated for training.
6. What is gradient descent?
Gradient descent is the method used to train neural networks. The gradient tells you, for every weight, how the loss would change if that weight increased slightly. Gradient descent moves each weight a small step in the opposite direction, which lowers the loss, and repeats this many times. The step size is the learning rate. In practice, gradients are computed on small random batches of data (mini-batch gradient descent), which is faster and adds noise that can help the model generalise.
Learn more: How Neural Networks Learn: Loss, Gradient Descent and Backpropagation
7. What is backpropagation?
Backpropagation is the algorithm that calculates the gradient of the loss with respect to every weight in a network. It first runs a forward pass to compute predictions and the loss, then works backwards from the output layer, using the chain rule from calculus to pass the error back through each layer and work out how much each weight contributed to it. These gradients are then used by gradient descent to update the weights. Frameworks such as PyTorch do this automatically with automatic differentiation, so you rarely write it by hand, but interviewers expect you to explain it.
Learn more: How Neural Networks Learn: Loss, Gradient Descent and Backpropagation
8. What is the learning rate, and what happens if it is too high or too low?
The learning rate controls how big a step gradient descent takes when updating weights. If it is too low, training is very slow and may stall before reaching a good solution. If it is too high, updates overshoot, the loss jumps around or grows, and training can diverge or produce “nan” values. A good learning rate makes the loss fall quickly and steadily. In practice, people try a few values on a log scale, use adaptive optimisers such as Adam, and use schedules that warm up the rate at the start and lower it over time.
Learn more: How Neural Networks Learn: Loss, Gradient Descent and Backpropagation
9. What is an epoch, a batch and an iteration?
An epoch is one full pass through the whole training dataset. A batch (or mini-batch) is the small subset of examples used to compute one gradient update, for example 32 or 256 examples. An iteration is one update step, one batch processed. So with 10,000 examples and a batch size of 100, one epoch has 100 iterations. Batch size affects memory use, speed and the noise in gradient estimates: larger batches give smoother gradients and use hardware better, while smaller batches add noise that can help generalisation.
10. What is overfitting in deep learning, and how do you prevent it?
Overfitting happens when a network learns its training data too well, including noise, so training loss keeps falling while validation loss starts rising. Deep networks are prone to it because they have many parameters. Prevention: get more data or use data augmentation (for example flipped or cropped images), use dropout, add weight decay (L2 regularisation), use early stopping based on validation loss, choose a smaller model, and start from pre-trained models with transfer learning. Always monitor training and validation curves together to spot it early.
11. What is dropout?
Dropout is a regularisation technique that, during training, randomly sets a fraction of neurons’ outputs to zero at each step, for example 20 to 50%. This stops the network from relying too heavily on any single neuron and forces it to learn more reliable, spread-out representations, which reduces overfitting. At inference time dropout is turned off and all neurons are used, with outputs scaled appropriately. In PyTorch this is why you must switch between model.train() and model.eval(): dropout behaves differently in each mode.
12. What is a convolutional neural network (CNN)?
A CNN is a neural network designed for grid-like data such as images. Instead of connecting every pixel to every neuron, it uses small filters that slide across the image, computing a weighted sum at each position. Each filter learns to detect a local pattern, such as an edge or a texture, wherever it appears. Stacking convolutional layers, usually with pooling layers that shrink the image, lets the network build up from edges to shapes to whole objects. Because filters are shared across the image, CNNs need far fewer parameters than fully connected networks.
Learn more: CNNs, RNNs and Transformers: The Main Neural Network Designs
13. What is a recurrent neural network (RNN)?
An RNN processes a sequence one step at a time, such as one word or one time point, and keeps a hidden state that carries information from earlier steps to later ones. This makes it suitable for text, speech and time series. Basic RNNs struggle to remember information over long sequences because of vanishing gradients, so variants such as LSTMs and GRUs add gates that control what to keep and what to forget. RNNs process steps one after another, which makes training slow, and transformers have largely replaced them for language tasks.
Learn more: CNNs, RNNs and Transformers: The Main Neural Network Designs
14. What is transfer learning?
Transfer learning means starting from a model that was already trained on a large general dataset, such as millions of images or a huge text corpus, and adapting it to your task. You keep the learned layers, replace or add a final layer for your classes, and then train on your smaller dataset, either only the new layers or all layers with a small learning rate (fine-tuning). It works because early layers learn general features, like edges or word meanings, that are useful for many tasks. It is the default approach for images, text and audio, saving data, time and compute.
15. What is the difference between deep learning and classic machine learning?
Classic machine learning models, such as logistic regression, decision trees and gradient boosting, usually rely on features designed by people and work very well on structured, tabular data, with modest data and computing needs. Deep learning learns its own features from raw data through many layers, which makes it much stronger on images, audio and text, but it needs more data, more compute (often GPUs) and is harder to interpret. For a typical business table of customers and transactions, gradient boosting is often as good or better and much cheaper; for unstructured data, deep learning is the standard. See AI vs machine learning vs deep learning.
Intermediate deep learning interview questions
16. What is the vanishing gradient problem?
In deep networks, backpropagation multiplies many gradients together as it moves backwards through the layers. If those numbers are small, as with sigmoid or tanh activations whose slopes are below 1, the gradient shrinks exponentially, so early layers receive almost no learning signal and stop improving. The opposite, exploding gradients, happens when values are large. Solutions include ReLU-family activations, careful weight initialisation (He or Xavier), batch or layer normalisation, residual connections that give gradients a shortcut path, LSTM-style gates in recurrent networks, and gradient clipping for exploding gradients.
Learn more: How Neural Networks Learn: Loss, Gradient Descent and Backpropagation
17. What is batch normalisation?
Batch normalisation normalises a layer’s inputs across the current mini-batch to have roughly zero mean and unit variance, then applies a learned scale and shift. It makes training faster and more stable, allows higher learning rates, and reduces sensitivity to initialisation. During training it uses batch statistics; at inference it uses running averages collected during training, which is another reason to call model.eval() before predicting. It works less well with very small batches, where layer normalisation, which normalises across features for each example, is often preferred; transformers use layer normalisation.
Learn more: How Neural Networks Learn: Loss, Gradient Descent and Backpropagation
18. What optimisers do you know, and how does Adam differ from SGD?
Stochastic gradient descent (SGD) updates every weight with the same learning rate times its gradient. SGD with momentum adds a moving average of past gradients, which smooths updates and speeds up progress in consistent directions. Adam combines momentum with a per-weight adaptive learning rate based on the recent size of each weight’s gradients, so it works well with little tuning and is a common default. AdamW is Adam with weight decay applied correctly, and is standard for training transformers. SGD with momentum and a good schedule can still generalise better in some vision tasks.
Learn more: How Neural Networks Learn: Loss, Gradient Descent and Backpropagation
19. What is weight initialisation, and why does it matter?
Weight initialisation sets the starting values of the weights before training. If all weights start equal, every neuron in a layer learns the same thing. If they are too large or too small, activations and gradients explode or vanish as they pass through layers, and training stalls. Schemes such as Xavier (Glorot) initialisation, suited to tanh and sigmoid, and He initialisation, suited to ReLU, scale random starting values based on the number of inputs to each layer, keeping signals in a healthy range. Frameworks apply sensible defaults, but it matters when building custom layers or very deep networks.
Learn more: How Neural Networks Learn: Loss, Gradient Descent and Backpropagation
20. What is the softmax function?
Softmax turns a vector of raw scores (logits) into probabilities that are all positive and add up to 1. It exponentiates each score and divides by the sum of all the exponentials, so larger scores get much larger probabilities. It is used in the output layer for multi-class classification and inside attention mechanisms. A temperature parameter can make it sharper or flatter. In practice, frameworks combine softmax with cross-entropy loss in one numerically stable function, so you usually pass raw logits to the loss rather than applying softmax yourself.
21. What is data augmentation?
Data augmentation creates new training examples by applying realistic changes to existing ones, so the model sees more variety without new labelling. For images: flips, crops, rotations, colour changes and blurring. For audio: adding noise, shifting pitch or speed. For text: paraphrasing, back-translation or replacing words with synonyms. It reduces overfitting and makes models more robust to the variations they will meet in real use. The changes must keep the label valid: flipping a photo of a cat is fine, but rotating a handwritten 6 upside down turns it into a 9.
22. What is pooling in a CNN?
Pooling layers shrink the spatial size of feature maps by summarising small regions, most often with max pooling, which keeps the largest value in each region, or average pooling. This reduces computation and the number of parameters in later layers, and makes the network less sensitive to small shifts in the position of features. Many modern architectures use convolutions with a stride greater than one instead of separate pooling layers, and use global average pooling at the end to turn each feature map into a single number before the final classifier.
Learn more: CNNs, RNNs and Transformers: The Main Neural Network Designs
23. What are LSTM and GRU, and why were they introduced?
LSTM (long short-term memory) and GRU (gated recurrent unit) are recurrent network cells designed to remember information over long sequences, which basic RNNs struggle with because of vanishing gradients. An LSTM keeps a separate cell state and uses gates, small learned switches, to decide what to forget, what new information to store and what to output. A GRU is a simpler version with fewer gates and parameters, often performing similarly. They were the standard for language and time series before transformers, and are still used for smaller sequence problems and some time-series tasks.
Learn more: CNNs, RNNs and Transformers: The Main Neural Network Designs
24. What is attention, and why did transformers replace RNNs?
Attention lets each position in a sequence directly weigh information from every other position, by comparing a query vector with key vectors and using the resulting weights to mix value vectors. Transformers are built from attention layers and process all positions in parallel, rather than one step at a time like RNNs. That makes them much faster to train on modern hardware, better at long-range relationships, and able to scale to huge datasets and models. These advantages made transformers the dominant architecture for language and, increasingly, images and audio. See attention and transformers.
25. What is fine-tuning, and how is it different from feature extraction?
Both start from a pre-trained model. In feature extraction, you freeze all the pre-trained layers and train only a new final layer on your data, using the model as a fixed feature generator; it is fast and works with very little data. In fine-tuning, you also update some or all pre-trained layers, usually with a small learning rate so you do not destroy what they learned; it usually gives better results when you have more data or your data differs from the original. For large language models, parameter-efficient methods such as LoRA fine-tune only a small set of added weights.
Learn more: Fine-Tuning LLMs and LoRA Explained Simply
26. How do you handle class imbalance in a deep learning model?
Use evaluation metrics that reflect the rare class, such as precision, recall, F1 or the precision-recall curve, not accuracy. Then options include:
- Weighting the loss so mistakes on the rare class cost more, for example class weights in cross-entropy, or focal loss.
- Oversampling the rare class, or undersampling the common one, in each batch.
- Augmenting rare-class examples.
- Choosing the decision threshold from business costs rather than defaulting to 0.5. Collecting more examples of the rare class, when possible, often helps most.
27. What is early stopping?
Early stopping monitors the validation loss (or another validation metric) during training and stops when it has not improved for a set number of epochs, called the patience, then restores the weights from the best epoch. It prevents the model from continuing to overfit after it has learned the general patterns, and saves training time. It is simple and effective, but you need a separate validation set, and with noisy validation curves a reasonable patience value avoids stopping too early.
28. What is weight decay (L2 regularisation)?
Weight decay adds a penalty that pushes weights towards zero during training, discouraging the network from relying on very large weights that fit noise. In plain SGD it is equivalent to adding the sum of squared weights to the loss (L2 regularisation). With adaptive optimisers such as Adam, the two are not equivalent, which is why AdamW applies the decay directly to the weights; it is the standard for training transformers. The decay strength is a hyperparameter tuned on validation data.
29. How do you choose the batch size?
Batch size is a trade-off. Larger batches use GPUs more efficiently and give smoother gradient estimates, but need more memory and sometimes generalise slightly worse unless the learning rate and schedule are adjusted. Smaller batches use less memory and add noise that can help generalisation, but train more slowly per example. In practice, choose the largest batch that fits in memory and still trains well, and scale the learning rate with it. Gradient accumulation lets you simulate a larger batch by summing gradients over several smaller ones before updating.
30. What is the difference between model.train() and model.eval() in PyTorch?
They switch layers that behave differently during training and inference. In train mode, dropout randomly zeroes activations and batch normalisation uses the statistics of the current batch and updates its running averages. In eval mode, dropout is turned off and batch normalisation uses the stored running averages. Forgetting to call model.eval() before validation or prediction gives noisy, worse and non-repeatable results. For inference, also wrap the code in torch.no_grad() (or torch.inference_mode()) so gradients are not tracked, which saves memory and time.
Learn more: PyTorch Basics: Tensors, Models and the Training Loop
31. How would you debug a neural network that is not learning?
Work from simple to complex. Check the data: inputs and labels match, are scaled sensibly and are not shuffled apart. Try to overfit a tiny subset, such as ten examples; if the model cannot, there is a bug in the model, loss or training loop. Check the loss function matches the output (for example raw logits into a loss that applies softmax itself). Try a few learning rates. Watch for vanishing or exploding gradients and nan values. Confirm optimizer.zero_grad() is called each step. Start from a known simple architecture before adding complexity.
Learn more: Neural Networks Explained Simply, with a Network You Can Run
32. What are embeddings in deep learning?
An embedding is a learned dense vector that represents a discrete item, such as a word, a product or a user, so that similar items get similar vectors. An embedding layer is essentially a lookup table of trainable vectors, learned along with the rest of the network. Embeddings replace huge, sparse one-hot encodings with compact vectors that capture meaning, and they are used in language models, recommendation systems and search. Pre-trained text embeddings power semantic search and retrieval. See embeddings.
33. What is the difference between a GPU and a CPU for deep learning?
Training a neural network is dominated by large matrix multiplications that can be done in parallel. A CPU has a few powerful cores suited to varied, sequential tasks; a GPU has thousands of simpler cores built for doing the same operation on many numbers at once, so it trains networks many times faster. GPUs also have high memory bandwidth, and their memory often limits model and batch size. Other accelerators, such as TPUs, are designed specifically for these workloads. Small models and inference for simple models can run fine on CPUs.
Learn more: Neural Networks Explained Simply, with a Network You Can Run
34. What is a learning rate schedule, and what is warm-up?
A learning rate schedule changes the learning rate during training instead of keeping it fixed. Common schedules lower it over time, in steps or smoothly with cosine decay, so the model makes big moves early and fine adjustments later, which usually reaches a better result. Warm-up starts with a very small learning rate and increases it over the first few hundred or thousand steps, which avoids unstable early updates when weights are random; it is standard for transformers. One-cycle schedules raise and then lower the rate. Schedules are usually as important as the choice of optimiser.
Learn more: How Neural Networks Learn: Loss, Gradient Descent and Backpropagation
35. What is an autoencoder?
An autoencoder is a network trained to reproduce its input at the output, through a narrow middle layer called the bottleneck. The encoder compresses the input into a small representation, and the decoder reconstructs the input from it. Because the bottleneck is small, the network must learn the most important features. Uses include dimensionality reduction, denoising (training to reconstruct clean inputs from noisy ones) and anomaly detection, where inputs the model reconstructs poorly are likely unusual. Variational autoencoders add a probabilistic twist that makes them generative models.
Advanced deep learning interview questions (experienced)
36. What are residual connections, and why do they help?
A residual (skip) connection adds a layer’s input directly to its output, so the layer learns only a correction to its input rather than a full transformation. This gives gradients a direct path backwards through the network, which greatly reduces vanishing gradients and makes it possible to train networks with dozens or hundreds of layers. Residual networks (ResNets) used them to train very deep image models, and every transformer block uses residual connections around its attention and feed-forward parts, combined with layer normalisation.
Learn more: CNNs, RNNs and Transformers: The Main Neural Network Designs
37. How does a transformer block work?
A transformer block has two main parts, each wrapped in a residual connection and layer normalisation. First, multi-head self-attention lets every token gather information from other tokens, with several heads learning different kinds of relationships; in decoder models a causal mask stops tokens from seeing later positions. Second, a position-wise feed-forward network processes each token on its own, adding capacity. Position information is added through positional embeddings or rotary encodings. Large models stack many such blocks. See attention and transformers.
38. What is mixed-precision training?
Mixed-precision training performs most calculations in 16-bit floating point (FP16 or bfloat16) instead of 32-bit, while keeping a 32-bit copy of the weights for updates. Modern GPUs run 16-bit maths much faster and it halves memory use, so you can train bigger models or larger batches. FP16 has a small range, so loss scaling is used to stop tiny gradients from becoming zero; bfloat16 has the same range as 32-bit and usually needs no scaling. In PyTorch, automatic mixed precision (torch.autocast) handles most of this.
39. How do you train a model that does not fit on one GPU?
Several techniques, often combined. Data parallelism copies the model to each GPU and splits each batch between them, averaging gradients; it scales speed but not model size. To fit larger models: fully sharded data parallelism shards weights, gradients and optimiser state across GPUs; tensor parallelism splits individual layers across GPUs; pipeline parallelism places different layers on different GPUs. On a single GPU, gradient checkpointing trades extra computation for memory, and mixed precision and gradient accumulation help too. Frameworks and libraries handle most of the mechanics.
40. What is knowledge distillation?
Knowledge distillation trains a smaller student model to imitate a larger teacher model. Instead of learning only from the hard labels, the student also learns from the teacher’s full output probabilities (soft targets), often softened with a temperature, which carry more information about how classes relate. The student ends up much more accurate than if trained from scratch at the same size, while being faster and cheaper to run. It is widely used to deploy models on phones and edge devices and to make production models cheaper.
Learn more: Fine-Tuning LLMs and LoRA Explained Simply
41. What is quantisation, and when would you use it?
Quantisation stores weights, and sometimes activations, with fewer bits, such as 8-bit integers instead of 32-bit floats. It cuts model size by up to four times or more and often speeds up inference, which matters for phones, edge devices and serving large models cheaply. Post-training quantisation converts a trained model directly and is quick but can lose some accuracy; quantisation-aware training simulates low precision during training to recover accuracy. Always measure accuracy on your validation set after quantising, since sensitive layers may need to stay at higher precision.
42. How do you explain the predictions of a deep learning model?
Several methods exist, each with limits. For images, saliency maps and Grad-CAM highlight which pixels or regions most influenced the prediction. For any model, SHAP values and integrated gradients estimate each input feature’s contribution. For transformers, attention weights show where information flowed, but are not a complete explanation. Testing with counterfactual inputs, such as changing one word or one region, shows what the model is sensitive to. Combine explanations with careful evaluation on slices of data, because a plausible-looking explanation does not prove the model is right.
43. What is the difference between generative and discriminative models?
A discriminative model learns to separate classes or predict an output from an input, such as classifying an image as a cat or estimating churn probability; it models the probability of the label given the input. A generative model learns the data distribution itself, so it can produce new examples that look like the training data: text, images, audio. Examples of generative deep learning models are large language models, diffusion models for images, variational autoencoders and generative adversarial networks (GANs). Generative models can also be used for classification, but are usually harder to train.
44. What are diffusion models?
Diffusion models generate images, audio and other data by learning to reverse a gradual noising process. During training, noise is added to real images in many small steps, and a network learns to predict and remove that noise at each step. To generate, the model starts from pure random noise and denoises it step by step into a realistic image, often guided by a text description. They produce high-quality, diverse outputs and are more stable to train than GANs, at the cost of slower generation, which techniques such as fewer steps and distillation reduce.
45. How do you deploy a deep learning model efficiently?
Export the trained model to an inference-friendly format, such as TorchScript, ONNX or a serving engine’s format, and choose hardware to match latency and cost needs. Reduce cost with quantisation, distillation or pruning, checking accuracy after each change. Batch requests to use GPUs fully, and cache repeated results. Serve behind a stable API with input validation, versioning, monitoring of latency, errors and input drift, and a fallback. Keep the preprocessing identical to training, since mismatches are a common cause of silent accuracy loss. See model serving.
46. What would you do if your model performs well on validation but poorly in production?
Look for differences between the validation setup and real use. Common causes:
- Different preprocessing in production, such as image resizing or text cleaning done differently.
- Data drift, where production inputs differ from training data.
- Leakage that inflated validation scores.
- A validation set that does not represent real users.
- Label definitions that differ from the business outcome. Compare input distributions, reproduce production predictions with the training pipeline, and evaluate on a fresh sample of production data. Then fix the root cause and add monitoring for it. See model monitoring and drift.
47. What is self-supervised learning?
Self-supervised learning trains models on unlabelled data by creating the labels from the data itself. For text, a model predicts the next word or hidden words; for images, it learns that two augmented views of the same picture belong together, or reconstructs masked patches. This lets models learn general representations from huge amounts of unlabelled data, which are then fine-tuned for specific tasks with far fewer labels. Pre-training of large language models is self-supervised, which is why it can use web-scale text without manual labelling.
48. How do you decide between building a deep learning model and using a pre-trained API?
Start with the requirement: quality needed, latency, cost at expected volume, data privacy and how specialised the task is. For common tasks such as transcription, translation, general image tagging or text understanding, pre-trained models through an API or open-weight models are often good enough and much faster to ship. Build or fine-tune your own when the domain is specialised, data cannot leave your environment, volume makes API costs too high, or you need control over behaviour. Either way, compare options on your own evaluation set before deciding.
49. What is a generative adversarial network (GAN)?
A GAN trains two networks against each other. The generator creates fake examples from random noise, and the discriminator tries to tell fake examples from real ones. The generator improves by learning to fool the discriminator, and the discriminator improves by catching it, until the generated examples look realistic. GANs produced the first very realistic generated faces and images, but they are hard to train: they can become unstable, or collapse to producing only a few kinds of output (mode collapse). Diffusion models have largely replaced them for image generation, though GANs are still used where speed matters.
50. Tell me about a deep learning project you worked on.
Structure the answer:
- The problem, and why deep learning was the right tool.
- The data: how much, how it was labelled, and its problems.
- The approach, usually starting from a pre-trained model, and why.
- Training choices such as augmentation, regularisation and the learning rate schedule.
- How you evaluated it, with numbers and a baseline.
- How it was deployed or used, and the impact. Mention one thing that went wrong, such as overfitting or a production mismatch, and how you fixed it. Interviewers look for sound judgement and honest evaluation more than architecture names.
How to prepare
- Explain the training loop in your own words: forward pass, loss, backpropagation, gradient descent and the learning rate. It comes up in almost every interview.
- Know the fixes for common training problems: overfitting, vanishing gradients, unstable loss, and a network that does not learn.
- Be able to compare architectures: when to use a CNN, an RNN or a transformer, and why transfer learning is the default.
- Practise one project story with numbers: data, baseline, model, evaluation and what went wrong.
- Work through the Deep Learning course and the MLOps course for the production questions.