Deep Learning Interview Questions: Practical and Conceptual

Published · by JOBB GROUP FZE

Deep learning interview questions rarely stop at definitions. An interviewer who asks “What is dropout?” usually follows with “So why did your validation accuracy drop when you turned it off?” They are testing whether you can reason about why a network trains, fails or generalises, and whether you have debugged real models.

The questions below are annotated. For each one you get what it is really testing, an illustrative strong answer and the weak answer that loses marks. The first set suits graduates and junior candidates; the second suits experienced practitioners. Code examples use PyTorch.

Deep learning interview questions for freshers

1. “What is the vanishing gradient problem, and how do you reduce it?”

What it tests: whether you understand backpropagation as repeated multiplication through layers.

Strong answer: “During backpropagation, the gradient for each layer is a product of terms from the layers above it. If those terms are consistently small, the gradients for the lower layers, closest to the input, become tiny and those layers barely learn. Using ReLU instead of sigmoid or tanh helps, because its gradient is not squashed for positive inputs. Residual connections and normalisation also help in deep networks.”

Google’s Machine Learning Crash Course describes exactly this: lower-layer gradients can become very small, and ReLU can help prevent it.

Weak answer: “The gradients disappear.” It names the symptom but not the cause.

2. “What is a dead ReLU, and what would you do about it?”

What it tests: whether you know the downside of ReLU, not just its benefit.

Strong answer: “If a unit’s input is negative for every example, ReLU outputs zero and its gradient is zero, so the unit stops updating. A learning rate that is too high can push units into that state. Lowering the learning rate helps, and so does LeakyReLU, which keeps a small slope for negative inputs.”

Both remedies appear in Google’s guidance on training neural networks.

3. “How does dropout work, and why does it reduce overfitting?”

What it tests: mechanism plus intuition.

Strong answer: “During training, dropout randomly sets a fraction of activations to zero at each gradient step. The network can’t rely on any single unit, so it learns redundant, spread-out features. A higher dropout rate means stronger regularisation; a rate of 0 means no dropout, and as the rate approaches 1 the network learns nothing useful. Dropout is switched off at inference, which is why calling model.eval() matters.”

Weak answer: “It removes neurons from the model.” Dropout is temporary and random; it doesn’t remove anything permanently.

4. “Your training loss keeps falling but validation loss starts rising. What is happening?”

What it tests: reading a loss curve.

Strong answer: “The model is overfitting: it is memorising the training set rather than learning patterns that generalise. I’d try regularisation such as dropout or weight decay, more or augmented data, a smaller model, or stopping training at the point where validation loss was lowest.”

Google’s crash course gives the same diagnosis for diverging training and test loss, and suggests making the model simpler or increasing regularisation.

5. “What is the difference between model.eval() and torch.no_grad()?”

What it tests: hands-on PyTorch experience. Many candidates assume the two do the same thing.

Strong answer: “They are separate. model.eval() changes how certain layers behave: dropout is switched off and batch normalisation uses its running statistics instead of batch statistics. It does not stop gradient tracking. torch.no_grad() stops autograd recording operations, which saves memory and time, but does not change layer behaviour. For inference you normally want both.”

model.eval()
with torch.no_grad():
    preds = model(x_val)

model.train()   # switch back before the next training step

The PyTorch autograd notes say this directly: evaluation mode is not a way to disable gradient computation and is completely separate from no-grad mode and inference mode.

Weak answer: “eval() turns off gradients.” This is the most common mistake on this question.

6. “Why do we need non-linear activation functions?”

Strong answer: “Without them, a stack of linear layers collapses into a single linear transformation, however deep it is. Non-linear activations let the network represent curved decision boundaries and complex functions.”

A short, exact answer is better than a long one here. Interviewers use it as a warm-up.

Deep learning interview questions for experienced candidates

Experienced candidates are expected to diagnose problems from evidence and to explain design choices in modern architectures. Expect follow-ups such as “How would you check that?” after every answer.

7. “Walk me through how you would debug a model whose loss becomes NaN.”

What it tests: a systematic debugging process.

Strong answer, step by step:

  1. Check the data first. NaNs in the inputs, often from a division by zero during feature engineering, are a common cause. So are bursts of outliers, which can end up in the same batch if shuffling is poor. Google’s crash course lists both.
  2. Check the learning rate. Too high a rate can make updates explode. Lower it and see whether the problem goes away.
  3. Check numerically unstable operations: log of zero, division by a value that can be zero, softmax computed by hand instead of a stable built-in loss.
  4. Check for exploding gradients. Log gradient norms per layer. Batch normalisation and a lower learning rate both help; gradient clipping is a common safeguard.
  5. Reproduce on a tiny subset. If the model can’t overfit ten examples, the bug is in the code, not the data volume.

8. “How do you read these loss curves?”

Interviewers sometimes sketch a curve and ask what it means. This table summarises the usual patterns. The first four rows draw on Google’s guide to interpreting loss curves; the last is general practice:

What you see Likely cause First thing to try
Loss oscillates wildly Learning rate too high, or bad examples in the data Lower the learning rate; validate data against a schema
Loss suddenly jumps to NaN or a huge value NaNs in the input or a burst of outliers Inspect the batch where it happened
Training loss falls, validation loss rises Overfitting Regularise, simplify, add data, stop earlier
Loss is erratic or stuck in a repeating pattern Training data not shuffled well Shuffle examples properly
Both losses high and flat Underfitting, or a bug Check the model can overfit a tiny subset; increase capacity

9. “Why does the transformer scale attention scores by the square root of the key dimension?”

What it tests: whether you have read beyond tutorials.

Strong answer: “Scaled dot-product attention is computed like this:”

Attention(Q, K, V) = softmax(Q K^T / sqrt(d_k)) V

“The authors of the original transformer paper explain that for large d_k the dot products grow large in magnitude, pushing the softmax into regions where its gradients are extremely small. Dividing by the square root of d_k keeps the scores in a range where the softmax still learns.”

Likely follow-ups:

  • “Why multiple heads?” The paper says multi-head attention lets the model attend to information from different representation subspaces at once; a single head averages that information together.
  • “Why positional encodings?” The transformer has no recurrence or convolution, so it has no built-in sense of order. Positional encodings are added to the input embeddings so the model can use token position.

10. “What problem do residual connections solve?”

Strong answer: “Deeper networks are harder to train. The ResNet paper reframed each block to learn a residual function relative to its input, so the block outputs its input plus a learned correction. If the best thing a block can do is nothing, it only has to push the correction towards zero. The authors showed these networks were easier to optimise and could gain accuracy from much greater depth. The skip path also gives gradients a direct route back to earlier layers.”

11. “Batch normalisation behaves differently in training and inference. Why does that matter?”

Strong answer: “In training, batch norm normalises using the statistics of the current batch and updates running averages. In evaluation mode it uses those running averages. If you forget model.eval(), predictions depend on what else is in the batch, and a batch of one can give very poor results. If you train with very small batches, the batch statistics are noisy, and group or layer normalisation may work better.”

12. “You have 50,000 labelled images and a deadline. Train from scratch or fine-tune?”

What it tests: practical judgement. There is no single right answer, so the reasoning matters most.

Strong answer: “I’d start by fine-tuning a model pretrained on a large, related dataset, because it already has useful low-level features and usually needs less data and compute. I’d first train only the new head, then unfreeze more layers with a lower learning rate. I’d train from scratch only if my images are very different from the pretraining data, such as medical scans or satellite imagery, and a fine-tuned baseline clearly underperforms. Either way, I’d set aside a validation set that reflects production data before I start.”

Questions about your own projects

For experienced roles, expect at least half the interview to be about models you have built. Prepare to answer these for each project on your CV:

  • What was the baseline, and how much did the deep learning model improve on it?
  • How did you split the data, and how did you prevent leakage between training and test sets?
  • Which metric did you optimise, and why that one rather than accuracy?
  • What failed along the way, and how did you find out?
  • How was the model deployed and monitored, and what happened when the data shifted?

Write your project bullet points with the same evidence: the dataset size, the metric and the improvement over the baseline. The data analyst resume builder is a reasonable starting point for data and machine learning roles, and our resume questions hub covers how to describe technical projects in a CV.

How to answer when you don’t know

Deep learning moves quickly, and nobody knows every architecture. If you are asked about something you have not used, say so, then reason from first principles: “I haven’t used that, but if it works by X, I’d expect it to help with Y and cost Z.” Interviewers generally respect honest reasoning more than a confident guess.

Frequently asked questions

What deep learning topics are asked most in interviews?

Backpropagation and gradients, activation functions, overfitting and regularisation, dropout and batch normalisation, optimisers and learning rates, loss curves, CNNs, transformers and attention, and transfer learning. Experienced roles add debugging, deployment and monitoring.

What deep learning questions are asked for freshers?

Expect vanishing gradients, why activations must be non-linear, how dropout works, how to spot overfitting from loss curves and the difference between training and evaluation mode.

What deep learning questions are asked for experienced candidates?

Expect structured debugging (for example a NaN loss), attention and transformer design choices, residual connections, batch normalisation pitfalls, fine-tuning strategy and detailed questions about your own models in production.

Should I learn PyTorch or TensorFlow for interviews?

Use the framework named in the job advert. If none is named, either is fine; the concepts interviewers test are the same. Know your framework’s training and evaluation modes well.

Do I need to know the maths?

You should be comfortable with gradients, the chain rule, matrix multiplication and basic probability. Interviewers rarely want long derivations, but they do expect you to explain why a technique works.

Sources

Ready to build your resume?

Start with a sample or a blank page. No sign-up needed. Free to edit; US$1.00 only if you download a PDF.

Build My ATS ResumeExplore Resume Templates