Feature engineering, deep learning and modern AI

Feature engineering

Better features beat fancier algorithms on tabular data.

• Numeric: scaling (standardisation for linear models, kNN, SVM, neural nets), log transforms for skewed values, binning.
• Categorical: one-hot encoding (few categories), target/mean encoding (many categories, computed within folds to avoid leakage), ordinal encoding for ordered levels.
• Dates: day of week, month, holidays, time since an event.
• Text: TF-IDF, or embeddings from a language model.
• Aggregates: a customer's order count in the last 30 days, average spend, days since the last login.
• Missing values: impute and add a "was missing" flag.

Neural networks in one page

A neural network stacks layers of neurons: each computes a weighted sum of its inputs plus a bias, then applies a non-linear activation (ReLU, sigmoid, softmax). Training:

1. Forward pass: compute predictions.
2. Loss: measure error (cross-entropy for classification, MSE for regression).
3. Backpropagation: compute the gradient of the loss with respect to every weight (chain rule).
4. Optimiser: update the weights a small step against the gradient (SGD, Adam), with a learning rate.

Repeat over mini-batches for several epochs. Fight overfitting with dropout, weight decay, data augmentation and early stopping.

Architectures: CNNs for images (convolutions detect local patterns), RNNs/LSTMs for sequences (largely replaced by), Transformers, which use self-attention to relate every token to every other token; the basis of modern language and vision models. Transfer learning: start from a pre-trained model and fine-tune it on your smaller dataset.

LLMs, embeddings and RAG

• A large language model is a Transformer trained to predict the next token on huge text corpora, then tuned to follow instructions.
• Embeddings turn text into vectors so that similar meanings sit close together; cosine similarity measures closeness. They power semantic search and recommendations (this platform uses them to match learning material to your resume).
• Prompting: clear instructions, examples (few-shot), asking for structured output (JSON), and giving the model the context it needs.
• RAG (retrieval-augmented generation): retrieve relevant documents (via embeddings and a vector database), put them in the prompt, and ask the model to answer from them. It reduces hallucination and keeps answers current without retraining.
• Fine-tuning changes a model's behaviour or style using your examples; use it after prompting and RAG have been exhausted.
• Evaluation of LLM features: test sets of real questions, automatic checks, human review, and monitoring for hallucinations, bias, prompt injection and cost.

Getting models into production (MLOps basics)

• Package the model with its preprocessing (a pipeline), version both, and serve via an API or batch job.
• Monitor input data drift and prediction drift; retrain on a schedule or on triggers.
• Track experiments (MLflow, Weights & Biases), make training reproducible (seeds, data versions).
• Log predictions and outcomes to measure real-world performance; keep a human in the loop for high-stakes decisions.

Portfolio advice

Interviewers prefer one end-to-end project over five notebook copies: a real question, your own data collection or cleaning, a baseline, a tuned model with honest evaluation, error analysis, and a small deployed demo (Streamlit, Gradio or a FastAPI endpoint) with a README that explains decisions and limitations.

Check yourself

• Explain backpropagation in three sentences.
• What problem does RAG solve, and how does it work?
• What is data drift and how would you detect it?