NumPy Arrays, Shapes and Broadcasting Explained Visually (ML Lesson 31)

Here is a challenge. You have one million numbers, and you want to square every one of them and add them up. You can write a Python loop that visits each number one at a time. Or you can write a single short line of NumPy. Both give the same answer, but one of them is dramatically faster.

• 1,000,000 numbers
• Python loop
• One NumPy line

Welcome to lesson thirty-one of Machine Learning from Zero to Advanced. Last time, in lesson thirty, we wrapped up the first part of the course with reinforcement learning and deployment. Now we go back and strengthen the foundations. Today is about NumPy, the library underneath almost every machine learning tool in Python.

• Lesson 12: Python basics
• Lesson 14: X and w
• Today: think in arrays

By the end of this video you will read array shapes the way you read a sentence. You will know how indexing, masks, and reshaping work, and when NumPy copies your data and when it does not. The centerpiece is broadcasting, which explains how arrays of different shapes combine. We will not cover Pandas or plotting here, and we will not re-teach matrices from lesson four.

• Arrays, shape, axis
• Broadcasting rules
• Products and reductions
• Classic bugs

Why is the loop slow? A Python list holds pointers to separate objects, and every step of the loop asks the interpreter to look at an object, check its type, and decide what plus or times means for it. That bookkeeping is cheap once, but a million times it adds up. The arithmetic itself is tiny. The overhead is the cost.

• Each item is an object
• Type checked every step
• Interpreter overhead

A NumPy array stores its numbers side by side in one block of memory, all of the same type. When you write an operation on the whole array, the loop runs inside fast compiled code, with no per-item type checking. This style is called vectorization. In practice it is often tens or even hundreds of times faster than a plain Python loop, and the code is shorter too.

• Contiguous storage
• One type for all
• Loop runs in compiled code

Here is the vectorized version. Write x times x, and every element is squared at once. Wrap it in np dot sum, and you get the total. That reads exactly like the sigma notation from lesson two. A good habit starts here: whenever you feel the urge to write a for loop over data, ask whether one array expression can do the job.

• np.sum(x * x)
• No explicit loop
• Reads like the math

The core object is the ndarray, short for n-dimensional array. It is a grid of values that all share one type. The number of axes is called ndim. A list of numbers has one axis, a table has two, and a stack of tables has three. In lesson three we called these vectors, matrices, and tensors. NumPy uses the same object for all of them.

• Grid of same-typed values
• ndim: number of axes
• size: total elements

Every array has a dtype. A float64 takes eight bytes per number, so one million of them occupy about eight megabytes. A float32 takes four bytes, which is why deep learning often prefers it. Watch out for integer arrays. If you create an array of integers and assign 2.point 7 into one cell, NumPy quietly stores 2. The type is fixed when the array is created.

• float64: 8 bytes
• float32: 4 bytes
• int64 truncates decimals

The shape is a tuple that lists the length along each axis. Take this array: one, two; three, four; five, six. Its shape is three comma two, meaning three rows and two columns. In machine learning we follow a convention from lesson fourteen: rows are samples, columns are features. So a data matrix X has shape n by d, and we will keep using it.

• X.shape = (3, 2)
• 3 rows, 2 columns
• Rows are samples

Axis is where beginners stumble, so here is a rule. The axis you name is the axis that disappears. Sum our array with axis equals zero, and the three rows collapse into one: nine and twelve, a result of shape two. Sum with axis equals one, and the columns collapse: three, seven, eleven, a result of shape three. Name the axis, and that dimension vanishes.

• axis=0 collapses rows
• axis=1 collapses columns
• Result drops that axis

Take a equals arange of ten, the numbers zero through nine. The slice a, two colon five, gives two, three, four. The start is included, the stop is not. Adding a step, a colon colon two, picks every second element: zero, two, four, six, eight. For two dimensions, you give one slice per axis, separated by a comma.

• a[2:5] → 2, 3, 4
• Stop is excluded
• Step: a[::2]

On our three by two matrix X, X at zero is the first row, a sample. X, colon comma one, means all rows, column one, so it is the second feature across every sample. The lone colon means take everything along that axis. This pattern, colon comma j, is how you pull out feature j, and you will use it constantly.

• X[0] first row
• X[:, 1] second column
• X[:2, :] first two rows

A comparison on an array returns an array of true and false values, called a mask. For a equals zero through nine, a greater than six gives seven falses followed by three trues. Put that mask inside the brackets, and NumPy keeps only the positions marked true: seven, eight, nine. You can combine masks with an ampersand or a vertical bar, using parentheses around each condition.

• a > 6 gives True/False
• a[a > 6] → 7, 8, 9
• Filter without a loop

Reshape changes the shape without changing the data. Start with numbers zero to five, shape six. Reshape to two by three, and NumPy fills row by row: zero, one, two on top, three, four, five below. The total count must match, six equals two times three. If you write minus one for one dimension, NumPy works it out for you, so reshape with minus one comma three gives two rows.

• arange(6) → shape (6,)
• reshape(2, 3)
• -1 means infer

Here is a subtle point. A basic slice does not copy data. It gives you a view, a new window onto the same memory. If b equals a, two colon five, and then you set b at zero to ninety-nine, the original array a changes too, at index two. Reshape also usually returns a view. This saves memory, but it can surprise you.

• Slices are views
• Edit the view, edit the original
• Same buffer

If you want independence, call dot copy. Boolean masks and lists of indices always produce copies, while plain slices produce views. A good rule: if you plan to modify a piece of an array and you need the original intact, copy it first. If you are only reading, a view is perfect and free.

• a[2:5].copy()
• Mask indexing makes a copy
• Fancy indexing copies too

Now the centerpiece. Many ML steps need to combine arrays of different shapes. A classic one is centering: subtract each feature's mean from that feature's column. Our matrix X has shape three by two. The column means form a vector of shape two. These shapes differ, yet X minus the means works. That magic is called broadcasting.

• Subtract mean from each column
• Shapes differ: (3,2) and (2,)
• No loop needed

Rule one: write the two shapes one above the other, aligned on the right. If one has fewer dimensions, pad it on the left with ones. So three comma two sits above two, and the shorter shape becomes one comma two. Think of the vector of means as a matrix with a single row.

• Compare shapes from the right
• Pad the shorter with ones
• (2,) becomes (1, 2)

Rule two: go through the dimensions pair by pair. Each pair must either be equal, or one of them must be one. Here we compare three with one, which is fine, and two with two, which is fine. If instead we had a pair like three and two, neither equal nor one, NumPy raises an error. That error protects you.

• Each pair must match
• Or one of them is 1
• Otherwise: error

Rule three: any dimension of size one is stretched to match the other. Our single row of means is stretched into three identical rows, three, four, three, four, three, four. NumPy does this virtually, without actually copying the numbers, so it costs almost no memory. The output shape takes the larger size in each position.

• Size-1 axis is repeated
• No real copy is made
• Result shape: (3, 2)

Let us do it with numbers. X is one, two; three, four; five, six. The means, from X dot mean with axis zero, are three and four. Subtracting gives minus two, minus two; zero, zero; two, two. Check: each column now sums to zero. This centered matrix is the starting point for covariance and PCA, which you met in lesson twenty-three.

• Means: [3, 4]
• X - mean
• Result: [-2,-2],[0,0],[2,2]

Now both arrays stretch. Let a be a column with entries zero, ten, twenty, shape three by one. Let b be a row with entries one, two, three, shape one by three. Add them. The column stretches across, the row stretches down, and the result is three by three: one, two, three; eleven, twelve, thirteen; twenty-one, twenty-two, twenty-three. This is exactly how you compute all pairwise differences or distances without loops.

• a shape (3,1)
• b shape (1,3)
• a + b shape (3,3)

Suppose you want to subtract each row's mean instead. The row means have shape three. Align it with three by two, and you compare three against two, which fails. The fix is to make the vector a column, shape three by one, either with reshape or by passing keepdims equals True to the mean. Then it stretches sideways across the columns.

• (3,2) with (3,) fails
• 3 vs 2 mismatch
• Fix: reshape to (3,1)

The star operator multiplies elementwise, matching positions. It is not a matrix product. For the matrix multiplication from lesson four, use the at symbol, or np dot. Mixing these up is a common source of wrong results that still run without errors, so check which one you meant every single time.

• * multiplies elementwise
• @ is matrix product
• Do not mix them up

Back to lesson fourteen. Linear regression predicts with X times w, where X is n by d and w has shape d. Take our X and let w be zero point five and two. Row one gives one times zero point five plus two times two, which is four point five. Row two gives nine point five, and row three gives fifteen. The inner dimension d must match, and the output has shape n.

• X shape (n, d)
• w shape (d,)
• y_hat shape (n,)

Einsum is a compact notation for these operations. Each input gets index letters, and any letter missing from the output is summed away. The string i j, comma, j, arrow i is exactly X times w. The string i j arrow j sums over rows. And i, comma, j, arrow i j builds an outer product. You do not need einsum yet, but recognizing it will help later with attention.

• 'ij,j->i' is X @ w
• 'ij->j' sums over rows
• 'i,j->ij' is outer product

The same axis logic applies to mean, max, min, standard deviation, and argmax, which returns the position of the largest value. By default the collapsed axis disappears. Passing keepdims equals True keeps it as a size-one axis. That is the trick that makes the result broadcast cleanly back against the original. Standardizing features, subtracting the mean and dividing by the standard deviation, is a one-line broadcast.

• sum, mean, max, argmax
• axis picks what collapses
• keepdims keeps the axis

Bug number one. An array of shape n and an array of shape n by one hold the same numbers, but they are different objects. A flat vector has no notion of row or column. A column vector is a two-dimensional array. Many functions accept both, then quietly return different shapes. Print dot shape often, especially on labels and predictions.

• Flat vector vs column
• Different shapes, different behavior
• Check .shape often

Bug number two is the dangerous one. Say your labels y have shape three, but your predictions y hat have shape three by one. Subtract them, and broadcasting happily produces a three by three grid of differences, nine numbers instead of three. Take the mean of squares, and you get a loss that is wrong but still looks like a number. Always make shapes explicit before computing a loss.

• y: (3,) minus y_hat: (3,1)
• Result is (3,3)
• Loss is wrong, no error

Bug number three is in-place mutation. The operators plus-equals and minus-equals change the array itself, and if that array is a view, the source changes too. You might standardize a slice and corrupt your raw data without noticing. Also, in-place adding a float to an integer array raises an error, since the result cannot fit the type. When in doubt, write x equals x minus m, which makes a new array.

• x -= m edits original
• Views propagate changes
• Integer array rejects float

Here is a short checklist. Before an operation, write down the shape you expect. After it, print the shape and dtype. Use assert statements such as assert y dot shape equals y hat dot shape, so a mismatch stops your program at once instead of corrupting a loss silently. A few seconds of shape checking save hours of confused debugging.

• Print shape and dtype
• Write expected shapes first
• Use assert on shapes

Let us recap. NumPy stores same-typed numbers in one block, so whole-array operations beat Python loops. Shape tells you the size along each axis, and the axis you name in a reduction is the one that disappears. Slices are views, masks make copies. Broadcasting aligns shapes on the right, requires each pair to be equal or one, and stretches the ones.

• Vectorize, avoid loops
• Axis named = axis removed
• Three broadcasting rules

You now have the array fluency that every later lesson in this course relies on. Next time, in lesson thirty-two, we build on top of NumPy with Pandas, the tool for labelled tables of real-world data.

• Lesson 32: Pandas Essentials
• Labelled tables on top of NumPy
• Subscribe for the series