Plotting and EDA: Look at Your Data Before You Model It

Imagine four datasets with nearly identical averages, spreads and correlations. A model fed only those summary numbers would treat them as the same. Yet when you draw them, one is a clean line, one a curve, one has an outlier, and one is a vertical stripe. This famous example is called Anscombe's quartet, and it is why we plot.

• Same summary numbers
• Four different pictures
• Always plot first

In lesson 31 we met NumPy arrays, and in lesson 32 we used Pandas to hold and slice tabular data. Today we turn that table into pictures. We will learn how Matplotlib works, use Seaborn for statistical charts, and build a checklist for exploratory data analysis, often shortened to EDA.

• Lesson 31: NumPy
• Lesson 32: Pandas
• Lesson 33: Plotting and EDA

We will cover the Matplotlib figure and axes model, then the main Seaborn plots. Next we build an EDA checklist, learn to pick a chart for each question, and see how charts can lie. We will use the penguins dataset that ships with Seaborn, so every number you see is real, not invented. We will only look for problems today. Fixing them is lesson 34.

• Matplotlib and Seaborn
• EDA checklist
• Real data: penguins

Matplotlib has a simple mental model. A figure is the whole canvas, like a sheet of paper. An axes is one plot drawn on that paper, with its own x axis, y axis, title and data. One figure can hold many axes. Confusing the word axes with the word axis is the most common beginner stumble.

• Figure: the canvas
• Axes: one plot
• Axis: one number line

There are two styles. The quick style calls plt dot plot and works on whatever plot is current. The object style calls plt dot subplots, which returns a figure and an axes, and you then call methods on the axes. For anything beyond a throwaway chart, use the object style, because it is explicit about which plot you are changing.

• plt.plot is implicit
• plt.subplots is explicit
• Prefer the object style

Calling subplots with one row and three columns gives three axes in a row. You index into them and fill each one. Here we draw a histogram of body mass, a histogram of flipper length, and a scatter of the two. Always label axes with a name and units, because a chart without units forces the reader to guess.

• subplots(1, 3)
• Index each axes
• Label with units

The penguins dataset records real birds measured at research stations in Antarctica. It has three species: Adelie, Chinstrap and Gentoo. The columns include island, bill length, bill depth, flipper length, body mass and sex. In Seaborn, one line loads it as a Pandas data frame, ready for the tools from lesson 32.

• Three species
• Four measurements
• Island and sex

Every EDA starts with the same two questions. How big is it, and what is in it? The shape attribute gives rows and columns; this dataset has three hundred forty-four rows and seven columns. The info method lists each column's type and count of non-missing values. Check that numbers are stored as numbers, not as text.

• df.shape: 344 by 7
• df.info(): types
• Numbers stored as numbers?

Next count missing values per column. In penguins, a couple of measurements are missing for two birds, and the sex column is missing for several more. We only note this today. Also scan the summary from describe: minimum and maximum should make physical sense. A negative body mass, or a flipper longer than a bird, signals a data entry problem.

• isna().sum()
• describe(): min and max
• Note it, fix it later

Before anything else, look at the thing you want to predict, the target. If it is a number, plot its distribution. If it is a class, count the classes. Suppose we want to predict species from measurements. Counting shows Adelie has the most birds, Gentoo is next, and Chinstrap has the fewest. That is a mild imbalance, worth knowing before choosing metrics.

• Know your target first
• Count the classes
• Mild imbalance here

A histogram splits a numeric range into bins and counts how many values fall in each. Tall bars mean many observations. It shows center, spread, skew and whether there is more than one hump. Seaborn's histplot draws it in one call. Body mass of all penguins is right skewed with a long tail of heavy birds, and that tail hints at a mixture of species.

• Bins count observations
• Shows shape and skew
• sns.histplot

The number of bins is a choice, and it changes the picture. With too few bins, you blur real structure into one blob. With too many, every bar is noise and random gaps look meaningful. Try several bin counts before trusting a shape. If a feature of the chart disappears when you change the bins, it was probably never real.

• Too few: blur
• Too many: noise
• Test several bin counts

A kernel density estimate, or KDE, replaces bars with a smooth curve. Place a small bump on every data point, then add all the bumps up. The width of each bump is called the bandwidth, and it plays the same role as bin width. A narrow bandwidth gives a wiggly curve, a wide one oversmooths. The curve is also not limited by the data, so it can spill past real minimums.

• One bump per point
• Add bumps together
• Bandwidth like bin width

Now color the histogram by species using the hue argument. The lumpy body mass shape separates into three overlapping groups, and Gentoo penguins sit clearly heavier than the other two. This is the power of a good EDA plot: one extra argument turned a confusing shape into an explanation. A feature that separates classes this well is likely useful to a model.

• hue splits by group
• Gentoo is heavier
• Separation helps models

A box plot summarizes a distribution in five numbers. The box spans the first to third quartile, so it holds the middle half of the data. The line inside is the median. Whiskers reach out to typical extremes, and points beyond are drawn individually as possible outliers. Box plots are compact, so they shine when comparing many groups side by side.

• Box: middle 50 percent
• Line: median
• Dots: possible outliers

A box plot cannot show two humps. Two very different distributions can share the same box. A violin plot fixes this by drawing a mirrored KDE around each group, so you see the full shape, with a small box plot inside. The price is more ink and a smoothing choice. Use violins when shape matters, and box plots when you just need a clean comparison.

• Box hides multiple humps
• Violin shows full shape
• Cost: smoothing choice

For two numeric variables, use a scatter plot. Each row becomes a dot, with one feature on the x axis and one on the y axis. Plot flipper length against body mass for penguins and the dots climb from lower left to upper right. Longer flippers go with heavier birds. That pattern is a relationship worth noting, and a hint that a model could use one to predict the other.

• One dot per row
• Look for trend and spread
• sns.scatterplot

Scatter plots fail when there are too many points. Dots land on top of each other, and a dense center looks the same as a sparse one. This is overplotting. Fixes include making points smaller or partly transparent, using a two-dimensional histogram, or drawing density contours. Transparency is the easiest, since overlapping dots then darken and density becomes visible.

• Dots hide each other
• Use transparency
• Or hexbin and contours

You can encode a third variable with color, marker shape or size. Color points by species in the flipper and mass scatter, and the cloud splits into three clusters, with Gentoo in the upper right. Use color for categories, and use size sparingly for numbers, because people judge area poorly. Too many encodings at once produces a chart nobody can read.

• Color for categories
• Size is hard to judge
• Do not overload

With several numeric features, plotting pairs one by one is slow. A pair plot draws a grid. Off the diagonal are scatter plots of every pair of features, and on the diagonal is each feature's own distribution. One call to Seaborn's pairplot with hue set to species gives a whole overview. You can spot clusters, trends and odd points across all features at once.

• Grid of all pairs
• Diagonal: distributions
• Great first overview

Look at bill length against bill depth for all penguins together. The trend slopes downward, as if longer bills meant shallower bills. Now color by species. Inside each species the trend slopes upward. The overall trend reversed because the groups differ. This is Simpson's paradox, and it is a strong reason to always check a relationship within subgroups before believing it.

• Overall trend slopes down
• Within groups slopes up
• Always check subgroups

A scatter plot shows a relationship, and correlation squeezes it into one number between minus one and plus one. Near plus one means two features rise together, near minus one means one rises as the other falls, and near zero means no straight line pattern. We are only reading it today. Lesson 38 derives where this number comes from.

• Range: minus 1 to plus 1
• Measures straight-line link
• Derived in lesson 38

A heatmap shows a whole correlation matrix at once, with each cell colored by its value. Compute it with the Pandas corr method on the numeric columns, then pass it to Seaborn's heatmap with annotations turned on. Use a diverging color scale centered on zero, so positive and negative stand out. In penguins, flipper length and body mass show a strong positive link.

• df.corr() then heatmap
• Diverging colors around zero
• Diagonal is always 1

A heatmap hides things. Correlation only measures straight-line links, so a perfect curve can score near zero. It is also sensitive to outliers, and as we saw, it can mix groups together. So treat the heatmap as a pointer, not a verdict. Spot a strong cell, then go look at the scatter plot behind it. Also, strongly correlated features carry overlapping information, which matters for models later.

• Only straight-line links
• Outliers can distort
• Always open the scatter

Two features moving together does not mean one causes the other. A third factor may drive both, the link may be chance, or the direction may be reversed. Ice cream sales and sunburn rise together because of summer weather. EDA finds patterns worth investigating, and machine learning can use those patterns for prediction, but a causal claim needs far more than a chart.

• Hidden third factor
• Could be chance
• Prediction is not cause

Pick the chart from the question, not from habit. Want to see how one number is distributed? Histogram or KDE. Compare a number across groups? Box or violin. See how two numbers relate? Scatter. Compare counts of categories? Bar chart. See trends over time? Line chart. Compare many correlations? Heatmap. If you cannot state the question, you are not ready to draw.

• Distribution: histogram, KDE
• Groups: box, violin
• Two numbers: scatter

Some mismatches are classic. A line chart connects points as if values between them exist, so avoid it for unordered categories like islands. Pie charts make small differences hard to see. A histogram of a category column is really a bar chart. And a mean bar chart alone hides spread, so add error bars or show the points themselves.

• No lines for categories
• Avoid crowded pies
• Show spread, not just mean

A bar chart encodes value as length, so its axis must start at zero. Cut the axis at ninety and a difference between ninety-two and ninety-four looks like a tripling. The numbers did not change, only the framing. For bars, start at zero. For line or scatter plots, a non-zero baseline can be fine, as long as the axis is clearly labeled.

• Bars must start at zero
• Same data, different story
• Label the baseline

Other tricks are quieter. Squash a plot tall and a gentle slope looks steep. Use a log scale without saying so and growth looks calm. Showing only averages hides everything, which is exactly what Anscombe's quartet exposed at the start. When you make a chart, ask what a skeptical reader could be misled by, and fix that.

• Aspect ratio changes slope
• Log scale needs a label
• Averages hide shape

Now let's assemble everything into a routine you can reuse on any dataset. One: check the shape. Two: check column types. Three: count missing values. Four: read the summary statistics and look for impossible values. Five: study the target, its distribution if numeric, or class balance if categorical. These five steps take minutes and prevent days of confusion.

• Shape and types
• Missing values
• Summary and target

Six: plot every feature on its own, with histograms for numbers and bar charts for categories. Seven: plot features against the target, using hue, box plots or violins. Eight: plot features against each other with a pair plot. Nine: draw the correlation heatmap and open the scatter plots behind any strong cell. Ten: write down every suspicious thing you saw.

• Features alone, then in pairs
• Heatmap, then scatters
• Write down suspicions

Run it on penguins. Shape is three hundred forty-four by seven. Missing values appear in a few measurements and in sex. The target, species, is mildly imbalanced. Body mass and flipper length separate Gentoo from the others. Bill length and depth reveal Simpson's paradox. Flipper and mass are strongly related. These findings are your notes. Deciding what to do about them comes next.

• Mild imbalance
• Species separate on size
• Simpson's paradox seen

A few habits keep EDA honest. Do not explore your final test data, or you leak information into your decisions; lesson 35 covers that properly. Do not go hunting until a pattern appears, because with enough charts, chance will show you something. And do not skip the boring checks. Most real data bugs are found in the first five steps.

• Do not peek at test data
• Beware chance patterns
• Do the boring checks

Let's recap. Matplotlib draws figures that hold axes, and the object style is clearer. Seaborn gives histograms, KDE, box, violin, scatter, pair and heatmap plots. Binning, bandwidth, overplotting and truncated axes can all distort what you see. Correlation is a pointer, not a cause. And the ten step checklist makes your exploration repeatable.

• Figure and axes
• Seaborn plot toolkit
• Charts can mislead

Our EDA found missing values, odd entries and different feature types. In lesson 34 we fix them: handling missing data, treating outliers, and preparing feature types for models.

• Lesson 34: data cleaning
• Try the checklist yourself
• Subscribe for more