CI/CD Pipelines Explained: From Push to Production

Part 17 of 17 · ~3 min

Common Mistakes Worth Remembering

Every one of these actually happened somewhere above.

• Letting a human's memory be the source of truth for what's deployed where. Priya's Friday docker pull of the wrong tag is the entire reason this pipeline exists — the fix wasn't "be more careful," it was removing the step where a person has to remember anything at all.
• Writing jobs as a sequential list out of habit instead of an actual dependency graph. lint and unit-tests cost nothing extra by running side by side; the only reason not to is not having noticed neither one is waiting on the other.
• Building twice instead of shipping one artifact everywhere. Two separate docker build runs of "the same commit," two and a half hours apart, is exactly the failure a single tagged image sidesteps — and the gap is invisible right up until the one thing that happened to move lines up with the one dependency that happened to care.
• Sending every push through the slow browser-driven suite instead of layering checks by speed. Cheap checks belong on every push; the expensive one belongs on a clock — and accepting, honestly, that the trade-off means a renamed CSS class can survive a full workday before the nightly run catches it.
• Treating "a human approves this" as a policy instead of a mechanism. Writing it in a doc doesn't stop a deploy; an environment: block with a required reviewer does, because the job is mechanically unable to proceed without the click.
• Putting a secret anywhere it can end up baked into something durable — a build argument, a hardcoded value, a debug step that prints environment variables to a log a fork-triggered PR might have access to.
• Choosing a deployment strategy without thinking through what it does to the database underneath it. Rolling updates mean old and new code run against the same schema at the same time, for real, for however long the rollout takes — not as a footnote, as an active constraint on every migration written during that window.
• Assuming a code rollback undoes everything a bad deploy did. It undoes the code. A schema change that already ran is still there, NOT NULL and all, and "rolled back" can mean "identically broken" if nobody accounts for that.
• Conflating "deployed" with "released." The auto-categorization bug got fixed by flipping a flag in seconds, specifically because turning a feature on and shipping the code for it had already been split into two separate decisions.
• Trusting a green pipeline as proof the app is healthy. It's proof the code that exists got built and passed its tests. It says nothing about a configuration value that lives outside the repository entirely — that's what a smoke test after deploy is actually for.

Everything here picks up exactly where Docker Fundamentals left off — an image, built once, correctly, is the raw material; this is the machine that gets it from a laptop to a customer's screen on a schedule, without anyone needing to remember which tag is the right one at 6:40 on a Friday. And once web and worker stop being two services one small team ships together and start being services several teams each own separately, the question of how many of these pipelines you're actually maintaining — and how independently they can move — is exactly what Microservices vs. Monolith is about. If you'd rather feel a test actually block a bad change than just read about it, the code lab is set up for exactly that.