Test & Code - Python Testing & Development

A recent Twitter thread by Simon Willison reminded me that I've been meaning to do an episode on the testing trophy. This discussion is about the distinction between unit and integration tests, what those terms mean, and where we should spend our testing time.Links:Simon Willison's Twitter ThreadThe Testing Trophy and Testing Classifications — Kent C DoddsWrite tests. Not too many. Mostly integration. — Kent C DoddsOn the Diverse And Fantastical Shapes of Testing — Martin Fowler

The idea of having a software as a service product sound great, doesn't it? Solve a problem with software. Have a nice looking landing page and website. Get paying customers. Eventually have it make enough revenue so you can turn it into your primary source of income. There's a lot of software talent out there. We could solve lots of problems.  But going from idea to product to first customer is non-trivial.  Especially as a side hustle.  This episode discusses some of the hurdles from idea to first customer. Brandon Braner is building Released.sh. It's a cool idea, but it's not done yet. Brandon and I talk about building side projects:finding a target audiencelimiting scope to something doable by one personbuilding a great looking landing pagefinding time to work on thingsprioritizing and planninglearning while buildingeven utilizing third party services to allow you to launch fasterand last, but not least, having funSpecial Guest: Brandon Braner.Links:ReleasedTailwind CSS Tailwind UIFigmaHerokuGoogle App En

Who should do QA?How does that change with different projects and teams?What does "doing QA" mean, anyway?Answering these questions are the goals of this episode.Links:Test Automation - Who Should be Involved? | Thoughtworks

In this episode, I talk with Paul Ganssle about a fun workflow that he calls pseudo-TDD. Pseudo-TDD is a way to keep your commit history clean and your tests passing with each commit. This workflow includes using pytest xfail and some semi-advanced version control features.Some strict forms of TDD include something like this:write a failing test that demonstrates a lacking feature or defectwrite the source code to get the test to passrefactor if necessaryrepeatIn reality, at least for me, the software development process is way more messy than this, and not so smooth and linear.Pauls workflow allow you to develop non-linearly, but commit cleanly.Links:A pseudo-TDD workflow using expected failuresepisode 171: How and why I use pytest's xfail - Paul Ganssleepisode 165: pytest xfail policy and workflowepisode 162: Flavors of TDD

In the preface of "Python Testing with pytest" I list some reasons to use pytest, under a section called "why pytest?". Someone asked me recently, a different but related question "why NOT unittest?".unittest is an xUnit style framework. For me, xUnit style frameworks are fatally flawed for software testing.That's what this episode is about, my opinion of "Why NOT unittest?", or more broadly, "What are the fatal flaws of xUnit?"Links:Python Testing with pytest, Second Editionunittest docsunittest assert methodsxUnit - Wikipedia

A prototype is a a preliminary model of something, from which other forms are developed or copied. In software, we think of prototypes as early things, or a proof of concept. We don't often think of prototyping during daily software development or maintenance. I think we should. This episode is about growing better designed software with the help of a prototype mindset.Links:Selecting a programming language can be a form of premature optimization — Brett Cannon's blog post

Paul Ganssle, is a software developer at Google, core Python dev, and open source maintainer for many projects, has some thoughts about pytest's xfail. He was an early skeptic of using xfail, and is now an proponent of the feature. In this episode, we talk about some open source workflows that are possible because of xfail. Special Guest: Paul Ganssle.Links:How and why I use pytest's xfail — Paul's blog post mentioned in the episodeCraft Minimal Bug Reports — Matthew Rocklin's articleepisode 111: Subtests in Python with unittest and pytest - Paul Ganssleepisode 165: pytest xfail policy and workflowepisode 166: unittest expectedFailure and xfail

Prayson Daniel, a principle data scientist, discusses testing machine learning pipelines with pytest.Prayson is using pytest for some pretty cool stuff, including:unit tests, of coursetesting pipeline stagescounterfactual testingperformance testingAll with pytest. So cool.Special Guest: Prayson Daniel.Links:Python Bytes 250, with Prayson Daniel — Listen to this for more of an introduction to Prayson

Performance monitoring and error detection is just as important with services and microservices as with any system, but with added complexity. Omri Sass joins the show to explain telemetry and monitoring of services and of systems with services. Special Guest: Omri Sass.

To understand complex code, it can be helpful to remove abstractions, even if it results in larger functions. This episode walks through a process I use to refactor code that I need to debug and fix, but don't completely understand.

Paul has a tutorial on testing and TDD with React and TypeScript.  We discuss workflow and the differences, similarities between testing with React/TypeScript and Python. We also discuss what lessons that we can bring from front end testing to Python testing.Special Guest: Paul Everitt.Links:React, TypeScript, and TDD — Paul Everitt's tutorialReact Testing Library

xfail isn't just for pytest tests. Python's unittest has @unittest.expectedFailure.In this episode, we cover:using @unittest.expectedFailurethe results of passing and failing tests with expectedFailureusing pytest as a test runner for unittestusing pytest markers on unittest testsDocs for expectedFailure:  https://docs.python.org/3/library/unittest.html#skipping-tests-and-expected-failuresSome sample code.  unittest only:import unittest class ExpectedFailureTestCase(unittest.TestCase): @unittest.expectedFailure def test_fail(self): self.assertEqual(1, 0, "broken") @unittest.expectedFailure def test_pass(self): self.assertEqual(1, 1, "not broken") unittest with pytest markers:import unittest import pytest class ExpectedFailureTestCase(unittest.TestCase): @pytest.mark.xfail def test_fail(self): self.assertEqual(1, 0, "broken") @pytest.mark.xfail def test_pass(self): self.assertEqual(1, 1, "not broken")

A discussion of how to use the xfail feature of pytest to help with communication on software projects.The episode covers:What is xfailWhy I use itUsing reason effectively by including issue tracking numbersUsing xfail_strictAdding --runxfail when transitioning from development to feature freezeWhat to do about test failuresHow all of this might help with team communication

An overview of the pytest flags that help with debugging. From Chapter 13, Debugging Test Failures, of Python Testing with pytest, 2nd edition.pytest includes quite a few command-line flags that are useful for debugging. We talk about thes flags in this episode.Flags for selecting which tests to run, in which order, and when to stop:-lf / --last-failed: Runs just the tests that failed last.-ff / --failed-failed: Runs all the tests, starting with the last failed.-x / --exitfirst: Stops the tests session afterEd: after?Author: yep the first failure.--maxfail=num: Stops the tests after num failures.-nf / --new-first: Runs all the tests, ordered by file modification time.--sw / --stepwise: Stops the tests at the first failure. Starts the tests at the last failure next time.--sw-skip / --stepwise-skip: Same as --sw, but skips the first failure.Flags to control pytest output:-v / --verbose Displays all the test names, passing or failing.--tb=[auto/long/short/line/native/no] Controls the traceback style.-l / --showl

pip : "pip installs packages" or maybe "Package Installer for Python" pip is an invaluable tool when developing with Python. A lot of people know pip as a way to install third party packages from pypi.org You can also use pip to install from other indexes (or is it indices?)You can also use pip to install a package in a local directory. That's the part I want to jump in and explore with Stéphane Bidoul. The way pip installs from a local directory is about to change, and the story is fascinating.Special Guest: Stéphane Bidoul.Links:The Odoo Community AssociationPEP 610 -- Recording the Direct URL Origin of installed distributions | Python.orgPEP 660 -- Editable installs for pyproject.toml based builds (wheel based) | Python.org — Bidoulpip install --no-index --find-links Solving issues related to out-of-tree builds · Issue #7555 · pypa/pippip list json format

What flavor of TDD do you practice? In this episode we talk about:Classical vs Mockist TDDDetroit vs London (I actually refer to it in the episode as Chicago instead of Detroit. Oh well.)Static vs BehaviorInside Out vs Outside InDouble Loop TDDBDDFDDTracer BulletsRules of TDDTeam StructureLean TDD This is definitely an episode I'd like feedback on. Reach out to me for further questions or if I missed some crucial variant of TDD that you know and love.Links:Mocks Aren't Stubs - Martin FowlerMockists Are Dead. Long Live Classicists.Double Loop TDDBDD: Behavior-driven developmentFDD: Feature-driven developmentMy reaction to “Is TDD Dead?” - pythontest.comTest First Programming / Test First DevelopmentHumorous list of TDD variants — BDD = Buzzword Driven Development, CDD = Calendar Driven Development, etc

Software development processes create value, and have waste, in the Lean sense of the word waste. Lean manufacturing and lean software development changed the way we look at value and waste. This episode looks at lean definitions of waste, so we can see it clearly when we encounter it.I'm going to use the term waste and value in future episodes. I'm using waste in a Lean sense, so we can look at software processes critically, see the value chain, and try to reduce waste.Lean manufacturing and lean software development caused people to talk about and examine waste and value, even in fields where we didn't really think about waste that much to begin with.Software is just ones and zeros. Is there waste?  When I delete a file, nothing goes into the landfill.The mistake I'm making here is confusing the common English definition of waste when what we're talking about is the Lean definition of waste.This episode tries to clear up the confusion.Links:Big Design Up FrontLightweight MethodologiesManifesto for Agile Sof

Should your code be DRY or DAMP or something completely different? How about your test code? Do different rules apply? Wait, what do all of these acronyms mean?We'll get to all of these definitions, and then talk about how it applies to both production code and test code in this episode. Links:The Pragmatic Programmer, 20th Anniversary EditionDon't repeat yourself - Wikipediaa-ha - Take On MeRule of three - WikipediaWhat does “DAMP not DRY” mean when talking about unit tests? - Stack Overflow

When learning data science and machine learning techniques, you need to work on a data set. Matt Harrison had a great idea: Why not use your own Twitter analytics data? So, he did that with his own data, and shares what he learned in this episode, including some of his secrets to gaining followers.In this episode we talk about:Looking at your own Twitter analytics data.Using Python, pandas, Jupyter for data cleaning and exploratory analysisData visualizationMachine learning, principal component analysis, clusteringModel drift and re-running analysisWhat kind of tweets perform wellAnd much moreLinks:Applied Pandas: Twitter Analytics — the course

Iterative processes that include writing test code and production code together, such as TDD, help make coding fun. All of us that care about developing quality code with the help of testing can learn from each other, regardless of programming language.Today we step outside our normal Python comfort zone and talk with Gio about TDD in Swift.Gio Lodi, author of TDD in Swift, joins the show to discuss Test Driven Development, software workflows, bottom up vs top down, rapid feedback, developer vs customer facing tests, and more.Links:TDD in Swift with SwiftUI and Combinemokacoding - Gio's blogXCTestSoftware Design: Tidy First? - Kent Beck on Substack