levelmax World 4 Testing Properly
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. B
L
Level 4.1 ~12 min Target: PHPStan max

What's worth testing

Four tests cover the loyalty points calculation: it returns an int, it never returns null, it rejects a string, and £25 earns 25 points. They all pass. But the first three can't fail while the signature stays the same, and PHPStan already checks those things at every call site, not only in the test.

Three of the four tests check things the types already guarantee. Only one checks the answer.

What the types prove, and what they don't

Already proved by types at max Still needs a test
It returns an int £25 earns 25 points
It never returns null £24.99 earns 24, not 25
Callers pass an int An order of exactly £100 earns double

Here's a quick check for any test. Imagine replacing the method's body with return 0;. A test that still passes doesn't check behaviour. assertIsInt() still passes. assertSame(25, …) doesn't.

PHPStan reads your tests too

From this world on, PHPStan analyses your tests at max as well as the code under test. It reports two kinds of tests that can't fail:

Call to static method PHPUnit\Framework\Assert::assertIsInt() with int will always evaluate to true.
Parameter #1 $pence of static method LoyaltyPoints::forOrder() expects int<0, max>, '2500' given.

The first is an assertion PHPStan has already proved. The second is a test of PHP's type checks, which strict types already enforce. Both mean the test can be deleted. Don't silence them with an ignore.

PHPStan only reports what it can prove. assertGreaterThan(0, $points) on an int goes through, so the return 0; check still matters.

How this world grades you

In this world you write the tests, and the code under test is read-only. Passing isn't enough, because a test that checks nothing also passes.

So each exercise plants bugs. Once your tests pass against the real code, they run again against copies of it with one bug planted in each. A good test suite fails against every one. The runner lists each bug your tests miss, and the level clears when they catch them all. This is mutation testing, done by hand. 4.5 does it with a tool.

The test suite40 tests, all green
A bug in the mathsSurvives all 40
31 of them were checking the return type.

Power-up

Before
<?php

use PHPUnit\Framework\TestCase;

final class LoyaltyPointsTest extends TestCase
{
    public function testItReturnsAnInt(): void
    {
        self::assertIsInt(LoyaltyPoints::forOrder(2500));
    }

    public function testItNeverReturnsNull(): void
    {
        self::assertNotNull(LoyaltyPoints::forOrder(2500));
    }

    public function testItRejectsPenceAsAString(): void
    {
        $this->expectException(TypeError::class);
        LoyaltyPoints::forOrder('2500');
    }
}
After
<?php

use PHPUnit\Framework\TestCase;

final class LoyaltyPointsTest extends TestCase
{
    public function testPartOfAPoundEarnsNothing(): void
    {
        self::assertSame(24, LoyaltyPoints::forOrder(2499));
    }

    public function testOrdersFrom100PoundsEarnDoublePoints(): void
    {
        self::assertSame(200, LoyaltyPoints::forOrder(10000));
    }
}

Quick check

1. forOrder(int $pence): int is analysed at max. Which test is worth keeping?
2. You replace a method's body with return 0; and one of its tests still passes. What does that tell you?
3. A test suite has 100% line coverage. What do you know?

CHALLENGE

The tests for LoyaltyPoints::forOrder() (read only) all pass. PHPStan analyses your tests at max as well, and Analyse reports three of them.

In tests/LoyaltyPointsTest.php, delete the tests that only prove what the types already do. Then add tests until they catch every planted bug. Once your tests pass, Test runs them against each bug and lists the ones they miss.

Running on PHP 8.3…

Run the tests, run PHPStan, or submit for both. Ctrl/⌘ + Enter submits from the editor.