Introduktion til ejendomsbaseret test i Python

I denne artikel lærer vi en unik og effektiv tilgang til test kaldet ejendomsbaseret test. Vi bruger Python , pytest og hypotese til at implementere denne testmetode.

Artiklen vil bruge grundlæggende pytest-koncepter til at forklare ejendomsbaseret test. Jeg anbefaler, at du læser denne artikel for hurtigt at opfriske din pytest-viden.

Vi starter med den konventionelle enheds / funktionelle testmetode kendt som eksempelbaseret test -som de fleste af os bruger. Vi forsøger at finde manglerne og derefter gå til den ejendomsbaserede tilgang for at fjerne disse mangler.

Hvert stort magisk trick består af tre dele eller handlinger. Den første del hedder “The Pledge”. Tryllekunstneren viser dig noget almindeligt : et kort kort, en fugl eller en mand. Han viser dig dette objekt. Måske beder han dig om at inspicere det for at se, om det virkelig er ægte, uændret, normalt. Men selvfølgelig ... er det sandsynligvis ikke.

Del 1: Eksempelbaseret test

Tilgangen til eksempelbaseret test har følgende trin:

  • givet en testindgang I
  • når den sendes til at fungere under test
  • skal returnere et output O

Så dybest set giver vi et fast input og forventer et fast output.

At forstå dette begreb i lægmandssæt:

Antag, at vi har en maskine, der tager plast af enhver form og enhver farve som input og producerer en perfekt rund plastkugle i samme farve som output.

For at teste denne maskine ved hjælp af eksempelbaseret test følger vi nedenstående fremgangsmåde:

  1. tage en blåfarvet rå plastik ( faste testdata )
  2. før plasten til maskinen
  3. forvent en blåfarvet plastkugle som output ( fast testoutput )

Lad os se den samme tilgang på en programmatisk måde.

Forudsætning: Sørg for, at du har Python (ver 2.7 eller nyere) og pytest installeret.

Opret en katalogstruktur som denne:

- demo_tests/ - test_example.py

Vi skriver en lille funktion suminde i filen test_example.py. Dette accepterer to tal - num1og num2 - som parametre og returnerer tilføjelsen af ​​begge tal som resultat.

def sum(num1, num2): """It returns sum of two numbers""" return num1 + num2

Lad os nu skrive en test for at teste denne sumfunktion efter den konventionelle metode.

import pytest
#make sure to start function name with testdef test_sum(): assert sum(1, 2) == 3

Her kan man se, at vi passerer de to værdier 1og 2og forventer summen til gengæld 3.

Kør testene ved at krydse til demo_testsmappen og derefter køre følgende kommando:

pytest test_example.py -v

Er denne test tilstrækkelig til at verificere funktionens sumfunktion?

Du tænker måske, selvfølgelig ikke. Vi skriver flere tests ved hjælp af pytest parametrizefunktionen, der udfører denne test_sumfunktion for alle de givne værdier.

import pytest
@pytest.mark.parametrize('num1, num2, expected',[(3,5,8), (-2,-2,-4), (-1,5,4), (3,-5,-2), (0,5,5)])def test_sum(num1, num2, expected): assert sum(num1, num2) == expected

Brug af fem tests har givet mere tillid til funktionaliteten. Alle forbipasserende føles som lyksalighed.

Men hvis du ser nærmere på, gør vi det samme, som vi gjorde ovenfor, men for flere antal værdier. Vi dækker stadig ikke flere af edge cases.

Så vi har opdaget det første smertepunkt med denne testmetode:

Udgave 1: Testens udtømmende karakter afhænger af den person, der skriver testen

De kan vælge at skrive 5 eller 50 eller 500 testsager, men er stadig usikre på, om de sikkert har dækket de fleste, hvis ikke alle, edge-sagerne.

Dette bringer os til vores andet smertepunkt:

Udgave 2 - Ikke-robuste tests på grund af uklar / tvetydig kravforståelse

Da vi fik at vide at skrive vores sumfunktion, hvilke konkrete detaljer blev formidlet?

Fik vi at vide:

  • hvilken slags input vores funktion kan forvente?
  • hvordan vores funktion skal opføre sig i uventede inputscenarier?
  • hvilken slags output skal vores funktion returnere?

For at være mere præcis, hvis du overvejer den sumfunktion, vi har skrevet ovenfor:

  • ved vi, om num1, num2skal være en inteller float? Kan de også sendes som type string eller anden datatype?
  • what is the minimum and maximum value of num1 and num2 that we should support?
  • how should the function behave if we get null inputs?
  • should the output returned by the sum function be int or float or string or any other data type?
  • in what scenarios should it display error messages?

Also, the worst case scenario of the above test case writing approach is that these test cases can be fooled to pass by buggy functions.

Let’s re-write our sum function in a way that errors are introduced but the tests which we have written so far still passes.

def sum(num1, num2): """Buggy logic""" if num1 == 3 and num2 == 5: return 8 elif num1 == -2 and num2 == -2 : return -4 elif num1 == -1 and num2 == 5 : return 4 elif num1 == 3 and num2 == -5: return -2 elif num1 == 0 and num2 == 5: return 5

Now let’s dive into property-based testing to see how these pain points are mitigated there.

Anden handling hedder “Turn”. Tryllekunstneren tager det almindelige og får det til at gøre noget ekstraordinært. Nu leder du efter hemmeligheden ... men du finder den ikke, for du ser selvfølgelig ikke rigtig ud. Du vil ikke rigtig vide det. Du vil lade dig narre.

Del 2: Ejendomsbaseret test

Introduktion og testdata generering

Ejendomsbaseret test blev først introduceret af QuickCheck- rammen i Haskell . Ifølge dokumentationen til hurtig kontrol, som er et andet egenskabsbaseret testbibliotek-

Ejendomsbaserede testrammer kontrollerer sandheden af ​​ejendomme. En egenskab er en sætning som: for alle (x, y,…) som forudgående betingelse (x, y,…) holder ejendom (x, y,…) er sandt .

To understand this let’s go back to our plastic ball generating machine example.

The property based testing approach of that machine will be:

  1. take a huge selection of plastics as input (all(x, y, …))
  2. make sure all of them are colored (precondition(x, y, …))
  3. the output satisfies following property (property(x, y, …)) -
  • output is round/spherical in shape
  • output is colored
  • color of the output is one of the colors present in color band

Notice how from fixed values of input and output we have generalized our test data and output in such a way that the property should hold true for all the valid inputs. This is property-based testing.

Also, notice that when thinking in terms of properties we have to think harder and in a different way. Like when we came up with the idea that since our output is a ball it should be round in shape, another question will strike you - whether the ball should be hollow or solid?

So, by making us think harder and question more about the requirement, the property-based testing approach is making our implementation of the requirement robust.

Now, let’s return to our sum function and test it by using the property-based approach.

The first question which arises here is: what should be the input of the sum function?

For the scope of this article we will assume that any pair of integers from the integer set is a valid input.

So, any set of integer values lying in the above coordinate system will be a valid input to our function.

The next question is: how to get such input data?

The answer to this is: a property-based testing library provides you the feature to generate huge set of desired input data following a precondition.

In Python, Hypothesis is a property-testing library which allows you to write tests along with pytest. We are going to make use of this library.

The entire documentation of Hypothesis is beautifully written and available ➡️ hereand I recommend you to go through it.

To install Hypothesis:

pip install hypothesis

and we are good to use hypothesis with pytest.

Now, let’s rewrite test_sum function — which we wrote earlier — with new data sets generated by Hypothesis.

from hypothesis import given
import hypothesis.strategies as st
import pytest
@given(st.integers(), st.integers())def test_sum(num1, num2): assert sum(num1, num2) == num1 + num2
  • The first line simply imports given from Hypothesis. The @given decorator takes our test function and turns it into a parametrized one. When called, this will run the test function over a wide range of matching data. This is the main entry point to Hypothesis.
  • The second line imports strategies from Hypothesis. strategies provides the feature to generate test data. Hypothesis provides strategies for most built-in types with arguments to constrain or adjust the output. As well, higher-order strategies can be composed to generate more complex types.
  • You can generate any or mix of the following things using strategies:
'nothing','just', 'one_of','none','choices', 'streaming','booleans', 'integers', 'floats', 'complex_numbers', 'fractions','decimals','characters', 'text', 'from_regex', 'binary', 'uuids','tuples', 'lists', 'sets', 'frozensets', 'iterables','dictionaries', 'fixed_dictionaries','sampled_from', 'permutations','datetimes', 'dates', 'times', 'timedeltas','builds','randoms', 'random_module','recursive', 'composite','shared', 'runner', 'data','deferred','from_type', 'register_type_strategy', 'emails'
  • Here we have generated integers()set using strategies and passed it to @given.
  • So, our test_sum function should run for all the iterations of given input.

Let’s run it and see the result.

You might be thinking, I can’t see any difference here. What’s so special about this run?

Well, to see the magical difference, we need to run our test by setting the verbose option. Don’t confuse this verbose with the -v option of pytest.

from hypothesis import given, settings, Verbosity
import hypothesis.strategies as stimport pytest
@settings(verbosity=Verbosity.verbose)@given(st.integers(), st.integers())def test_sum(num1, num2): assert sum(num1, num2) == num1 + num2

settings allows us to tweak the default test behavior of Hypothesis.

Now let’s re-run the tests. Also include -s this time to capture the stream output in pytest.

pytest test_example.py -v -s

Look at the sheer number of test-cases generated and run. You can find all sorts of cases here, such as 0, large numbers, and negative numbers.

You might be thinking, it’s impressive, but I can’t find my favorite test case pair (1,2 ) here. What if I want that to run?

Well, fear not, Hypothesis allows you to run a given set of test cases every time if you want by using the @example decorator.

from hypothesis import given, settings, Verbosity, example
import hypothesis.strategies as stimport pytest
@settings(verbosity=Verbosity.verbose)@given(st.integers(), st.integers())@example(1, 2)def test_sum(num1, num2): assert sum(num1, num2) == num1 + num2

Also, notice that each run will always generate a new jumbled up test case following the test generation strategy, thus randomizing the test run.

So, this solves our first pain point- the exhaustiveness of test cases.

Thinking hard to come up with properties to test

So far, we saw one magic of property-based testing which generates desired test data on the fly.

Now let’s come to the part where we need to think hard and in a different way to create such tests which are valid for all test inputs but unique to sum function.

1 + 0 = 10 + 1 = 15 + 0 = 5-3 + 0 = -38.5 + 0 = 8.5

Well, that’s interesting. It seems like adding 0 to a number results in the same number as sum. This is called the identity property of addition.

Let’s see one more:

2 + 3 = 53 + 2 = 5
5 + (-2) = 3-2 + 5 = 3

It looks like we found one more unique property. In addition the order of parameters doesn’t matter. Placed left or right of the + sign they give the same result. This is called the commutative property of addition.

There is one more, but I want you to come up with it.

Now, we will re-write our test_sum to test these properties:

from hypothesis import given, settings, Verbosity
import hypothesis.strategies as stimport pytest
@settings(verbosity=Verbosity.verbose)@given(st.integers(), st.integers())def test_sum(num1, num2): assert sum(num1, num2) == num1 + num2
 # Test Identity property assert sum(num1, 0) = num1 #Test Commutative property assert sum(num1, num2) == sum(num2, num1)

Vores test er nu udtømmende - vi har også konverteret testene for at gøre dem mere robuste. Således løste vi vores andet smertepunkt: ikke-robuste testsager .

Bare for nysgerrigheds skyld, lad os prøve at narre denne test med den buggy-kode, vi brugte før.

Som et gammelt ordsprog siger - nar mig en gang, skam dig, nar mig to gange, skam mig.

Du kan se, at det fik en fejl. Falsifying example: test_sum(num1=0, num2=0). Det betyder simpelthen, at vores forventede ejendom ikke var i overensstemmelse med disse par testsager, og dermed fiaskoen.

Men du ville ikke klappe endnu. Fordi det ikke er nok at få noget til at forsvinde; du er nødt til at bringe det tilbage. Derfor har hvert magisk trick en tredje akt, den sværeste del, den del, vi kalder ”The Prestige”.

Del 3: Krympefejl

Shrinking is the process by which Hypothesis tries to produce human-readable examples when it finds a failure. It takes a complex example and turns it into a simpler one.

To demonstrate this feature, let’s add one more property to our test_sum function which says num1 should be less than or equal to 30.

from hypothesis import given, settings, Verbosity
import hypothesis.strategies as stimport pytest
@settings(verbosity=Verbosity.verbose)@given(st.integers(), st.integers())def test_sum(num1, num2): assert sum(num1, num2) == num1 + num2
 # Test Identity property assert sum(num1, 0) = num1 #Test Commutative property assert sum(num1, num2) == sum(num2, num1) assert num1 <= 30

After running this test, you will get an interesting output log on the terminal here:

collected 1 item
test_example.py::test_sum Trying example: test_sum(num1=0, num2=-1)Trying example: test_sum(num1=0, num2=-1)Trying example: test_sum(num1=0, num2=-29696)Trying example: test_sum(num1=0, num2=0)Trying example: test_sum(num1=-1763, num2=47)Trying example: test_sum(num1=6, num2=1561)Trying example: test_sum(num1=-24900, num2=-29635)Trying example: test_sum(num1=-13783, num2=-20393)
#Till now all test cases passed but the next one will fail
Trying example: test_sum(num1=20251, num2=-10886)assert num1 <= 30AssertionError: assert 20251 <= 30
#Now the shrinking feature kicks in and it will try to find the simplest value for which the test still fails
Trying example: test_sum(num1=0, num2=-2)Trying example: test_sum(num1=0, num2=-1022)Trying example: test_sum(num1=-165, num2=-29724)Trying example: test_sum(num1=-14373, num2=-29724)Trying example: test_sum(num1=-8421504, num2=-8421376)Trying example: test_sum(num1=155, num2=-10886)assert num1 <= 30AssertionError: assert 155 <= 30
# So far it has narrowed it down to 155
Trying example: test_sum(num1=0, num2=0)Trying example: test_sum(num1=0, num2=0)Trying example: test_sum(num1=64, num2=0)assert num1 <= 30AssertionError: assert 64 <= 30
# Down to 64
Trying example: test_sum(num1=-30, num2=0)Trying example: test_sum(num1=0, num2=0)Trying example: test_sum(num1=0, num2=0)Trying example: test_sum(num1=31, num2=0)
# Down to 31
Trying example: test_sum(num1=-30, num2=0)Falsifying example: test_sum(num1=31, num2=0)FAILED
# And it finally concludes (num1=31, num2=0) is the simplest test data for which our property doesn't hold true.

One more good feature — its going to remember this failure for this test and will include this particular test case set in the future runs to make sure that the same regression doesn’t creep in.

Dette var en mild introduktion til magien ved ejendomsbaseret test. Jeg anbefaler alle jer at prøve denne tilgang i din daglige test. Næsten alle større programmeringssprog har egenskabsbaseret teststøtte.

Du kan finde hele koden brugt her i min? github repo.

Hvis du kunne lide indholdet, så vis nogle ❤️