This tutorial will guide you through the process of creating your own containers.
First things first, why would anyone want to create a custom containers?
The great idea about “containers” in functional programming is that it can be literally anything. There are endless use-cases.
You can create your own primitives for working with some language-or-framework specific problem, or just model your business domain.
You can copy ideas from other languages or just compose existing containers
for better usability
(like IOResult
is the composition of IO
and Result
).
Example
We are going to implement a Pair
container for this example.
What is a Pair
? Well, it is literally a pair of two values.
No more, no less. Similar to a Tuple[FirstType, SecondType]
.
But with extra goodies.
Note
You can find all code samples here.
After you came up with the idea, you will need to make a decision: what capabilities my container must have?
Basically, you should decide what Interfaces you will subtype and what
methods and laws will be present in your type.
You can create just a returns.interfaces.mappable.MappableN
or choose a full featured returns.interfaces.container.ContainerN
.
You can also choose some specific interfaces to use,
like returns.interfaces.specific.result.ResultLikeN
or any other.
Summing up, decide what laws and methods you need to solve your problem. And then subtype the interfaces that provide these methods and laws.
Example
What interfaces a Pair
type needs?
returns.interfaces.equable.Equable
,
because two Pair
instances can be compared
returns.interfaces.mappable.MappableN
,
because the first type can be composed with pure functions
returns.interfaces.bindable.BindableN
,
because a Pair
can be bound to a function returning a new Pair
based on the first type
returns.interfaces.altable.AltableN
,
because the second type can be composed with pure functions
returns.interfaces.lashable.LashableN
,
because a Pair
can be bound to a function returning a new Pair
based on the second type
Now, after we know about all interfaces we would need, let’s find pre-defined aliases we can reuse.
Turns out, there are some of them!
returns.interfaces.bimappable.BiMappableN
which combines MappableN
and AltableN
returns.interfaces.swappable.SwappableN
is an alias for BiMappableN
with a new method called .swap
to change values order
Let’s look at the result:
Note
A special note on returns.primitives.container.BaseContainer
.
It is a very useful class with lots of pre-defined features, like:
immutability, better cloning, serialization, and comparison.
You can skip it if you wish, but it is highlighly recommended.
Later we will talk about an actual implementation of all required methods.
So, let’s start writing some code!
We would need to implement all interface methods,
otherwise mypy
won’t be happy.
That’s what it currently says on our type definition:
error: Final class test_pair1.Pair has abstract attributes "alt", "bind", "equals", "lash", "map", "swap"
Looks like it already knows what methods should be there!
Ok, let’s drop some initial and straight forward implementation. We will later make it more complex step by step.
1from typing import Callable, Tuple, TypeVar, final
2
3from returns.interfaces import bindable, equable, lashable, swappable
4from returns.primitives.container import BaseContainer, container_equality
5from returns.primitives.hkt import Kind2, SupportsKind2, dekind
6
7_FirstType = TypeVar('_FirstType')
8_SecondType = TypeVar('_SecondType')
9
10_NewFirstType = TypeVar('_NewFirstType')
11_NewSecondType = TypeVar('_NewSecondType')
12
13
14@final
15class Pair(
16 BaseContainer,
17 SupportsKind2['Pair', _FirstType, _SecondType],
18 bindable.Bindable2[_FirstType, _SecondType],
19 swappable.Swappable2[_FirstType, _SecondType],
20 lashable.Lashable2[_FirstType, _SecondType],
21 equable.Equable,
22):
23 """
24 A type that represents a pair of something.
25
26 Like to coordinates ``(x, y)`` or two best friends.
27 Or a question and an answer.
28
29 """
30
31 def __init__(
32 self,
33 inner_value: Tuple[_FirstType, _SecondType],
34 ) -> None:
35 """Saves passed tuple as ``._inner_value`` inside this instance."""
36 super().__init__(inner_value)
37
38 # `Equable` part:
39
40 equals = container_equality # we already have this defined for all types
41
42 # `Mappable` part via `BiMappable`:
43
44 def map(
45 self,
46 function: Callable[[_FirstType], _NewFirstType],
47 ) -> 'Pair[_NewFirstType, _SecondType]':
48 return Pair((function(self._inner_value[0]), self._inner_value[1]))
49
50 # `BindableN` part:
51
52 def bind(
53 self,
54 function: Callable[
55 [_FirstType],
56 Kind2['Pair', _NewFirstType, _SecondType],
57 ],
58 ) -> 'Pair[_NewFirstType, _SecondType]':
59 return dekind(function(self._inner_value[0]))
60
61 # `AltableN` part via `BiMappableN`:
62
63 def alt(
64 self,
65 function: Callable[[_SecondType], _NewSecondType],
66 ) -> 'Pair[_FirstType, _NewSecondType]':
67 return Pair((self._inner_value[0], function(self._inner_value[1])))
68
69 # `LashableN` part:
70
71 def lash(
72 self,
73 function: Callable[
74 [_SecondType],
75 Kind2['Pair', _FirstType, _NewSecondType],
76 ],
77 ) -> 'Pair[_FirstType, _NewSecondType]':
78 return dekind(function(self._inner_value[1]))
79
80 # `SwappableN` part:
81
82 def swap(self) -> 'Pair[_SecondType, _FirstType]':
83 return Pair((self._inner_value[1], self._inner_value[0]))
You can check our resulting source with mypy
. It would be happy this time.
As you can see our existing interfaces do not cover everything. We can potentially want several extra things:
A method that takes two arguments and returns a new Pair
instance
A named constructor to create a Pair
from a single value
A named constructor to create a Pair
from two values
We can define an interface just for this! It would be also nice to add all other interfaces there as supertypes.
That’s how it is going to look:
1class PairLikeN(
2 bindable.BindableN[_FirstType, _SecondType, _ThirdType],
3 swappable.SwappableN[_FirstType, _SecondType, _ThirdType],
4 lashable.LashableN[_FirstType, _SecondType, _ThirdType],
5 equable.Equable,
6):
7 """Special interface for types that look like a ``Pair``."""
8
9 @abstractmethod
10 def pair(
11 self: _PairLikeKind,
12 function: Callable[
13 [_FirstType, _SecondType],
14 KindN[_PairLikeKind, _NewFirstType, _NewSecondType, _ThirdType],
15 ],
16 ) -> KindN[_PairLikeKind, _NewFirstType, _NewSecondType, _ThirdType]:
17 """Allows to work with both arguments at the same time."""
18
19 @classmethod
20 @abstractmethod
21 def from_paired(
22 cls: Type[_PairLikeKind],
23 first: _NewFirstType,
24 second: _NewSecondType,
25 ) -> KindN[_PairLikeKind, _NewFirstType, _NewSecondType, _ThirdType]:
26 """Allows to create a PairLikeN from just two values."""
27
28 @classmethod
29 @abstractmethod
30 def from_unpaired(
31 cls: Type[_PairLikeKind],
32 inner_value: _NewFirstType,
33 ) -> KindN[_PairLikeKind, _NewFirstType, _NewFirstType, _ThirdType]:
34 """Allows to create a PairLikeN from just a single object."""
Awesome! Now we have a new interface to implement. Let’s do that!
1 def pair(
2 self,
3 function: Callable[
4 [_FirstType, _SecondType],
5 Kind2['Pair', _NewFirstType, _NewSecondType],
6 ],
7 ) -> 'Pair[_NewFirstType, _NewSecondType]':
8 return dekind(function(self._inner_value[0], self._inner_value[1]))
1 @classmethod
2 def from_unpaired(
3 cls,
4 inner_value: _NewFirstType,
5 ) -> 'Pair[_NewFirstType, _NewFirstType]':
6 return Pair((inner_value, inner_value))
Looks like we are done!
The best part about this type is that it is pure. So, we can write our tests inside docs!
We are going to use doctests builtin module for that.
This gives us several key benefits:
All our docs has usage examples
All our examples are correct, because they are executed and tested
We don’t need to write regular boring tests
Let’s add docs and doctests! Let’s use map
method as a short example:
1 def map(
2 self,
3 function: Callable[[_FirstType], _NewFirstType],
4 ) -> 'Pair[_NewFirstType, _SecondType]':
5 """
6 Changes the first type with a pure function.
7
8 >>> assert Pair((1, 2)).map(str) == Pair(('1', 2))
9
10 """
11 return Pair((function(self._inner_value[0]), self._inner_value[1]))
By adding these simple tests we would already have 100% coverage. But, what if we can completely skip writing tests, but still have 100%?
Let’s discuss how we can achieve that with “Laws as values”.
We already ship lots of laws with our interfaces. See our docs on laws and checking them.
Moreover, you can also define your own laws!
Let’s add them to our PairLikeN
interface.
Let’s start with laws definition:
1class _LawSpec(LawSpecDef):
2 @law_definition
3 def pair_equality_law(
4 raw_value: _FirstType,
5 container: 'PairLikeN[_FirstType, _SecondType, _ThirdType]',
6 ) -> None:
7 """Ensures that unpaired and paired constructors work fine."""
8 assert_equal(
9 container.from_unpaired(raw_value),
10 container.from_paired(raw_value, raw_value),
11 )
12
13 @law_definition
14 def pair_left_identity_law(
15 pair: Tuple[_FirstType, _SecondType],
16 container: 'PairLikeN[_FirstType, _SecondType, _ThirdType]',
17 function: Callable[
18 [_FirstType, _SecondType],
19 KindN['PairLikeN', _NewFirstType, _NewSecondType, _ThirdType],
20 ],
21 ) -> None:
22 """Ensures that unpaired and paired constructors work fine."""
23 assert_equal(
24 container.from_paired(*pair).pair(function),
25 function(*pair),
26 )
And them let’s add them to our PairLikeN
interface:
1class PairLikeN(
2 bindable.BindableN[_FirstType, _SecondType, _ThirdType],
3 swappable.SwappableN[_FirstType, _SecondType, _ThirdType],
4 lashable.LashableN[_FirstType, _SecondType, _ThirdType],
5 equable.Equable,
6):
7 """Special interface for types that look like a ``Pair``."""
8
9 _laws: ClassVar[Sequence[Law]] = (
10 Law2(_LawSpec.pair_equality_law),
11 Law3(_LawSpec.pair_left_identity_law),
12 )
13
14 @abstractmethod
15 def pair(
16 self: _PairLikeKind,
17 function: Callable[
18 [_FirstType, _SecondType],
19 KindN[_PairLikeKind, _NewFirstType, _NewSecondType, _ThirdType],
20 ],
21 ) -> KindN[_PairLikeKind, _NewFirstType, _NewSecondType, _ThirdType]:
22 """Allows to work with both arguments at the same time."""
23
24 @classmethod
25 @abstractmethod
26 def from_paired(
27 cls: Type[_PairLikeKind],
28 first: _NewFirstType,
29 second: _NewSecondType,
30 ) -> KindN[_PairLikeKind, _NewFirstType, _NewSecondType, _ThirdType]:
31 """Allows to create a PairLikeN from just two values."""
32
33 @classmethod
34 @abstractmethod
35 def from_unpaired(
36 cls: Type[_PairLikeKind],
37 inner_value: _NewFirstType,
38 ) -> KindN[_PairLikeKind, _NewFirstType, _NewFirstType, _ThirdType]:
39 """Allows to create a PairLikeN from just a single object."""
The last to do is to call check_all_laws(Pair, use_init=True)
to generate 10 hypothesis
test cases with hundreds real test cases inside.
Here’s the final result of our brand new Pair
type:
1from abc import abstractmethod
2from typing import (
3 Callable,
4 ClassVar,
5 NoReturn,
6 Sequence,
7 Tuple,
8 Type,
9 TypeVar,
10 final,
11)
12
13from returns.contrib.hypothesis.laws import check_all_laws
14from returns.interfaces import bindable, equable, lashable, swappable
15from returns.primitives.asserts import assert_equal
16from returns.primitives.container import BaseContainer, container_equality
17from returns.primitives.hkt import Kind2, KindN, SupportsKind2, dekind
18from returns.primitives.laws import Law, Law2, Law3, LawSpecDef, law_definition
19
20_FirstType = TypeVar('_FirstType')
21_SecondType = TypeVar('_SecondType')
22_ThirdType = TypeVar('_ThirdType')
23
24_NewFirstType = TypeVar('_NewFirstType')
25_NewSecondType = TypeVar('_NewSecondType')
26
27_PairLikeKind = TypeVar('_PairLikeKind', bound='PairLikeN')
28
29
30class _LawSpec(LawSpecDef):
31 @law_definition
32 def pair_equality_law(
33 raw_value: _FirstType,
34 container: 'PairLikeN[_FirstType, _SecondType, _ThirdType]',
35 ) -> None:
36 """Ensures that unpaired and paired constructors work fine."""
37 assert_equal(
38 container.from_unpaired(raw_value),
39 container.from_paired(raw_value, raw_value),
40 )
41
42 @law_definition
43 def pair_left_identity_law(
44 pair: Tuple[_FirstType, _SecondType],
45 container: 'PairLikeN[_FirstType, _SecondType, _ThirdType]',
46 function: Callable[
47 [_FirstType, _SecondType],
48 KindN['PairLikeN', _NewFirstType, _NewSecondType, _ThirdType],
49 ],
50 ) -> None:
51 """Ensures that unpaired and paired constructors work fine."""
52 assert_equal(
53 container.from_paired(*pair).pair(function),
54 function(*pair),
55 )
56
57
58class PairLikeN(
59 bindable.BindableN[_FirstType, _SecondType, _ThirdType],
60 swappable.SwappableN[_FirstType, _SecondType, _ThirdType],
61 lashable.LashableN[_FirstType, _SecondType, _ThirdType],
62 equable.Equable,
63):
64 """Special interface for types that look like a ``Pair``."""
65
66 _laws: ClassVar[Sequence[Law]] = (
67 Law2(_LawSpec.pair_equality_law),
68 Law3(_LawSpec.pair_left_identity_law),
69 )
70
71 @abstractmethod
72 def pair(
73 self: _PairLikeKind,
74 function: Callable[
75 [_FirstType, _SecondType],
76 KindN[_PairLikeKind, _NewFirstType, _NewSecondType, _ThirdType],
77 ],
78 ) -> KindN[_PairLikeKind, _NewFirstType, _NewSecondType, _ThirdType]:
79 """Allows to work with both arguments at the same time."""
80
81 @classmethod
82 @abstractmethod
83 def from_paired(
84 cls: Type[_PairLikeKind],
85 first: _NewFirstType,
86 second: _NewSecondType,
87 ) -> KindN[_PairLikeKind, _NewFirstType, _NewSecondType, _ThirdType]:
88 """Allows to create a PairLikeN from just two values."""
89
90 @classmethod
91 @abstractmethod
92 def from_unpaired(
93 cls: Type[_PairLikeKind],
94 inner_value: _NewFirstType,
95 ) -> KindN[_PairLikeKind, _NewFirstType, _NewFirstType, _ThirdType]:
96 """Allows to create a PairLikeN from just a single object."""
97
98
99PairLike2 = PairLikeN[_FirstType, _SecondType, NoReturn]
100PairLike3 = PairLikeN[_FirstType, _SecondType, _ThirdType]
101
102
103@final
104class Pair(
105 BaseContainer,
106 SupportsKind2['Pair', _FirstType, _SecondType],
107 PairLike2[_FirstType, _SecondType],
108):
109 """
110 A type that represents a pair of something.
111
112 Like to coordinates ``(x, y)`` or two best friends.
113 Or a question and an answer.
114
115 """
116
117 def __init__(
118 self,
119 inner_value: Tuple[_FirstType, _SecondType],
120 ) -> None:
121 """Saves passed tuple as ``._inner_value`` inside this instance."""
122 super().__init__(inner_value)
123
124 # `Equable` part:
125
126 equals = container_equality # we already have this defined for all types
127
128 # `Mappable` part via `BiMappable`:
129
130 def map(
131 self,
132 function: Callable[[_FirstType], _NewFirstType],
133 ) -> 'Pair[_NewFirstType, _SecondType]':
134 """
135 Changes the first type with a pure function.
136
137 >>> assert Pair((1, 2)).map(str) == Pair(('1', 2))
138
139 """
140 return Pair((function(self._inner_value[0]), self._inner_value[1]))
141
142 # `BindableN` part:
143
144 def bind(
145 self,
146 function: Callable[
147 [_FirstType],
148 Kind2['Pair', _NewFirstType, _SecondType],
149 ],
150 ) -> 'Pair[_NewFirstType, _SecondType]':
151 """
152 Changes the first type with a function returning another ``Pair``.
153
154 >>> def bindable(first: int) -> Pair[str, str]:
155 ... return Pair((str(first), ''))
156
157 >>> assert Pair((1, 'b')).bind(bindable) == Pair(('1', ''))
158
159 """
160 return dekind(function(self._inner_value[0]))
161
162 # `AltableN` part via `BiMappableN`:
163
164 def alt(
165 self,
166 function: Callable[[_SecondType], _NewSecondType],
167 ) -> 'Pair[_FirstType, _NewSecondType]':
168 """
169 Changes the second type with a pure function.
170
171 >>> assert Pair((1, 2)).alt(str) == Pair((1, '2'))
172
173 """
174 return Pair((self._inner_value[0], function(self._inner_value[1])))
175
176 # `LashableN` part:
177
178 def lash(
179 self,
180 function: Callable[
181 [_SecondType],
182 Kind2['Pair', _FirstType, _NewSecondType],
183 ],
184 ) -> 'Pair[_FirstType, _NewSecondType]':
185 """
186 Changes the second type with a function returning ``Pair``.
187
188 >>> def lashable(second: int) -> Pair[str, str]:
189 ... return Pair(('', str(second)))
190
191 >>> assert Pair(('a', 2)).lash(lashable) == Pair(('', '2'))
192
193 """
194 return dekind(function(self._inner_value[1]))
195
196 # `SwappableN` part:
197
198 def swap(self) -> 'Pair[_SecondType, _FirstType]':
199 """
200 Swaps ``Pair`` elements.
201
202 >>> assert Pair((1, 2)).swap() == Pair((2, 1))
203
204 """
205 return Pair((self._inner_value[1], self._inner_value[0]))
206
207 # `PairLikeN` part:
208
209 def pair(
210 self,
211 function: Callable[
212 [_FirstType, _SecondType],
213 Kind2['Pair', _NewFirstType, _NewSecondType],
214 ],
215 ) -> 'Pair[_NewFirstType, _NewSecondType]':
216 """
217 Creates a new ``Pair`` from an existing one via a passed function.
218
219 >>> def min_max(first: int, second: int) -> Pair[int, int]:
220 ... return Pair((min(first, second), max(first, second)))
221
222 >>> assert Pair((2, 1)).pair(min_max) == Pair((1, 2))
223 >>> assert Pair((1, 2)).pair(min_max) == Pair((1, 2))
224
225 """
226 return dekind(function(self._inner_value[0], self._inner_value[1]))
227
228 @classmethod
229 def from_paired(
230 cls,
231 first: _NewFirstType,
232 second: _NewSecondType,
233 ) -> 'Pair[_NewFirstType, _NewSecondType]':
234 """
235 Creates a new pair from two values.
236
237 >>> assert Pair.from_paired(1, 2) == Pair((1, 2))
238
239 """
240 return Pair((first, second))
241
242 @classmethod
243 def from_unpaired(
244 cls,
245 inner_value: _NewFirstType,
246 ) -> 'Pair[_NewFirstType, _NewFirstType]':
247 """
248 Creates a new pair from a single value.
249
250 >>> assert Pair.from_unpaired(1) == Pair((1, 1))
251
252 """
253 return Pair((inner_value, inner_value))
254
255
256# Running hypothesis auto-generated tests:
257check_all_laws(Pair, use_init=True)
Note
You can find all type-tests here.
The next thing we want is to write a type-test!
What is a type-test? This is a special type of tests for your typing.
We run mypy
on top of tests and use snapshots to assert the result.
We recommend to use pytest-mypy-plugins. Read more about how to use it.
Let’s start with a simple test
to make sure our .pair
function works correctly:
Warning
Please, don’t use env:
property the way we do here.
We need it since we store our example in tests/
folder.
And we have to tell mypy
how to find it.
1- case: test_pair_type
2 disable_cache: false
3 env:
4 # We only need this because we store this example in `tests/`
5 # and not in our source code. Please, do not copy this line!
6 - MYPYPATH=./tests/test_examples/test_your_container
7
8 # TODO: remove this config after
9 # mypy/typeshed/stdlib/unittest/mock.pyi:120:
10 # error: Class cannot subclass "Any" (has type "Any")
11 # is fixed.
12 mypy_config:
13 disallow_subclassing_any = False
14 main: |
15 # Let's import our `Pair` type we defined earlier:
16 from test_pair4 import Pair
17
18 reveal_type(Pair)
19
20 def function(first: int, second: str) -> Pair[float, bool]:
21 ...
22
23 my_pair: Pair[int, str] = Pair.from_paired(1, 'a')
24 reveal_type(my_pair.pair(function))
25 out: |
26 main:4: note: Revealed type is "def [_FirstType, _SecondType] (inner_value: Tuple[_FirstType`1, _SecondType`2]) -> test_pair4.Pair[_FirstType`1, _SecondType`2]"
27 main:10: note: Revealed type is "test_pair4.Pair[builtins.float, builtins.bool]"
Ok, now, let’s try to raise an error by using it incorrectly:
1- case: test_pair_error
2 disable_cache: false
3 env:
4 # We only need this because we store this example in `tests/`
5 # and not in our source code. Please, do not copy this line!
6 - MYPYPATH=./tests/test_examples/test_your_container
7
8 # TODO: remove this config after
9 # mypy/typeshed/stdlib/unittest/mock.pyi:120:
10 # error: Class cannot subclass "Any" (has type "Any")
11 # is fixed.
12 mypy_config:
13 disallow_subclassing_any = False
14 main: |
15 # Let's import our `Pair` type we defined earlier:
16 from test_pair4 import Pair
17
18 # Oups! This function has first and second types swapped!
19 def function(first: str, second: int) -> Pair[float, bool]:
20 ...
21
22 my_pair = Pair.from_paired(1, 'a')
23 my_pair.pair(function) # this should and will error
24 out: |
25 main:9: error: Argument 1 to "pair" of "Pair" has incompatible type "Callable[[str, int], Pair[float, bool]]"; expected "Callable[[int, str], KindN[Pair[Any, Any], float, bool, Any]]" [arg-type]
The last (but not the least!) thing you need
to know is that you can reuse all code
we already have for this new Pair
type.
This is because of our Higher Kinded Types feature.
So, let’s say we want to use native map_()
pointfree function with our new Pair
type.
Let’s test that it will work correctly:
1- case: test_pair_map
2 disable_cache: false
3 env:
4 # We only need this because we store this example in `tests/`
5 # and not in our source code. Please, do not copy this line!
6 - MYPYPATH=./tests/test_examples/test_your_container
7
8 # TODO: remove this config after
9 # mypy/typeshed/stdlib/unittest/mock.pyi:120:
10 # error: Class cannot subclass "Any" (has type "Any")
11 # is fixed.
12 mypy_config:
13 disallow_subclassing_any = False
14 main: |
15 from test_pair4 import Pair
16 from returns.pointfree import map_
17
18 my_pair: Pair[int, int] = Pair.from_unpaired(1)
19 reveal_type(my_pair.map(str))
20 reveal_type(map_(str)(my_pair))
21 out: |
22 main:5: note: Revealed type is "test_pair4.Pair[builtins.str, builtins.int]"
23 main:6: note: Revealed type is "test_pair4.Pair[builtins.str, builtins.int]"
Yes, it works!
Now you have fully working, typed, documented, lawful, and tested primitive. You can build any other primitive you need for your business logic or infrastructure.