Make your functions return something meaningful, typed, and safe!

Features¶

Provides a bunch of primitives to write declarative business logic
Enforces better architecture
Fully typed with annotations and checked with mypy, PEP561 compatible
Pythonic and pleasant to write and to read (!)
Support functions and coroutines, framework agnostic

Installation¶

pip install returns

You might also want to configure mypy correctly and install our plugin to fix this existing issue:

# In setup.cfg or mypy.ini:
[mypy]
plugins =
  returns.contrib.mypy.decorator_plugin

Make sure you know how to get started, check out our docs!

Contents¶

Result container that let’s you to get rid of exceptions
IO marker that marks all impure operations and structures them
Maybe container that allows you to write None-free code

Result container¶

Please, make sure that you are also aware of Railway Oriented Programming.

Straight-forward approach¶

Consider this code that you can find in any python project.

import requests

def fetch_user_profile(user_id: int) -> 'UserProfile':
    """Fetches UserProfile dict from foreign API."""
    response = requests.get('/api/users/{0}'.format(user_id))
    response.raise_for_status()
    return response.json()

Seems legit, does not it? It also seems like a pretty straight forward code to test. All you need is to mock requests.get to return the structure you need.

But, there are hidden problems in this tiny code sample that are almost impossible to spot at the first glance.

Hidden problems¶

Let’s have a look at the exact same code, but with the all hidden problems explained.

import requests

def fetch_user_profile(user_id: int) -> 'UserProfile':
    """Fetches UserProfile dict from foreign API."""
    response = requests.get('/api/users/{0}'.format(user_id))

    # What if we try to find user that does not exist?
    # Or network will go down? Or the server will return 500?
    # In this case the next line will fail with an exception.
    # We need to handle all possible errors in this function
    # and do not return corrupt data to consumers.
    response.raise_for_status()

    # What if we have received invalid JSON?
    # Next line will raise an exception!
    return response.json()

Now, all (probably all?) problems are clear. How can we be sure that this function will be safe to use inside our complex business logic?

We really can not be sure! We will have to create lots of try and except cases just to catch the expected exceptions.

Our code will become complex and unreadable with all this mess!

Pipeline example¶

import requests
from returns.result import Result, pipeline, safe

class FetchUserProfile(object):
    """Single responsibility callable object that fetches user profile."""

    @pipeline
    def __call__(self, user_id: int) -> Result['UserProfile', Exception]:
        """Fetches UserProfile dict from foreign API."""
        response = self._make_request(user_id).unwrap()
        return self._parse_json(response)

    @safe
    def _make_request(self, user_id: int) -> requests.Response:
        response = requests.get('/api/users/{0}'.format(user_id))
        response.raise_for_status()
        return response

    @safe
    def _parse_json(self, response: requests.Response) -> 'UserProfile':
        return response.json()

Now we have a clean and a safe way to express our business need. We start from making a request, that might fail at any moment.

Now, instead of returning a regular value it returns a wrapped value inside a special container thanks to the @safe decorator.

It will return Success[Response] or Failure[Exception]. And will never throw this exception at us.

When we will need raw value, we can use .unwrap() method to get it. If the result is Failure[Exception] we will actually raise an exception at this point. But it is safe to use .unwrap() inside @pipeline functions. Because it will catch this exception and wrap it inside a new Failure[Exception]!

And we can clearly see all result patterns that might happen in this particular case:

Success[UserProfile]
Failure[HttpException]
Failure[JsonDecodeException]

And we can work with each of them precisely. It is a good practice to create Enum classes or Union types with a list of all the possible errors.

IO marker¶

But is that all we can improve? Let’s look at FetchUserProfile from another angle. All its methods look like regular ones: it is impossible to tell whether they are pure or impure from the first sight.

It leads to a very important consequence: we start to mix pure and impure code together. We should not do that!

When these two concepts are mixed we suffer really bad when testing or reusing it. Almost everything should be pure by default. And we should explicitly mark impure parts of the program.

Explicit IO¶

Let’s refactor it to make our IO explicit!

import requests
from returns.io import IO, impure
from returns.result import Result, pipeline, safe

class FetchUserProfile(object):
    """Single responsibility callable object that fetches user profile."""

    @pipeline
    def __call__(self, user_id: int) -> IO[Result['UserProfile', Exception]]:
        """Fetches UserProfile dict from foreign API."""
        return self._make_request(user_id).map(
            lambda response: self._parse_json(response.unwrap())
        )

    @impure
    @safe
    def _make_request(self, user_id: int) -> requests.Response:
        response = requests.get('/api/users/{0}'.format(user_id))
        response.raise_for_status()
        return response

    @safe
    def _parse_json(self,response: requests.Response) -> 'UserProfile':
        return response.json()

Now we have explicit markers where the IO did happen and these markers cannot be removed.

Whenever we access FetchUserProfile we now know that it does IO and might fail. So, we act accordingly!

Maybe container¶

None is called the worst mistake in the history of Computer Science.

So, what can we do? You can use Optional and write a lot of if some is not None conditions. But, having them here and there makes your code unreadable.

Or you can use Maybe container! It consists of Some and Nothing types, representing existing state and empty (instead of None) state respectively.

from typing import Optional
from returns.maybe import Maybe, maybe

@maybe
def bad_function() -> Optional[int]:
    ...

maybe_result: Maybe[float] = bad_function().map(
    lambda number: number / 2,
)
# => Maybe will return Some[float] only if there's a non-None value
#    Otherwise, will return Nothing

It follows the same composition rules as the Result type. You can be sure that .map() method won’t be called for Nothing. Forget about None-related errors forever!

More!¶

Want more? Go to the docs! Or read these articles:

Do you have an article to submit? Feel free to open a pull request!

Inspirations¶

This module is heavily based on:

Contents¶

Container: the concept¶

Container is a concept that allows you to write code around the existing wrapped values while maintaining the execution context.

List of supported containers:

IO to mark explicit IO actions
Result to handle possible exceptions
Maybe to handle None cases

We will show you container’s simple API of one attribute and several simple methods.

Basics¶

The main idea behind a container is that it wraps some internal state. That’s what ._inner_value is used for.

And we have several functions to create new containers based on the previous state. And we can see how this state is evolving during the execution.

graph LR F1["Container(Initial)"] --> F2["Container(UserId(1))"] F2 --> F3["Container(UserAccount(156))"] F3 --> F4["Container(FailedLoginAttempt(1))"] F4 --> F5["Container(SentNotificationId(992))"]

State evolution.¶

Working with containers¶

We use two methods to create a new container from the previous one. bind and map.

The difference is simple:

map works with functions that return regular value
bind works with functions that return new container of the same type

bind() is used to literally bind two different containers together.

from returns.result import Result, Success

def may_fail(user_id: int) -> Result[int, str]:
    ...

# Can be assumed as either Success[int] or Failure[str]:
result: Result[int, str] = Success(1).bind(may_fail)

And we have map() to use containers with regular functions.

from typing import Any
from returns.result import Success, Result

def double(state: int) -> int:
    return state * 2

result: Result[int, Any] = Success(1).map(double)
# => Success(2)
result: Result[int, Any] = result.map(lambda state: state + 1)
# => Success(3)

The same work with built-in functions as well:

from returns.io import IO

IO('bytes').map(list)
# => <IO: ['b', 'y', 't', 'e', 's']>

Note:

All containers support these methods.

Railway oriented programming¶

When talking about error handling we use a concept of Railway oriented programming. It mean that our code can go on two tracks:

Successful one: where everything goes perfectly: HTTP requests work, database is always serving us data, parsing values does not failed
Failed one: where something went wrong

We can switch from track to track: we can fail something or we can rescue the situation.

graph LR S1 -- Map --> S3 S3 --> S5 S5 --> S7 F2 -- Map Failure --> F4 F4 --> F6 F6 --> F8 S1 -- Fail --> F2 F2 -- Fix --> S3 S3 -- Fail --> F4 S5 -- Fail --> F6 F6 -- Rescue --> S7 style S1 fill:green style S3 fill:green style S5 fill:green style S7 fill:green style F2 fill:red style F4 fill:red style F6 fill:red style F8 fill:red

Railway oriented programming.¶

Returning execution to the right track¶

We also support two special methods to work with “failed” types like Failure:

rescue() is the opposite of bind method that works only when container is in failed state
fix() transforms error to value (failure became success) that works only when container is in failed state, is the opposite of map method
map_failure() transforms error to another error that works only when container is in failed state, is the opposite of map method

fix can be used to fix some fixable errors during the pipeline execution:

from returns.result import Failure, Result

def double(state: int) -> float:
    return state * 2.0

result: Result[Any, float] = Failure(1).map_failure(double)
# => Failure(2.0)

result: Result[float, int] = Failure(1).fix(double)
# => Success(2.0)

rescue should return one of Success or Failure types. It can also rescue your flow and get on the successful track again:

from returns.result import Result, Failure, Success

def tolerate_exception(state: Exception) -> Result[int, Exception]:
    if isinstance(state, ZeroDivisionError):
        return Success(0)
    return Failure(state)

result: Result[int, Exception] = Failure(
    ZeroDivisionError(),
).rescue(tolerate_exception)
# => Success(0)

result2: Result[int, Exception] = Failure(
    ValueError(),
).rescue(tolerate_exception)
# => Failure(ValueError())

Note:

Not all containers support these methods.
IO cannot be fixed or rescued.

Unwrapping values¶

And we have two more functions to unwrap inner state of containers into a regular types:

.value_or returns a value if it is possible, returns default_value otherwise
.unwrap returns a value if it is possible, raises UnwrapFailedError otherwise

from returns.result import Failure, Success
from returns.maybe import Some, Nothing

Success(1).value_or(None)
# => 1

Some(0).unwrap()
# => 0

Failure(1).value_or(default_value=100)
# => 100

Failure(1).unwrap()
# => Traceback (most recent call last): UnwrapFailedError

Nothing.unwrap()
# => Traceback (most recent call last): UnwrapFailedError

The most user-friendly way to use .unwrap() method is with pipeline. We even discourage using .unwrap() without a @pipeline.

For failing containers you can use .failure to unwrap the failed state:

Failure(1).failure()
# => 1

Success(1).failure()
# => Traceback (most recent call last): UnwrapFailedError

Be careful, since this method will raise an exception when you try to .failure() a successful container.

Note:

Not all containers support these methods.
IO cannot be unwrapped.

Immutability¶

We like to think of returns as immutable structures. You cannot mutate the inner state of the created container, because we redefine __setattr__ and __delattr__ magic methods.

You cannot also set new attributes to container instances, since we are using __slots__ for better performance and strictness.

Well, nothing is really immutable in python, but you were warned.

Type safety¶

We try to make our containers optionally type safe.

What does it mean?

It is still good old python, do whatever you want without mypy
If you are using mypy you will be notified about type violations

We also ship PEP561 compatible .pyi files together with the source code. In this case these types will be available to users when they install our application.

We also ship custom mypy plugins to overcome some existing problems, please make sure to use them, since they increase your developer experience and type-safety:

decorator_plugin to solve untyped decorator issue

[mypy]
plugins =
  returns.contrib.mypy.decorator_plugin

You can have a look at the suggested mypy configuration in our own repository.

Composition¶

You can and should compose different containers together. Here’s the full table of compositions that make sense:

IO[Result[A, B]] ✅
IO[Maybe[A]] ✅
IO[IO[A]] 🤔, use join
Maybe[Maybe[A]] 🤔, use join
Result[Result[A, B], C] 🤔, use join
Result[Maybe[A], B] 🤔,
use maybe_to_result
Maybe[Result[A, B]] 🤔,
use result_to_maybe
Result[IO[A], B] 🚫
Result[A, IO[A]] 🚫
Result[A, Maybe[B]] 🚫
Result[A, Result[B, C]] 🚫
Maybe[IO[A]] 🚫

You can use Converters to convert Maybe and Result containers. So, you don’t have to compose them.

You can also use join to merge nested containers.

Converters¶

We have several helper functions to convert containers from Maybe to Result and back again:

maybe_to_result that converts Maybe to Result
result_to_maybe that converts Result to Maybe

That’s how they work:

from returns.converters import maybe_to_result, result_to_maybe
from returns.maybe import Maybe
from returns.result import Result

result: Result[int, Exception]
maybe: Maybe[int] = result_to_maybe(result)
new_result: Result[int, None] = maybe_to_result(maybe)

Take a note, that type changes. Also, take a note that Success(None) will be converted to Nothing.

You can also use join to merge nested containers together:

from returns.converters import join
from returns.maybe import Maybe
from returns.result import Success
from returns.io import IO

assert join(IO(IO(1))) == IO(1)
assert Maybe(Maybe(1)) == Maybe(1)
assert Success(Success(1)) == Success(1)

API Reference¶

graph TD; Bindable Mappable BaseContainer Unwrapable UnwrapableFailure Fixable Rescueable Protocol --> Bindable Protocol --> Mappable Protocol --> Fixable Protocol --> Rescueable Protocol --> Unwrapable Protocol --> UnwrapableFailure

class BaseContainer(inner_value)[source]¶

Bases: object

Utility class to provide all needed magic methods to the context.

class Bindable(*args, **kwargs)[source]¶

Bases: typing_extensions.Protocol

Represents a “context” in which calculations can be executed.

Bindable allows you to bind together a series of calculations while maintaining the context of that specific container.

bind(function)[source]¶

Applies ‘function’ to the result of a previous calculation.

And returns a new container. Works for containers that represent success. Is the opposite of Rescueable.rescue().

Return type: Bindable[~_NewValueType]

class Mappable(*args, **kwargs)[source]¶

Bases: typing_extensions.Protocol

Allows to chain wrapped values with regular functions.

Behaves like functor.

map(function)[source]¶

Applies ‘function’ to the contents of the functor.

And returns a new functor value. Is the opposite of Fixable.fix().

Return type: Mappable[~_NewValueType]

class Fixable(*args, **kwargs)[source]¶

Bases: typing_extensions.Protocol

Represents containers that can be fixed and rescued.

fix(function)[source]¶

Applies ‘function’ to the error and transforms failure to success.

And returns a new functor value. Works for containers that represent failure. Is the opposite of Mappable.map().

Return type: Fixable[~_NewValueType, +_ErrorType]

class Rescueable(*args, **kwargs)[source]¶

Bases: typing_extensions.Protocol

Represents a “context” in which calculations can be executed.

Rescueable allows you to bind together a series of calculations while maintaining the context of that specific container.

rescue(function)[source]¶

Applies ‘function’ to the result of a previous calculation.

And returns a new container. Works for containers that represent failure. Is the opposite of bind().

Return type: Rescueable[~_NewValueType, ~_NewErrorType]

class Unwrapable(*args, **kwargs)[source]¶

Bases: typing_extensions.Protocol

Represents containers that can unwrap and return its wrapped value.

value_or(default_value)[source]¶

Forces to unwrap value from container or return a default.

Return type: Union[+_ValueType, ~_NewValueType]

unwrap()[source]¶

Custom magic method to unwrap inner value from container.

Should be redefined for ones that actually have values. And for ones that raise an exception for no values.

This method is the opposite of failure().

Return type: +_ValueType

class UnwrapableFailure(*args, **kwargs)[source]¶

Bases: typing_extensions.Protocol

Allows to unwrap failures.

map_failure(function)[source]¶

Uses ‘function’ to transform one error to another.

And returns a new functor value. Works for containers that represent failure. Is the opposite of map().

Return type: Fixable[+_ValueType, ~_NewErrorType]

failure()[source]¶

Custom magic method to unwrap inner value from the failed container.

This method is the opposite of unwrap().

Return type: +_ErrorType

result_to_maybe(result_container)[source]¶

Converts Result container to Maybe container.

Return type: Maybe[~_ValueType]

maybe_to_result(maybe_container)[source]¶

Converts Maybe container to Result container.

Return type: Result[~_ValueType]

join(container)[source]¶: Joins two nested containers together.

Result¶

Result is obviously a result of some series of computations. It might succeed with some resulting value. Or it might return an error with some extra details.

Result consist of two types: Success and Failure. Success represents successful operation result and Failure indicates that something has failed.

from returns.result import Result, Success, Failure

def find_user(user_id: int) -> Result['User', str]:
    user = User.objects.filter(id=user_id)
    if user.exists():
        return Success(user[0])
    return Failure('User was not found')

user_search_result = find_user(1)
# => Success(User{id: 1, ...})

user_search_result = find_user(0)  # id 0 does not exist!
# => Failure('User was not found')

When is it useful? When you do not want to use exceptions to break your execution scope. Or when you do not want to use None to represent empty values, since it will raise TypeError somewhere and other None exception-friends.

is_successful¶

is_succesful is used to tell whether or not your result is a success. We treat only treat types that does not throw as a successful ones, basically: Success.

from returns.result import Success, Failure, is_successful

is_successful(Success(1))
# => True

is_successful(Failure('text'))
# => False

pipeline¶

What is a pipeline? It is a more user-friendly syntax to work with containers that support both async and regular functions.

Consider this task. We were asked to create a method that will connect together a simple pipeline of three steps:

We validate passed username and email
We create a new Account with this data, if it does not exists
We create a new User associated with the Account

And we know that this pipeline can fail in several places:

Wrong username or email might be passed, so the validation will fail
Account with this username or email might already exist
User creation might fail as well, since it also makes an HTTP request to another micro-service deep inside

Here’s the code to illustrate the task.

from returns.result import Result, Success, Failure, pipeline


class CreateAccountAndUser(object):
    """Creates new Account-User pair."""

    # TODO: we need to create a pipeline of these methods somehow...

    # Protected methods

    def _validate_user(
        self, username: str, email: str,
    ) -> Result['UserSchema', str]:
        """Returns an UserSchema for valid input, otherwise a Failure."""

    def _create_account(
        self, user_schema: 'UserSchema',
    ) -> Result['Account', str]:
        """Creates an Account for valid UserSchema's. Or returns a Failure."""

    def _create_user(
        self, account: 'Account',
    ) -> Result['User', str]:
        """Create an User instance. If user already exists returns Failure."""

Using bind technique¶

We can implement this feature using a traditional bind method.

class CreateAccountAndUser(object):
    """Creates new Account-User pair."""

    def __call__(self, username: str, email: str) -> Result['User', str]:
        """Can return a Success(user) or Failure(str_reason)."""
        return self._validate_user(username, email).bind(
            self._create_account,
        ).bind(
            self._create_user,
        )

    # Protected methods
    # ...

And this will work without any problems. But, is it easy to read a code like this? No, it is not.

What alternative we can provide? @pipeline!

Using pipeline¶

And here’s how we can refactor previous version to be more clear.

class CreateAccountAndUser(object):
    """Creates new Account-User pair."""

    @pipeline
    def __call__(self, username: str, email: str) -> Result['User', str]:
        """Can return a Success(user) or Failure(str_reason)."""
        user_schema = self._validate_user(username, email).unwrap()
        account = self._create_account(user_schema).unwrap()
        return self._create_user(account)

    # Protected methods
    # ...

Let’s see how this new .unwrap() method works:

if you result is Success it will return its inner value
if your result is Failure it will raise a UnwrapFailedError

And that’s where @pipeline decorator becomes in handy. It will catch any UnwrapFailedError during the pipeline and then return a simple Failure result.

sequenceDiagram participant pipeline participant validation participant account creation participant user creation pipeline->>validation: runs the first step validation-->>pipeline: returns Failure(validation message) if fails validation->>account creation: passes Success(UserSchema) if valid account creation-->>pipeline: return Failure(account exists) if fails account creation->>user creation: passes Success(Account) if valid user creation-->>pipeline: returns Failure(http status) if fails user creation-->>pipeline: returns Success(user) if user is created

Pipeline execution.¶

See, do notation allows you to write simple yet powerful pipelines with multiple and complex steps. And at the same time the produced code is simple and readable.

And that’s it!

safe¶

safe is used to convert regular functions that can throw exceptions to functions that return Result type.

Supports both async and regular functions.

from returns.result import safe

@safe  # Will convert type to: Callable[[int], Result[float, Exception]]
def divide(number: int) -> float:
    return number / number

divide(1)
# => Success(1.0)

divide(0)
# => Failure(ZeroDivisionError)

Limitations¶

Typing will only work correctly if decorator_plugin is used. This happens due to mypy issue.

API Reference¶

graph TD; Result _Failure _Success Generic --> Result BaseContainer --> Result Result --> _Failure Result --> _Success

class Result(inner_value)[source]¶

Bases: typing.Generic, returns.primitives.container.BaseContainer

Base class for _Failure and _Success.

abstract map(function)[source]¶

Abstract method to compose container with pure function.

Return type: Result[~_NewValueType, ~_ErrorType]

abstract bind(function)[source]¶

Abstract method to compose container with other container.

Return type: Result[~_NewValueType, ~_NewErrorType]

abstract fix(function)[source]¶

Abstract method to compose container with pure function.

Return type: Result[~_NewValueType, ~_ErrorType]

abstract map_failure(function)[source]¶

Abstract method to compose container with pure function.

Return type: Result[~_ValueType, ~_NewErrorType]

abstract rescue(function)[source]¶

Abstract method to compose container with other container.

Return type: Result[~_NewValueType, ~_NewErrorType]

abstract value_or(default_value)[source]¶

Get value or default value.

Return type: Union[~_ValueType, ~_NewValueType]

abstract unwrap()[source]¶

Get value or raise exception.

Return type: ~_ValueType

abstract failure()[source]¶

Get failed value or raise exception.

Return type: ~_ErrorType

Success(inner_value)[source]¶

Public unit function of protected _Success type.

Return type: Result[~_ValueType]

Failure(inner_value)[source]¶

Public unit function of protected _Failure type.

Return type: Result[~_ErrorType]

safe(function)[source]¶

Decorator to covert exception throwing function to ‘Result’ container.

Should be used with care, since it only catches ‘Exception’ subclasses. It does not catch ‘BaseException’ subclasses.

Supports both async and regular functions.

Maybe¶

The Maybe container is used when a series of computations could return None at any point.

Maybe container¶

Maybe consist of two types: Some and Nothing. We have a convenient method to create different Maybe types based on just a single value:

from returns.maybe import Maybe

Maybe.new(1)
# => Some(1)

Maybe.new(None)
# => Nothing

Usage¶

It might be very useful for complex operations like the following one:

from dataclasses import dataclass
from typing import Optional

@dataclass
class Address(object):
    street: Optional[str]

@dataclass
class User(object):
    address: Optional[Address]

@dataclass
class Order(object):
    user: Optional[User]

order: Order  # some existing Order instance
street: Maybe[str] = Maybe.new(order.user).map(
    lambda user: user.address,
).map(
    lambda address: address.street,
)
# => `Some('address street info')` if all fields are not None
# => `Nothing` if at least one field is `None`

Optional type¶

One may ask: “How is that different to the Optional[] type?” That’s a really good question!

Consider the same code to get the street name without Maybe and using raw Optional values:

order: Order  # some existing Order instance
street: Optional[str] = None
if order.user is not None:
    if order.user.address is not None:
        street = order.user.address.street

It looks way uglier and can grow even more uglier and complex when new logic will be introduced.

@maybe decorator¶

Sometimes we have to deal with functions that dears to return Optional values!

We have to work with it the carefully and write if x is not None: everywhere. Luckily, we have your back! maybe function decorates any other function that returns Optional and converts it to return Maybe instead:

from typing import Optional
from returns.maybe import Maybe, maybe

@maybe
def number(num: int) -> Optional[int]:
    if num > 0:
        return num
    return None

result: Maybe[int] = number(1)
# => Some(1)

API Reference¶

graph TD;

class Maybe(inner_value)[source]¶

Bases: typing.Generic, returns.primitives.container.BaseContainer

Represents a result of a series of commutation that can return None.

An alternative to using exceptions or constant is None checks. Maybe is an abstract type and should not be instantiated directly. Instead use Some and Nothing.

classmethod new(inner_value)[source]¶

Creates new instance of Maybe container based on a value.

Return type: Maybe[~_ValueType]

abstract map(function)[source]¶

Abstract method to compose container with a pure function.

Return type: Maybe[~_NewValueType]

abstract bind(function)[source]¶

Abstract method to compose container with other container.

Return type: Maybe[~_NewValueType]

abstract fix(function)[source]¶

Abstract method to compose container with a pure function.

Return type: Maybe[~_NewValueType]

abstract rescue(function)[source]¶

Abstract method to compose container with other container.

Return type: Maybe[~_NewValueType]

abstract value_or(default_value)[source]¶

Get value or default value.

Return type: Union[~_ValueType, ~_NewValueType]

abstract unwrap()[source]¶

Get value or raise exception.

Return type: ~_ValueType

Some(inner_value)[source]¶

Public unit function of protected _Some type.

Return type: Maybe[~_ValueType]

Nothing = <returns.maybe._Nothing object>¶: Public unit value of protected _Nothing type.

maybe(function)[source]¶

Decorator to covert None returning function to Maybe container.

Supports both async and regular functions.

IO¶

IO is ugly.

Why? Let me illustrate it with the example.

IO marker¶

Imagine we have this beautiful pure function:

def can_book_seats(
    number_of_seats: int,
    reservation: 'Reservation',
) -> bool:
    return reservation.capacity >= number_of_seats + reservation.booked

What’s good about it? We can test it easily. Even without setting up any testing framework, simple doctests will be enough.

This code is beautiful, because it is simple.

We can later use its result to notify users about their booking request:

def notify_user_about_booking_result(is_successful: bool) -> 'MessageID':
    ...

notify_user_about_booking_result(is_successful)  # works just fine!

Impure functions¶

But, imagine that our requirements had changed. And now we have to grab the number of already booked tickets from some other provider and fetch the maximum capacity from the database:

import requests
import db

def can_book_seats(
    number_of_seats: int,
    place_id: int,
) -> bool:
    capacity = db.get_place_capacity(place_id)  # sql query
    booked = requests('https://partner.com/api').json()['booked']  # http req
    return capacity >= number_of_seats + booked

Now testing this code will become a nightmare! It will require to setup:

real database and tables
fixture data
requests mocks for different outcomes
and the whole Universe!

Our complexity has sky-rocketed! And the most annoying part is that all other functions that call can_book_seats now also have to do the same setup. It seams like IO is indelible mark (some people also call it “effect”).

And at some point it time we will start to mix pure and impure code together.

Separating two worlds¶

Well, our IO mark is indeed indelible and should be respected.

Once you have an IO operation you can mark it appropriately. And it infects all other functions that call it. And impurity becomes explicit:

import requests
import db
from returns.io import IO

def can_book_seats(
    number_of_seats: int,
    place_id: int,
) -> IO[bool]:
    capacity = db.get_place_capacity(place_id)  # sql query
    booked = requests('https://partner.com/api').json()['booked']
    return IO(capacity >= number_of_seats + booked)

Now this function returns IO[bool] instead of a regular bool. It means, that it cannot be used where regular bool can be:

def notify_user_about_booking_result(is_successful: bool) -> 'MessageID':
    ...

is_successful: IO[bool] = can_book_seats(number_of_seats, place_id)
notify_user_about_booking_result(is_successful)  # Boom!
# => Argument 1 has incompatible type "IO[bool]"; expected "bool"

See? It is now impossible for a pure function to use IO[bool]. It is impossible to unwrap or get a value from this container. Once it is marked as IO it will never return to the pure state.

Well, there’s a hack actually: unsafe_perform_io

It also needs to be explicitly mapped to produce new IO result:

message_id: IO['MessageID'] = can_book_seats(
    number_of_seats,
    place_id,
).map(
    notify_user_about_booking_result,
)

Or it can be annotated to work with impure results:

def notify_user_about_booking_result(
    is_successful: IO[bool],
) -> IO['MessageID']:
    ...

is_successful: IO[bool] = can_book_seats(number_of_seats, place_id)
notify_user_about_booking_result(is_successful)  # Works!

Now, all our impurity is explicit. We can track it, we can fight it, we can design it better. By saying that, it is assumed that you have a functional core and imperative shell.

impure¶

We also have this handy decorator to help you with the existing impure things in Python:

from returns.io import impure

name: IO[str] = impure(input)('What is your name?')

You can also decorate your own functions with @impure for better readability and clearness:

import requests
from returns.io import impure

@impure
def get_user() -> 'User':
    return requests.get('https:...').json()

Limitations¶

Typing will only work correctly if decorator_plugin is used. This happens due to mypy issue.

FAQ¶

What is the difference between IO[T] and T?¶

What kind of input parameter should my function accept IO[T] or simple T?

It really depends on your domain / context. If the value is pure, than use raw unwrapped values. If the value is fetched, input, received, selected, than use IO container.

Most web applications are just covered with IO.

Why IO should be at the top level of composition?¶

As we state in Composition docs we allow to compose different containers together.

We prefer IO[Result[A, B]] and sticking to the single version allows better composition. The same rule is applied to Maybe and all other containers we have.

Composing IO at the top level is easier because you can join things easily.

And other containers not always make sense. If some operation performs IO it should mark all internals.

Why can’t we unwrap values or use @pipeline with IO?¶

Our design decision was not let people unwrap IO containers, so it will indeed infect the whole call-stack with its effect.

Otherwise, people might hack the system in some dirty (from our point of view) but valid (from the python’s point of view) ways.

Warning:

Of course, you can directly access
the internal state of the IO with `._internal_state`,
but your are considered to be a grown-up!

Use wemake-python-styleguide to restrict `._` access in your code.

API Reference¶

graph TD; IO Generic --> IO BaseContainer --> IO

class IO(inner_value)[source]¶

Bases: typing.Generic, returns.primitives.container.BaseContainer

Explicit marker for impure function results.

We call it “marker” since once it is marked, it cannot be unmarked.

IO is also a container. But, it is different in a way that it cannot be unwrapped / rescued / fixed. There’s no way to directly get its internal value.

map(function)[source]¶

Applies function to the inner value.

Applies ‘function’ to the contents of the IO instance and returns a new IO object containing the result. ‘function’ should accept a single “normal” (non-container) argument and return a non-container result.

Return type: IO[~_NewValueType]

bind(function)[source]¶

Applies ‘function’ to the result of a previous calculation.

‘function’ should accept a single “normal” (non-container) argument and return IO type object.

Return type: IO[~_NewValueType]

impure(function)[source]¶

Decorator to mark function that it returns IO container.

Supports both async and regular functions.

Unsafe operations¶

This doc describes compatibility functions and escape mechanisms that we consider unsafe to your types and code-base.

Use them with great responsibility!

unsafe_perform_io¶

Sometimes you really need to get the raw value from IO container. For example:

def index_view(request, user_id):
    user: IO[User] = get_user(user_id)
    return render('index.html', { user: user })  # ???

In this case your web-framework will not render your user correctly. Since it does not expect it to be wrapped inside IO containers. And we obviously cannot map or bind this function.

What to do? Use unsafe_perform_io:

from returns.unsafe import unsafe_perform_io

def index_view(request, user_id):
    user: IO[User] = get_user(user_id)
    return render('index.html', { user: unsafe_perform_io(user) })  # Ok

We need it as an escape and compatibility mechanism for our imperative shell.

It is recommended to use import-linter to restrict imports from returns.unsafe expect the top-level modules.

Inspired by Haskell’s unsafePerformIO

API Reference¶

unsafe_perform_io(wrapped_in_io)[source]¶

Compatibility utility and escape mechanism from IO world.

Just unwraps the internal value from IO container. Should be used with caution! Since it might be overused by tired developers.

It is recommended to have only one place (module / file) in your program where you allow unsafe operations.

We recommend to use import-linter to enforce this rule:

https://github.com/seddonym/import-linter

Return type: ~_ValueType

Helper functions¶

We feature several helper functions to make your developer experience better.

compose¶

We also ship an utility function to compose two different functions together.

from returns.functions import compose

bool_after_int = compose(int, bool)
bool_after_int('1')  # => True
bool_after_int('0')  # => False

Composition is also type-safe. The only limitation is that we only support functions with one argument and one return to be composed.

Only works with regular functions (not async).

raise_exception¶

Sometimes you really want to reraise an exception from Failure[Exception] due to some existing API (or a dirty hack).

We allow you to do that with ease!

from returns.functions import raise_exception

class CreateAccountAndUser(object):
    """Creates new Account-User pair."""

    @pipeline
    def __call__(self, username: str) -> ...:
        """Imagine, that you need to reraise ValidationErrors due to API."""
        return self._validate_user(
            username,
            # TODO: change in #84 to `.map_failure()`
        ).fix(
            # What happens here is interesting, since you do not let your
            # unwrap to fail with UnwrapFailedError, but instead
            # allows you to reraise a wrapped exception.
            # In this case `ValidationError()` will be thrown
            # before `UnwrapFailedError`
            raise_exception,
        )

    def _validate_user(
        self, username: str,
    ) -> Result['User', ValidationError]:
        ...

Use this with caution. We try to remove exceptions from our code base. Original proposal is here.

API Reference¶

compose(first, second)[source]¶

Allows function composition.

Works as: second . first You can read it as “second after first”.

from returns.functions import compose

logged_int = compose(int, print)('123')
# => returns: 123
# => prints: 123

We can only compose functions with one argument and one return. Type checked.

Return type: Callable[[~_FirstType], ~_ThirdType]

raise_exception(exception)[source]¶

Helper function to raise exceptions as a function.

It might be required as a compatibility tool for existing APIs.

That’s how it can be used:

from returns.functions import raise_exception

# Some operation result:
user: Failure[UserDoesNotExistError]

# Here we unwrap internal exception and raise it:
user.fix(raise_exception)

See: https://github.com/dry-python/returns/issues/56

Return type: _Noreturn

Version history¶

We follow Semantic Versions since the 0.1.0 release.

0.9.0¶

Features¶

Provides a bunch of primitive interfaces to write your own containers
Adds .map_failure() method
Adds join() function to join nested containers

Bugfixes¶

Fixes type of Maybe.fix and Maybe.rescue to work with both lambda: 1 and lambda _: 1

Misc¶

Improves README

0.8.0¶

Features¶

Reintroduces the Maybe container, typed!
Introduces converters from one type to another
Adds mypy plugin to type decorators
Complete rewrite of Result types
Partial API change, now Success and Failure are not types, but functions
New internal types introduced: FixableContainer and ValueUnwrapContainer

Bugfixes¶

Fixes issue when you could return IO container from Result.bind
Fixes @pipeline return type

Misc¶

Reapplied all types to .py files
Improved docs about IO and Container concept
Adds docs about container composition
Moves from Alpha to Beta

0.7.0¶

Features¶

Adds IO marker
Adds unsafe module with unsafe functions
Changes how functions are located inside the project

Bugfixes¶

Fixes container type in @pipeline
Now is_successful is public
Now raise_exception is public

Misc¶

Changes how str() function works for container types
Total rename to “container” in the source code

Version 0.6.0¶

Features¶

safe and pipeline now supports asyncio
is_successful now returns Literal types if possible

Version 0.5.0¶

Features¶

Adds compose helper function
Adds public API to import returns
Adds raise_exception helper function
Adds full traceback to .unwrap()

Misc¶

Updates multiple dev-dependencies, including mypy
Now search in the docs is working again
Relicenses this project to BSD
Fixes copyright notice in the docs

Version 0.4.0 aka Goodbye, containers!¶

Features¶

Moves all types to .pyi files
Renames all classes according to new naming pattern
HUGE improvement of types
Renames fmap to map
Renames do_notation to pipeline, moves it to functions.py
Renames ebind to rescue
Renames efmap to fix
Renames container to Container
Removes Maybe container, since typing does not have NonNullable type

Version 0.3.1¶

Bugfixes¶

Adds py.typed file to be PEP561 compatible

Version 0.3.0, Renamed to `returns`¶

The project is renamed to returns and moved to dry-python org.

Features¶

Adds .pyi files for all modules, to enable mypy support for 3rd party users

Version 0.2.0¶

Features¶

Adds Maybe container
Adds immutability and __slots__ to all containers
Adds methods to work with failures
Adds safe decorator to convert exceptions to Result container
Adds is_successful() function to detect if your result is a success
Adds failure() method to unwrap values from failed containers

Bugfixes¶

Changes the type of .bind method for Success container
Changes how equality works, so now Failure(1) != Success(1)
Changes how new instances created on unused methods

Misc¶

Improves docs

Version 0.1.1¶

Bugfixes¶

Changes how PyPI renders package’s page

Misc¶

Improves README with new badges and installation steps

Version 0.1.0¶

Initial release. Featuring only Result and do_notation.

Features¶

Installation¶

Contents¶

Result container¶

Straight-forward approach¶

Hidden problems¶

Pipeline example¶

IO marker¶

Explicit IO¶

Maybe container¶

More!¶

Inspirations¶

Contents¶

Container: the concept¶

Basics¶

Working with containers¶

Railway oriented programming¶

Returning execution to the right track¶

Unwrapping values¶

Immutability¶

Type safety¶

Composition¶

Converters¶

API Reference¶

Result¶

is_successful¶

pipeline¶

Using bind technique¶

Using pipeline¶

safe¶

Limitations¶

API Reference¶

Maybe¶

Maybe container¶

Usage¶

Optional type¶

@maybe decorator¶

API Reference¶

IO¶

IO marker¶

Impure functions¶

Separating two worlds¶

impure¶

Limitations¶

FAQ¶

What is the difference between IO[T] and T?¶

Why IO should be at the top level of composition?¶

Why can’t we unwrap values or use @pipeline with IO?¶

Further reading¶

API Reference¶

Unsafe operations¶

unsafe_perform_io¶

API Reference¶

Helper functions¶

compose¶

raise_exception¶

API Reference¶

Version history¶

0.9.0¶

Features¶

Bugfixes¶

Misc¶

0.8.0¶

Features¶

Bugfixes¶

Misc¶

0.7.0¶

Features¶

Bugfixes¶

Misc¶

Version 0.6.0¶

Features¶

Version 0.5.0¶

Features¶

Misc¶

Version 0.4.0 aka Goodbye, containers!¶

Features¶

Version 0.3.1¶

Bugfixes¶

Version 0.3.0, Renamed to returns¶

Version 0.3.0, Renamed to `returns`¶