Make your functions return something meaningful, typed, and safe!

Features¶

Provides a bunch of primitives to write declarative business logic
Enforces Railway Oriented Programming
Fully typed with annotations and checked with mypy, PEP561 compatible
Pythonic and pleasant to write and to read (!)
Support functions and coroutines, framework agnostic

Installation¶

pip install returns

Make sure you know how to get started, check out our docs!

Why?¶

Consider this code that you can find in any python project.

Straight-forward approach¶

import requests

def fetch_user_profile(user_id: int) -> 'UserProfile':
    """Fetches UserProfile dict from foreign API."""
    response = requests.get('/api/users/{0}'.format(user_id))
    response.raise_for_status()
    return response.json()

Seems legit, does not it? It also seems like a pretty straight forward code to test. All you need is to mock requests.get to return the structure you need.

But, there are hidden problems in this tiny code sample that are almost impossible to spot at the first glance.

Hidden problems¶

Let’s have a look at the exact same code, but with the all hidden problems explained.

import requests

def fetch_user_profile(user_id: int) -> 'UserProfile':
    """Fetches UserProfile dict from foreign API."""
    response = requests.get('/api/users/{0}'.format(user_id))

    # What if we try to find user that does not exist?
    # Or network will go down? Or the server will return 500?
    # In this case the next line will fail with an exception.
    # We need to handle all possible errors in this function
    # and do not return corrupt data to consumers.
    response.raise_for_status()

    # What if we have received invalid JSON?
    # Next line will raise an exception!
    return response.json()

Now, all (probably all?) problems are clear. How can we be sure that this function will be safe to use inside our complex business logic?

We really can not be sure! We will have to create lots of try and except cases just to catch the expected exceptions.

Our code will become complex and unreadable with all this mess!

Pipeline example¶

import requests
from returns.functions import pipeline, safe
from returns.result import Result

class FetchUserProfile(object):
    """Single responsibility callable object that fetches user profile."""

    #: You can later use dependency injection to replace `requests`
    #: with any other http library (or even a custom service).
    _http = requests

    @pipeline
    def __call__(self, user_id: int) -> Result['UserProfile', Exception]:
        """Fetches UserProfile dict from foreign API."""
        response = self._make_request(user_id).unwrap()
        return self._parse_json(response)

    @safe
    def _make_request(self, user_id: int) -> requests.Response:
        response = self._http.get('/api/users/{0}'.format(user_id))
        response.raise_for_status()
        return response

    @safe
    def _parse_json(self, response: requests.Response) -> 'UserProfile':
        return response.json()

Now we have a clean and a safe way to express our business need. We start from making a request, that might fail at any moment.

Now, instead of returning a regular value it returns a wrapped value inside a special container thanks to the @safe decorator.

It will return Success[Response] or Failure[Exception]. And will never throw this exception at us.

When we will need raw value, we can use .unwrap() method to get it. If the result is Failure[Exception] we will actually raise an exception at this point. But it is safe to use .unwrap() inside @pipeline functions. Because it will catch this exception and wrap it inside a new Failure[Exception]!

And we can clearly see all result patterns that might happen in this particular case:

Success[UserProfile]
Failure[HttpException]
Failure[JsonDecodeException]

And we can work with each of them precisely.

What more? Go to the docs!

License¶

MIT.

Inspirations¶

This module is heavily based on:

Contents¶

Container: the concept¶

Container is a concept that allows you to write code without traditional error handling while maintaining the execution context.

We will show you its simple API of one attribute and several simple methods.

Basics¶

The main idea behind a container is that it wraps some internal state. That’s what _inner_value is used for.

And we have several functions to create new containers based on the previous state. And we can see how this state is evolving during the execution.

graph LR F1["Container(Initial)"] --> F2["Container(UserId(1))"] F2 --> F3["Container(UserAccount(156))"] F3 --> F4["Container(FailedLoginAttempt(1))"] F4 --> F5["Container(SentNotificationId(992))"]

State evolution.¶

Railway oriented programming¶

We use a concept of Railway oriented programming. It mean that our code can go on two tracks:

Successful one: where everything goes perfectly: HTTP requests work, database is always serving us data, parsing values does not failed
Failed one: where something went wrong

We can switch from track to track: we can fail something or we can rescue the situation.

graph LR S1 --> S3 S3 --> S5 S5 --> S7 F2 --> F4 F4 --> F6 F6 --> F8 S1 -- Fail --> F2 F2 -- Fix --> S3 S3 -- Fail --> F4 S5 -- Fail --> F6 F6 -- Rescue --> S7 style S1 fill:green style S3 fill:green style S5 fill:green style S7 fill:green style F2 fill:red style F4 fill:red style F6 fill:red style F8 fill:red

Railway oriented programming.¶

Working with containers¶

We use two methods to create new containers from the previous one. bind and map.

The difference is simple:

map works with functions that return regular values
bind works with functions that return other containers

Container.bind is used to literally bind two different containers together.

from returns.result import Result, Success

def make_http_call(user_id: int) -> Result[int, str]:
    ...

result = Success(1).bind(make_http_call)
# => Will be equal to either Success[int] or Failure[str]

So, the rule is: whenever you have some impure functions, it should return a container type instead.

And we use Container.map to use containers with pure functions.

from returns.result import Success

def double(state: int) -> int:
    return state * 2

result = Success(1).map(double)
# => Will be equal to Success(2)

Returning execution to the right track¶

We also support two special methods to work with “failed” types like Failure and Nothing:

Container.fix is the opposite of map method that works only when container is in failed state
Container.rescue is the opposite of bind method that works only when container is in failed state

fix can be used to fix some fixable errors during the pipeline execution:

from returns.result import Failure

def double(state: int) -> float:
    return state * 2.0

Failure(1).fix(double)
# => Will be equal to Success(2.0)

rescue can return any container type you want. It can also fix your flow and get on the successful track again:

from returns.result import Result, Failure, Success

def fix(state: Exception) -> Result[int, Exception]:
    if isinstance(state, ZeroDivisionError):
        return Success(0)
    return Failure(state)

Failure(ZeroDivisionError).rescue(fix)
# => Will be equal to Success(0)

Unwrapping values¶

And we have two more functions to unwrap inner state of containers into a regular types:

Container.value_or returns a value if it is possible, returns default_value otherwise
Container.unwrap returns a value if it is possible, raises UnwrapFailedError otherwise

from returns.result import Failure, Success

Success(1).value_or(None)
# => 1

Success(0).unwrap()
# => 0

Failure(1).value_or(default_value=100)
# => 100

Failure(1).unwrap()
# => Traceback (most recent call last): UnwrapFailedError

The most user-friendly way to use unwrap method is with pipeline.

For failing containers you can use Container.failure to unwrap the failed state:

Failure(1).failure()
# => 1

Success(1).failure()
# => Traceback (most recent call last): UnwrapFailedError

Be careful, since this method will raise an exception when you try to failure a successful container.

Immutability¶

We like to think of returns as immutable structures. You cannot mutate the inner state of the created container, because we redefine __setattr__ and __delattr__ magic methods.

You cannot also set new attributes to container instances, since we are using __slots__ for better performance and strictness.

Well, nothing is really immutable in python, but you were warned.

Type safety¶

We try to make our containers optionally type safe.

What does it mean?

It is still good old python, do whatever you want without mypy
If you are using mypy you will be notified about type violations

We also ship PEP561 compatible .pyi files together with the source code. In this case these types will be available to users when they install our application.

However, this is still good old python type system, and it has its drawbacks.

API Reference¶

graph TD; GenericContainerTwoSlots GenericContainerOneSlot Container _BaseContainer _BaseContainer --> Container Generic --> GenericContainerOneSlot Container --> GenericContainerOneSlot Generic --> GenericContainerTwoSlots Container --> GenericContainerTwoSlots

class Container(inner_value)[source]¶

Bases: returns.primitives.container._BaseContainer

Represents a “context” in which calculations can be executed.

You won’t create ‘Container’ instances directly. Instead, sub-classes implement specific contexts. Monads allow you to bind together a series of calculations while maintaining the context of that specific monad.

This is an abstract class with the API declaration.

_inner_value¶: Wrapped internal immutable state.

abstract map(function)[source]¶

Applies ‘function’ to the contents of the functor.

And returns a new functor value. Works for monads that represent success. Is the opposite of fix().

abstract bind(function)[source]¶

Applies ‘function’ to the result of a previous calculation.

And returns a new monad. Works for monads that represent success. Is the opposite of rescue().

abstract fix(function)[source]¶

Applies ‘function’ to the contents of the functor.

And returns a new functor value. Works for monads that represent failure. Is the opposite of map().

abstract rescue(function)[source]¶

Applies ‘function’ to the result of a previous calculation.

And returns a new monad. Works for monads that represent failure. Is the opposite of bind().

abstract value_or(default_value)[source]¶: Forces to unwrap value from monad or return a default.

abstract unwrap()[source]¶

Custom magic method to unwrap inner value from monad.

Should be redefined for ones that actually have values. And for ones that raise an exception for no values.

This method is the opposite of failure().

abstract failure()[source]¶

Custom magic method to unwrap inner value from the failed monad.

This method is the opposite of unwrap().

class GenericContainerOneSlot(inner_value)[source]¶

Bases: typing.Generic, returns.primitives.container.Container

Base class for monads with one typed slot.

Use this type for generic inheritance only. Use Container as a general type for polymorphism.

class GenericContainerTwoSlots(inner_value)[source]¶

Bases: typing.Generic, returns.primitives.container.Container

Base class for monads with two typed slot.

Use this type for generic inheritance only. Use Container as a general type for polymorphism.

Result¶

Result is obviously a result of some series of computations. It might succeed with some resulting value. Or it might return an error with some extra details.

Result consist of two types: Success and Failure. Success represents successful operation result and Failure indicates that something has failed.

from returns.result import Result, Success, Failure

def find_user(user_id: int) -> Result['User', str]:
    user = User.objects.filter(id=user_id)
    if user.exists():
        return Success(user[0])
    return Failure('User was not found')

user_search_result = find_user(1)
# => Success(User{id: 1, ...})

user_search_result = find_user(0)  # id 0 does not exist!
# => Failure('User was not found')

When is it useful? When you do not want to use exceptions to break your execution scope. Or when you do not want to use None to represent empty values, since it will raise TypeError somewhere and other None exception-friends.

API Reference¶

graph TD; Failure Result Success GenericContainerTwoSlots --> Result Result --> Failure Result --> Success

class Result(inner_value)[source]¶

Bases: returns.primitives.container.GenericContainerTwoSlots

Base class for Failure and Success.

class Failure(inner_value)[source]¶

Bases: returns.result.Result

Represents a calculation which has failed.

It should contain an error code or message. To help with readability you may alternatively use the alias ‘Failure’.

map(function)[source]¶: Returns the ‘Failure’ instance that was used to call the method.

bind(function)[source]¶: Returns the ‘Failure’ instance that was used to call the method.

fix(function)[source]¶

Applies function to the inner value.

Applies ‘function’ to the contents of the ‘Success’ instance and returns a new ‘Success’ object containing the result. ‘function’ should accept a single “normal” (non-monad) argument and return a non-monad result.

rescue(function)[source]¶

Applies ‘function’ to the result of a previous calculation.

‘function’ should accept a single “normal” (non-monad) argument and return Result a ‘Failure’ or ‘Success’ type object.

value_or(default_value)[source]¶: Returns the value if we deal with ‘Success’ or default otherwise.

unwrap()[source]¶: Raises an exception, since it does not have a value inside.

failure()[source]¶: Unwraps inner error value from failed monad.

class Success(inner_value)[source]¶

Bases: returns.result.Result

Represents a calculation which has succeeded and contains the result.

To help with readability you may alternatively use the alias ‘Success’.

map(function)[source]¶

Applies function to the inner value.

bind(function)[source]¶

Applies ‘function’ to the result of a previous calculation.

‘function’ should accept a single “normal” (non-monad) argument and return Result a ‘Failure’ or ‘Success’ type object.

fix(function)[source]¶: Returns the ‘Success’ instance that was used to call the method.

rescue(function)[source]¶: Returns the ‘Success’ instance that was used to call the method.

value_or(default_value)[source]¶: Returns the value if we deal with ‘Success’ or default otherwise.

unwrap()[source]¶: Returns the unwrapped value from the inside of this monad.

failure()[source]¶: Raises an exception, since it does not have an error inside.

Helper functions¶

We feature several helper functions to make your developer experience better.

is_successful¶

is_succesful is used to tell whether or not your result is a success. We treat only treat types that does not throw as a successful ones, basically: Success.

from returns.result import Success, Failure
from returns.functions import is_successful

is_successful(Success(1))
# => True

is_successful(Failure('text'))
# => False

pipeline¶

What is a pipeline? It is a more user-friendly syntax to work with containers that support both async and regular functions.

Consider this task. We were asked to create a method that will connect together a simple pipeline of three steps:

We validate passed username and email
We create a new Account with this data, if it does not exists
We create a new User associated with the Account

And we know that this pipeline can fail in several places:

Wrong username or email might be passed, so the validation will fail
Account with this username or email might already exist
User creation might fail as well, since it also makes an HTTP request to another micro-service deep inside

Here’s the code to illustrate the task.

from returns.functions import pipeline
from returns.result import Result, Success, Failure


class CreateAccountAndUser(object):
    """Creates new Account-User pair."""

    # TODO: we need to create a pipeline of these methods somehow...

    # Protected methods

    def _validate_user(
        self, username: str, email: str,
    ) -> Result['UserSchema', str]:
        """Returns an UserSchema for valid input, otherwise a Failure."""

    def _create_account(
        self, user_schema: 'UserSchema',
    ) -> Result['Account', str]:
        """Creates an Account for valid UserSchema's. Or returns a Failure."""

    def _create_user(
        self, account: 'Account',
    ) -> Result['User', str]:
        """Create an User instance. If user already exists returns Failure."""

Using bind technique¶

We can implement this feature using a traditional bind method.

class CreateAccountAndUser(object):
    """Creates new Account-User pair."""

    def __call__(self, username: str, email: str) -> Result['User', str]:
        """Can return a Success(user) or Failure(str_reason)."""
        return self._validate_user(username, email).bind(
            self._create_account,
        ).bind(
            self._create_user,
        )

    # Protected methods
    # ...

And this will work without any problems. But, is it easy to read a code like this? No, it is not.

What alternative we can provide? @pipeline!

Using pipeline¶

And here’s how we can refactor previous version to be more clear.

class CreateAccountAndUser(object):
    """Creates new Account-User pair."""

    @pipeline
    def __call__(self, username: str, email: str) -> Result['User', str]:
        """Can return a Success(user) or Failure(str_reason)."""
        user_schema = self._validate_user(username, email).unwrap()
        account = self._create_account(user_schema).unwrap()
        return self._create_user(account)

    # Protected methods
    # ...

Let’s see how this new .unwrap() method works:

if you result is Success it will return its inner value
if your result is Failure it will raise a UnwrapFailedError

And that’s where @pipeline decorator becomes in handy. It will catch any UnwrapFailedError during the pipeline and then return a simple Failure result.

sequenceDiagram participant pipeline participant validation participant account creation participant user creation pipeline->>validation: runs the first step validation-->>pipeline: returns Failure(validation message) if fails validation->>account creation: passes Success(UserSchema) if valid account creation-->>pipeline: return Failure(account exists) if fails account creation->>user creation: passes Success(Account) if valid user creation-->>pipeline: returns Failure(http status) if fails user creation-->>pipeline: returns Success(user) if user is created

Pipeline execution.¶

See, do notation allows you to write simple yet powerful pipelines with multiple and complex steps. And at the same time the produced code is simple and readable.

And that’s it!

safe¶

safe is used to convert regular functions that can throw exceptions to functions that return Result type.

Supports both async and regular functions.

from returns.functions import safe

@safe
def divide(number: int) -> float:
    return number / number

divide(1)
# => Success(1.0)

divide(0)
# => Failure(ZeroDivisionError)

Limitations¶

There’s one limitation in typing that we are facing right now due to mypy issue:

from returns.functions import safe

@safe
def function(param: int) -> int:
    return param

reveal_type(function)
# Actual => def (*Any, **Any) -> builtins.int
# Expected => def (int) -> builtins.int

This effect can be reduced with the help of Design by Contract with these implementations:

compose¶

We also ship an utility function to compose two different functions together.

from returns.functions import compose

bool_after_int = compose(int, bool)
bool_after_int('1')  # => True
bool_after_int('0')  # => False

Composition is also type-safe. The only limitation is that we only support functions with one argument and one return to be composed.

Only works with regular functions (not async).

raise_exception¶

Sometimes you really want to reraise an exception from Failure[Exception] due to some existing API (or a dirty hack).

We allow you to do that with ease!

from returns.functions import raise_exception

class CreateAccountAndUser(object):
    """Creates new Account-User pair."""

    @pipeline
    def __call__(self, username: str) -> ...:
        """Imagine, that you need to reraise ValidationErrors due to API."""
        user_schema = self._validate_user(
          username,
        ).fix(
          # What happens here is interesting, since you do not let your
          # unwrap to fail with UnwrapFailedError, but instead
          # allows you to reraise a wrapped exception.
          # In this case `ValidationError()` will be thrown
          # before `UnwrapFailedError`
          raise_exception,
        ).unwrap()

    def _validate_user(
      self, username: str,
    ) -> Result['User', ValidationError]:
      ...

Use this with caution. We try to remove exceptions from our code base. Original proposal is here.

API Reference¶

is_successful(container)[source]¶

Determins if a container was successful or not.

We treat container that raise UnwrapFailedError on .unwrap() not successful.

safe(function)[source]¶

Decorator to covert exception throwing function to ‘Result’ monad.

Show be used with care, since it only catches ‘Exception’ subclasses. It does not catch ‘BaseException’ subclasses.

Supports both async and regular functions.

pipeline(function)[source]¶

Decorator to enable ‘do-notation’ context.

Should be used for series of computations that rely on .unwrap method.

Supports both async and regular functions.

compose(first, second)[source]¶

Allows function composition.

Works as: second . first You can read it as “second after first”.

We can only compose functions with one argument and one return.

raise_exception(exception)[source]¶

Helper function to raise exceptions as a function.

That’s how it can be used:

from returns.functions import raise_exception

# Some operation result:
user: Failure[UserDoesNotExistError]
# Here we unwrap internal exception and raise it:
user.fix(raise_exception)

See: https://github.com/dry-python/returns/issues/56

Version history¶

We follow Semantic Versions since the 0.1.0 release.

Version 0.6.0¶

Features¶

safe and pipeline now supports asyncio
is_successful now returns Literal types if possible

Version 0.5.0¶

Features¶

Adds compose helper function
Adds public API to import returns
Adds raise_exception helper function
Adds full traceback to .unwrap()

Misc¶

Updates multiple dev-dependencies, including mypy
Now search in the docs is working again
Relicenses this project to BSD
Fixes copyright notice in the docs

Version 0.4.0 aka Goodbye, Monads!¶

Features¶

Moves all types to .pyi files
Renames all classes according to new naming pattern
HUGE improvement of types
Renames fmap to map
Renames do_notation to pipeline, moves it to functions.py
Renames ebind to rescue
Renames efmap to fix
Renames Monad to Container
Removes Maybe monad, since typing does not have NonNullable type

Version 0.3.1¶

Bugfixes¶

Adds py.typed file to be PEP561 compatible

Version 0.3.0, Renamed to `returns`¶

The project is renamed to returns and moved to dry-python org.

Features¶

Adds .pyi files for all modules, to enable mypy support for 3rd party users

Version 0.2.0¶

Features¶

Adds Maybe monad
Adds immutability and __slots__ to all monads
Adds methods to work with failures
Adds safe decorator to convert exceptions to Result monad
Adds is_successful() function to detect if your result is a success
Adds failure() method to unwrap values from failed monads

Bugfixes¶

Changes the type of .bind method for Success monad
Changes how equality works, so now Failure(1) != Success(1)
Changes how new instances created on unused methods

Misc¶

Improves docs

Version 0.1.1¶

Bugfixes¶

Changes how PyPI renders package’s page

Misc¶

Improves README with new badges and installation steps

Version 0.1.0¶

Initial release. Featuring only Result and do_notation.

Features¶

Installation¶

Why?¶

Straight-forward approach¶

Hidden problems¶

Pipeline example¶

License¶

Inspirations¶

Contents¶

Container: the concept¶

Basics¶

Railway oriented programming¶

Working with containers¶

Returning execution to the right track¶

Unwrapping values¶

Immutability¶

Type safety¶

API Reference¶

Result¶

API Reference¶

Helper functions¶

is_successful¶

pipeline¶

Using bind technique¶

Using pipeline¶

safe¶

Limitations¶

compose¶

raise_exception¶

API Reference¶

Version history¶

Version 0.6.0¶

Features¶

Version 0.5.0¶

Features¶

Misc¶

Version 0.4.0 aka Goodbye, Monads!¶

Features¶

Version 0.3.1¶

Bugfixes¶

Version 0.3.0, Renamed to returns¶

Features¶

Version 0.2.0¶

Features¶

Bugfixes¶

Misc¶

Version 0.1.1¶

Bugfixes¶

Misc¶

Version 0.1.0¶

Version 0.3.0, Renamed to `returns`¶