Doc. no:  P1133R0
Audience: LEWG
Date:     2018-06-21
Reply-To: Vinnie Falco (vinnie.falco@gmail.com)

Networking TS Associations For Call Wrappers

Contents

Abstract

In [networking.ts] (also called "TS" henceforth) a CompletionHandler is a callable object invoked when an asynchronous operation has completed. The TS specifies customization points allowing an executor, a proto-allocator, or both, to be associated with an instance of a completion handler. Authors of asynchronous algorithms frequently need to bind parameters to a completion handler and submit the resulting call wrapper to another algorithm. Unfortunately when doing so with a lambda, or a forwarding call wrapper such as std::bind, customizations associated with the original completion handler are lost. This paper proposes wording which forwards TS associations to call wrappers returned by the standard library.

Introduction

Asynchronous algorithms are started by a call to an initiating function which returns immediately. When the operation has completed, the implementation invokes a caller-provided function object called a completion handler with the result of the operation expressed as a parameter list. The following code shows calls to initiating functions with various types of completion handlers passed as the last argument.

Using a lambda:

async_read([…],
    [](std::error_code ec, std::size_t)
    {
        if(ec)
            std::cerr << ec.message() << "\n";
    });

Using a function object:

struct my_completion
{
    void operator()(std::error_code, std::size_t);
};

async_read([…], my_completion{});

Using a call wrapper:

struct session
{
    […]

    void on_read(std::error_code, std::size_t)
    {
        async_read(sock_, buffers_,
            std::bind(
                &session::on_read,
                this,
                std::placeholders::_1,
                std::placeholders::_2));
    }
};

To allow for greater control over memory allocations performed by the implementation or asynchronous algorithms, the TS allows an allocator to be associated with the type of the completion handler. This can be accomplished two ways.

Intrusively, by adding a nested type and member function:

struct my_completion
{
    using allocator_type = […]

    allocator_type get_allocator() const noexcept;

    void operator()(std::error_code, std::size_t);
};

Or by specializing a class template:

namespace std { namespace experimental { namespace net { template<class Allocator> struct associated_allocator<my_completion> { using type = […] static type get(my_completion const& h, Allocator const& a = Allocator()) noexcept; }; }}}

The TS also allows control over the executor used to invoke the completion handler, through customizations similar to the associated allocator.

Intrusively, by adding a nested type and member function:

struct my_completion
{
    using executor_type = […]

    executor_type get_executor() const noexcept;

    void operator()(std::error_code, std::size_t);
};

Or by specializing a class template:

namespace std {
namespace experimental {
namespace net {

template<class Executor>
struct associated_executor<my_completion>
{
    using type = […]

    static type get(my_completion const& h, Executor const& ex = Executor()) noexcept;
};

}}}

Problem

Algorithms expressed in terms of other asynchronous algorithms are called composed operations. An initiating function constructs the composed operation as a callable function object containing the composed operation state as well as ownership of the caller's completion handler, and then invokes the resulting object. The class below implements a composed operation which reads from a stream asynchronously into a dynamic buffer until a condition is met (the corresponding initiating function is intentionally omitted):

template<
    class AsyncReadStream,
    class DynamicBuffer,
    class Condition,
    class Handler>
class read_until_op
{
    AsyncReadStream& stream_;
    DynamicBuffer& buffer_;
    Condition cond_;
    Handler handler_;

public:
    read_until_op(
        AsyncReadStream& stream,
        DynamicBuffer& buffer,
        Condition const& cond,
        Handler handler)
        : stream_(stream)
        , buffer_(buffer)
        , cond_(cond)
        , handler_(std::move(handler))
    {
    }

    void operator()(std::error_code ec, std::size_t bytes_transferred)
    {
        if(! ec)
        {
            if(bytes_transferred > 0)
                buffer_.commit(bytes_transferred);

            if(! cond_())
                return stream_.async_read_some(
                    buffer_.prepare(1024),
                    std::move(*this));
        }

        handler_(ec);
    }
};

The implementation above contains subtle but significant defects which have been a common source of bugs for years when using Asio or Boost.Asio. Any allocator or executor associated with Handler is not propagated to read_until_op, and thus will not be used in the implementation of the call to async_read_some. In particular multi-threaded network programs using the code above will experience undefined behavior when the associated executor of a completion handler is a strand, which is used to guarantee that handlers are not invoked concurrently. The code above may be fixed by associating the composed operation with the same executor and allocator associated with the final completion handler. The simpler, intrusive method may be used here as we have access to the necessary executor through the supplied stream:

template<
    class AsyncReadStream,
    class DynamicBuffer,
    class Condition,
    class Handler>
class read_until_op
{
    AsyncReadStream& stream_;
    DynamicBuffer& buffer_;
    Condition cond_;
    Handler handler_;

public:
    read_until_op(
        AsyncReadStream& stream,
        DynamicBuffer& buffer,
        Condition const& cond,
        Handler handler)
        : stream_(stream)
        , buffer_(buffer)
        , cond_(cond)
        , handler_(std::move(handler))
    {
    }

    using allocator_type = std::experimental::net::associated_allocator<Handler>

    allocator_type get_allocator() const noexcept
    {
        return std::experimental::net::get_associated_allocator(h_);
    }

    using executor_type = std::experimental::net::associated_executor<
        Handler, decltype(std::declval<AsyncReadStream&>().get_executor())>

    executor_type get_executor() const noexcept
    {
        return std::experimental::net::get_associated_executor(handler_, stream_.get_executor());
    }

    void operator()(std::error_code ec, std::size_t bytes_transferred)
    {
        if(! ec)
        {
            if(bytes_transferred > 0)
                buffer_.commit(bytes_transferred);

            if(! cond_())
                return stream_.async_read_some(
                    buffer_.prepare(1024),
                    std::move(*this));
        }

        handler_(ec);
    }
};

The addition of the nested types and member functions give read_until_op the same allocator and executor associated with the caller-provided completion handler. These associations are visible to other initiating functions, such as in the call to the stream's async_read_some. Unfortunately, the code above still has yet another subtle problem with significant consequences. Consider the case where cond_() evaluates to true upon the first invocation. The call to the stream's asynchronous read operation will be skipped, and the handler will be invoked directly with an error code indicating success. This violates the TS requirement that the final completion handler is invoked through the associated executor:

13.2.7.12 Execution of completion handler on completion of asynchronous operation [async.reqmts.async.completion]

If an asynchronous operation completes immediately (that is, within the thread of execution calling the initiating function, and before the initiating function returns), the completion handler shall be submitted for execution as if by performing ex2.post(std::move(f), alloc2). Otherwise, the completion handler shall be submitted for execution as if by performing ex2.dispatch(std::move(f), alloc2).

The TS defines the helper functions post and dispatch to provide the allocator boilerplate in the executor expressions above. However, executors expect nullary function objects (invocable with no arguments). In order to submit a call to the handler in the code above, it is necessary to use a lambda to create a nullary function which invokes the handler with the bound error code. Use of the lambda, along with a bit of extra code to avoid a double dispatch for the case where the completion handler is invoked as a continuation of the call to async_read_some would look thusly:

void read_until_op::operator()(
    std::error_code ec, std::size_t bytes_transferred, bool is_continuation = true)
{
    if(! ec)
    {
        if(bytes_transferred > 0)
            buffer_.commit(bytes_transferred);

        if(! cond_())
            return stream_.async_read_some(
                buffer_.prepare(1024),
                std::move(*this));
    }

    if(! is_continuation)
        std::experimental::net::post(
            stream_.get_executor(),
            [self = std::move(*this), ec] { self_.handler_(ec); });
    else
        handler_(ec);
}

Astute observers may wonder why the handler is called directly when the composed operation is invoked as a continuation of the call to async_read_some instead of using the associated executor's dispatch function. The reason is that we are guaranteed that the composed operation was already invoked through the associated exector's dispatch function, because the implementation of async_read_some must meet the requirements of 13.2.7.12 [async.reqmts.async.completion].

Readers without a deep understanding of Asio or [networking.ts] may not realize that the code above contains another defect. It suffers from the same problem found in the original implementation. The lambda type does not have the same allocator and executor associations as the final completion handler, thus violating contracts.

Having the lambda forward the associated allocator and executor of the contained completion handler requires additional syntax and change to the language. This paper does not propose such a language change, as none of the possible changes explored by the author look palatable in the slightest.

Since the intent of the statement in question is to bind arguments to a callable, to produce a new callable, a logical alternative is to consider using std::bind, which looks like this:

std::experimental::net::post(
    stream_.get_executor(),
    std::bind(std::move(*this), ec));

However, once again the code contains a defect! It does not solve the problem, because the call wrapper returned by bind does not forward the necessary associations. But unlike the lambda, in this case the type is emitted by the library. Therefore, the library can provide the specializations for the call wrapper.

Solution

The solution proposed in this paper is to specialize the [networking.ts] class templates associated_allocator and associated_executor for selected call wrappers returned by standard library functions. Each set of specializations will simply forward the allocator and executor associated with the wrapped function object to the call wrapper. We note that there is at least one paper in flight (P0356R1) which adds new call wrappers to the language, bind_front and bind_back. They will need a similar treatment.

Here, some alternatives are explored and exploratory questions are answered:

Wording

This wording is relative to N4734.

  1. Modify 13.5 [async.assoc.alloc], as indicated:

    […]

    -3- The implementation provides partial specializations of associated_allocator for the forwarding call wrapper types returned by std::bind and std::ref. If g of type G is a forwarding call wrapper with target object fd of type FD, and a is a ProtoAllocator of type PA then:

  2. Modify 13.12 [async.assoc.exec], as indicated:

    […]

    -3- The implementation provides partial specializations of associated_allocator for the forwarding call wrapper types returned by std::bind and std::ref. If g of type G is a forwarding call wrapper with target object fd of type FD, and e is a Executor of type E then:

References

[1] https://github.com/boostorg/asio/blob/fbe86d86b1ac53e40444e5af03ca4a6c74c33bda/include/boost/asio/detail/bind_handler.hpp#L32

[2] https://github.com/chriskohlhoff/networking-ts-impl/blob/3524b4408d26a67af683bfd2aad6b0b6b5684b36/include/experimental/__net_ts/detail/bind_handler.hpp#L34

[3] https://www.boost.org/doc/libs/1_67_0/libs/beast/doc/html/beast/ref/boost__beast__bind_handler.html