Section: 16 [networking.ts::buffer], 17 [networking.ts::buffer.stream] Status: New Submitter: Vinnie Falco Opened: 2018-05-18 Last modified: 2018-05-20
Priority: Not Prioritized
View all issues with New status.
Discussion:
Addresses: networking.tsAsynchronous algorithms are started by a call to an initiating function. When these algorithms are constructed from calls to other initiating functions, the result is called a composed operation. For example, async_read may be implemented in terms of zero or more calls to a stream's async_read_some algorithm. For operations where the caller cannot easily determine ahead of time the storage requirements needed for an algorithm to meet its post-conditions, [networking.ts] introduces the DynamicBuffer concept:
A dynamic buffer encapsulates memory storage that may be automatically resized as required, where the memory is divided into two regions: readable bytes followed by writable bytes. [buffer.reqmts.dynamicbuffer]
Signatures for algorithms in the technical specification which accept dynamic buffers use forwarding references:
// 17.10 [networking.ts::buffer.async.read.until], asynchronous delimited read operations: template< class AsyncReadStream, class DynamicBuffer, class CompletionToken> DEDUCED async_read_until( AsyncReadStream& s, DynamicBuffer&& b, char delim, CompletionToken&& token);
Because the initiating function returns immediately, and the associated composed operation executes later, it is necessary for the algorithm to manage the lifetime of the dynamic buffer. Guidance for doing so is given in the TS:
13.2.7.5 Lifetime of initiating function arguments [async.reqmts.async.lifetime]1. Unless otherwise specified, the lifetime of arguments to initiating functions shall be treated as follows: […] the implementation does not assume the validity of the argument after the initiating function completes […] The implementation may make copies of the argument, and all copies shall be destroyed no later than immediately after invocation of the completion handler.
Given the guidance above, the most sensible approach is for the implementation to move or make a decay-copy of the argument. An implementation of the TS, authored by the principal architect of the TS itself, does precisely that:
template <
typename AsyncReadStream,
typename DynamicBuffer,
typename ReadHandler>
class read_until_delim_op
{
public:
template <typename DeducedBuffers>
read_until_delim_op(
AsyncReadStream& stream,
DeducedBuffers&& buffers,
char delim, ReadHandler& handler)
: […]
buffers_(std::forward<DeducedBuffers>(buffers))
[…]
{
}
[…]
DynamicBuffer buffers_;
[…]
};
template <
typename AsyncReadStream,
typename DynamicBuffer,
typename ReadHandler>
NET_TS_INITFN_RESULT_TYPE(ReadHandler,
void (std::error_code, std::size_t))
async_read_until(
AsyncReadStream& s,
DynamicBuffer&& buffers,
char delim,
ReadHandler&& handler)
{
// If you get an error on the following line it means that your handler does
// not meet the documented type requirements for a ReadHandler.
NET_TS_READ_HANDLER_CHECK(ReadHandler, handler) type_check;
async_completion<ReadHandler,
void (std::error_code, std::size_t)> init(handler);
detail::read_until_delim_op<
AsyncReadStream,
typename decay<DynamicBuffer>::type,
NET_TS_HANDLER_TYPE(ReadHandler, void (std::error_code, std::size_t))>(
s,
DynamicBuffer&&buffers,
delim,
init.completion_handler)(std::error_code(), 0, 1);
return init.result.get();
}
Given the semantics of dynamic buffers implied by the wording, instances of dynamic buffers behave more like references to storage types rather than storage types, as copies refer to the same underlying storage. This can be seen in the declaration of dynamic_string_buffer which meets the requirements of DynamicBuffer:
template <typename Elem, typename Traits, typename Allocator>
class dynamic_string_buffer
{
[…]
private:
std::basic_string<Elem, Traits, Allocator>& string_;
std::size_t size_;
const std::size_t max_size_;
};
A dynamic string buffer contains a reference to the underlying string. Copies of a dynamic string buffer refer to the same string. Note that the dynamic string buffer also contains some state: the size_ and max_size_ data members. This additional metadata informs the dynamic string buffer of the boundaries between the readable and writable bytes, as well as the maximum allowed size of the total of the readable and writable bytes.
When only one composed operation handles the dynamic buffer, things seem to work. However, if a composed operation wishes to invoke another composed operation and pass that dynamic buffer, a problem arises. Consider two composed operations f and g, which both operate on an instance of dynamic buffer. When f is invoked, it makes a copy of the dynamic buffer and then calls g with the copy. At this point, g must also make a copy. Copies share the underlying storage, but in the case of dynamic string buffers each copy maintains its own distinct metadata. When g has finished mutating the dynamic buffer and returns control back to f by invoking the completion handler, the metadata in the copy of the dynamic buffer held by f will not have the changes made by g. Another design problem caused by adding metadata to the dynamic buffer concept is illustrated in the following example code:
template<class MutableBufferSequence>
std::size_t read(const MutableBufferSequence&)
{
throw std::exception{};
}
int main()
{
std::string s;
assert(s.empty());
try
{
auto b = boost::asio::dynamic_buffer(s);
b.commit(read(b.prepare(32)));
}
catch(const std::exception&)
{
assert(s.empty()); // fails
}
}
While not technically incorrect, it may be surprising to the user that the string contains additional value-initialized data which was not part of the original readable bytes (which in this case was empty). The wording of the dynamic buffer concept does not address this case.
The solution we propose is to change the semantics of DynamicBuffer to represent a true storage type rather than a hybrid reference with metadata. Instances of dynamic buffers will be passed by reference, and callers will be required to manage the lifetime of dynamic buffer objects for the duration of any asynchronous operations. An additional benefit of this change is that it also solves the problem of exceptions described above.Proposed resolution:
This wording is relative to N4734.
[Drafting note: The project editor is kindly asked to replace all occurrences of DynamicBuffer&& with DynamicBuffer& as indicated by the provided wording changes below. — end drafting note]
Modify 16.1 [networking.ts::buffer.synop], header <experimental/buffer> synopsis, as indicated:
[…] // 16.11 [networking.ts::buffer.creation], buffer creation: […] template<class T, class Allocator> class dynamic_vector_buffer; template<class CharT, class Traits, class Allocator> class basic_dynamic_string_buffer;// 16.14 [networking.ts::buffer.dynamic.creation], dynamic buffer creation:template<class T, class Allocator> dynamic_vector_buffer<T, Allocator> dynamic_buffer(vector<T, Allocator>& vec) noexcept; template<class T, class Allocator> dynamic_vector_buffer<T, Allocator> dynamic_buffer(vector<T, Allocator>& vec, size_t n) noexcept;template<class CharT, class Traits, class Allocator> dynamic_string_buffer<CharT, Traits, Allocator> dynamic_buffer(basic_string<CharT, Traits, Allocator>& str) noexcept; template<class CharT, class Traits, class Allocator> dynamic_string_buffer<CharT, Traits, Allocator> dynamic_buffer(basic_string<CharT, Traits, Allocator>& str, size_t n) noexcept;[…] // 17.5 [networking.ts::buffer.read], synchronous read operations: […] template<class SyncReadStream, class DynamicBuffer> size_t read(SyncReadStream& stream, DynamicBuffer&&b); template<class SyncReadStream, class DynamicBuffer> size_t read(SyncReadStream& stream, DynamicBuffer&&b, error_code& ec); template<class SyncReadStream, class DynamicBuffer, class CompletionCondition> size_t read(SyncReadStream& stream, DynamicBuffer&&b, CompletionCondition completion_condition); template<class SyncReadStream, class DynamicBuffer, class CompletionCondition> size_t read(SyncReadStream& stream, DynamicBuffer&&b, CompletionCondition completion_condition, error_code& ec); // 17.6 [networking.ts::buffer.async.read], asynchronous read operations: […] template<class AsyncReadStream, class DynamicBuffer, class CompletionToken> DEDUCED async_read(AsyncReadStream& stream, DynamicBuffer&&b, CompletionToken&& token); template<class AsyncReadStream, class DynamicBuffer, class CompletionCondition, class CompletionToken> DEDUCED async_read(AsyncReadStream& stream, DynamicBuffer&&b, CompletionCondition completion_condition, CompletionToken&& token); // 17.7 [networking.ts::buffer.write], synchronous write operations: […] template<class SyncWriteStream, class DynamicBuffer> size_t write(SyncWriteStream& stream, DynamicBuffer&&; b); template<class SyncWriteStream, class DynamicBuffer> size_t write(SyncWriteStream& stream, DynamicBuffer&&b, error_code& ec); template<class SyncWriteStream, class DynamicBuffer, class CompletionCondition> size_t write(SyncWriteStream& stream, DynamicBuffer&&b, CompletionCondition completion_condition); template<class SyncWriteStream, class DynamicBuffer, class CompletionCondition> size_t write(SyncWriteStream& stream, DynamicBuffer&&b, CompletionCondition completion_condition, error_code& ec); // 17.8 [networking.ts::buffer.async.write], asynchronous write operations: […] template<class AsyncWriteStream, class DynamicBuffer, class CompletionToken> DEDUCED async_write(AsyncWriteStream& stream, DynamicBuffer&&b, CompletionToken&& token); template<class AsyncWriteStream, class DynamicBuffer, class CompletionCondition, class CompletionToken> DEDUCED async_write(AsyncWriteStream& stream, DynamicBuffer&&b, CompletionCondition completion_condition, CompletionToken&& token); // 17.9 [networking.ts::buffer.read.until], synchronous delimited read operations: template<class SyncReadStream, class DynamicBuffer> size_t read_until(SyncReadStream& s, DynamicBuffer&&b, char delim); template<class SyncReadStream, class DynamicBuffer> size_t read_until(SyncReadStream& s, DynamicBuffer&&b, char delim, error_code& ec); template<class SyncReadStream, class DynamicBuffer> size_t read_until(SyncReadStream& s, DynamicBuffer&&b, string_view delim); template<class SyncReadStream, class DynamicBuffer> size_t read_until(SyncReadStream& s, DynamicBuffer&&b, string_view delim, error_code& ec); // 17.10 [networking.ts::buffer.async.read.until], asynchronous delimited read operations: template<class AsyncReadStream, class DynamicBuffer, class CompletionToken> DEDUCED async_read_until(AsyncReadStream& s, DynamicBuffer&&b, char delim, CompletionToken&& token); template<class AsyncReadStream, class DynamicBuffer, class CompletionToken> DEDUCED async_read_until(AsyncReadStream& s, DynamicBuffer&&b, string_view delim, CompletionToken&& token); […]
Modify 16.2.4 [networking.ts::buffer.reqmts.dynamicbuffer], as indicated:
-1- […]
-2- A type X meets the DynamicBuffer requirements if it satisfies the requirements of Destructible (C++ 2014 [destructible])and MoveConstructible (C++ 2014 [moveconstructible]),as well as the additional requirements listed in Table 14.
Modify 16.12 [networking.ts::buffer.dynamic.vector], as indicated:
[…] template<class T, class Allocator = allocator<T>> class dynamic_vector_buffer { public: // types: using value_type = vector<T, Allocator>; using const_buffers_type = const_buffer; using mutable_buffers_type = mutable_buffer; // constructors: dynamic_vector_buffer() = default; explicit dynamic_vector_buffer(size_t maximum_size); explicit dynamic_vector_buffer(vector<T, Allocator>&vec)noexcept;dynamic_vector_buffer(vector<T, Allocator>&vec, size_t maximum_size)noexcept;dynamic_vector_buffer(dynamic_vector_buffer&&) = default; // members: size_t size() const noexcept; size_t max_size() const noexcept; void max_size(size_t maximum_size); size_t capacity() const noexcept; const_buffers_type data() const noexcept; mutable_buffers_type prepare(size_t n); void commit(size_t n); void consume(size_t n); span<const T> get() const noexcept value_type release(); private: vector<T, Allocator>&vec_; // exposition only size_t size_; // exposition onlyconstsize_t max_size_; // exposition only }; […]-2- […]
-3- […]explicit dynamic_vector_buffer(size_t maximum_size)-?- Effects: Default-constructs vec_. Initializes size_ with 0, and max_size_ with maximum_size.
explicit dynamic_vector_buffer(vector<T, Allocator>&vec)noexcept-4- Effects: Initializes vec_ with move(vec), size_ with vec_.size(), and max_size_ with vec_.max_size()
dynamic_vector_buffer(vector<T, Allocator>&vec, size_t maximum_size)noexcept;-5- Requires: vec.size() <= maximum_size
-6- Effects: Initializes vec_ with move(vec), size_ with vec_.size(), and max_size_ with maximum_size. […]size_t max_size() const noexcept;-8- Returns: max_size_.
void max_size(size_t maximum_size)[…]-?- Effects: Performs max_size_ = maximum_size.
void consume(size_t n);-15- Effects: […]
span<const T> get() const noexcept-?- Returns: span<const T>(vec_.data(), size_).
value_type release()-?- Returns: move(vec_).
Modify 16.13 [networking.ts::buffer.dynamic.string], as indicated:
template<class CharT, class Traits, class Allocator> class basic_dynamic_string_buffer { public: // types: using value_type = basic_string<CharT, Traits, Allocator>; using const_buffers_type = const_buffer; using mutable_buffers_type = mutable_buffer; // constructors: basic_dynamic_string_buffer() = default; explicit basic_dynamic_string_buffer(size_t maximum_size); explicit basic_dynamic_string_buffer(basic_string<CharT, Traits, Allocator>&str)noexcept;basic_dynamic_string_buffer(basic_string<CharT, Traits, Allocator>&str, size_t maximum_size)noexcept;basic_dynamic_string_buffer(basic_dynamic_string_buffer&&) = default; // members: size_t size() const noexcept; size_t max_size() const noexcept; void max_size(size_t maximum_size) size_t capacity() const noexcept; const_buffers_type data() const noexcept; mutable_buffers_type prepare(size_t n); void commit(size_t n) noexcept; void consume(size_t n); basic_string_view<CharT, Traits> get() const noexcept value_type release(); private: basic_string<CharT, Traits, Allocator>&str_; // exposition only size_t size_; // exposition onlyconstsize_t max_size_; // exposition only }; using dynamic_string_buffer = basic_dynamic_string_buffer<char, char_traits<char>, allocator<char>>-2- […]
-3- […]explicit basic_dynamic_string_buffer(size_t maximum_size)-?- Effects: Default-constructs str_. Initializes size_ with 0, and max_size_ with maximum_size.
[…]
explicit basic_dynamic_string_buffer(basic_string<CharT, Traits, Allocator>&str)noexcept-4- Effects: Initializes str_ with move(str), size_ with str_.size(), and max_size_ with str_.max_size()
basic_dynamic_string_buffer(string<CharT, Traits, Allocator>&str, size_t maximum_size)noexcept;-5- Requires: str.size() <= maximum_size.
-6- Effects: Initializes str_ with move(str), size_ with str_.size(), and max_size_ with maximum_size.
[…]
size_t max_size() const noexcept;-8- Returns: max_size_.
void max_size(size_t maximum_size)-?- Effects: Performs max_size_ = maximum_size.
[…]
void consume(size_t n);-15- Effects: […]
basic_string_view<CharT, Traits> get() const noexcept-?- Returns: basic_string_view<CharT, Traits>(str_).
value_type release()-?- Returns: move(str_).
Remove 16.14 [networking.ts::buffer.dynamic.creation] entirely
Modify 17.5 [networking.ts::buffer.read], as indicated:
[…] template<class SyncReadStream, class DynamicBuffer> size_t read(SyncReadStream& stream, DynamicBuffer&&b); template<class SyncReadStream, class DynamicBuffer> size_t read(SyncReadStream& stream, DynamicBuffer&&b, error_code& ec); template<class SyncReadStream, class DynamicBuffer, class CompletionCondition> size_t read(SyncReadStream& stream, DynamicBuffer&&b, CompletionCondition completion_condition); template<class SyncReadStream, class DynamicBuffer, class CompletionCondition> size_t read(SyncReadStream& stream, DynamicBuffer&&b, CompletionCondition completion_condition, error_code& ec); […]
Modify 17.6 [networking.ts::buffer.async.read], as indicated:
[…] template<class AsyncReadStream, class DynamicBuffer, class CompletionToken> DEDUCED async_read(AsyncReadStream& stream, DynamicBuffer&&b, CompletionToken&& token); template<class AsyncReadStream, class DynamicBuffer, class CompletionCondition, class CompletionToken> DEDUCED async_read(AsyncReadStream& stream, DynamicBuffer&&b, CompletionCondition completion_condition, CompletionToken&& token); […]-14- The program shall ensure both the AsyncReadStream object stream and the DynamicBuffer object b are
isvalid until the completion handler for the asynchronous operation is invoked.
Modify 17.7 [networking.ts::buffer.write], as indicated:
[…] template<class SyncWriteStream, class DynamicBuffer> size_t write(SyncWriteStream& stream, DynamicBuffer&&b); template<class SyncWriteStream, class DynamicBuffer> size_t write(SyncWriteStream& stream, DynamicBuffer&&b, error_code& ec); template<class SyncWriteStream, class DynamicBuffer, class CompletionCondition> size_t write(SyncWriteStream& stream, DynamicBuffer&&b, CompletionCondition completion_condition); template<class SyncWriteStream, class DynamicBuffer, class CompletionCondition> size_t write(SyncWriteStream& stream, DynamicBuffer&&b, CompletionCondition completion_condition, error_code& ec); […]
Modify 17.8 [networking.ts::buffer.async.write], as indicated:
[…] template<class AsyncWriteStream, class DynamicBuffer, class CompletionToken> DEDUCED async_write(AsyncWriteStream& stream, DynamicBuffer&&b, CompletionToken&& token); template<class AsyncWriteStream, class DynamicBuffer, class CompletionCondition, class CompletionToken> DEDUCED async_write(AsyncWriteStream& stream, DynamicBuffer&&b, CompletionCondition completion_condition, CompletionToken&& token); […]-14- The program shall ensure both the AsyncWriteStream object stream and the DynamicBuffer object b
memory associated with the dynamic buffer bare valid until the completion handler for the asynchronous operation is invoked.
Modify 17.9 [networking.ts::buffer.read.until], as indicated:
template<class SyncReadStream, class DynamicBuffer> size_t read_until(SyncReadStream& s, DynamicBuffer&&b, char delim); template<class SyncReadStream, class DynamicBuffer> size_t read_until(SyncReadStream& s, DynamicBuffer&&b, char delim, error_code& ec); template<class SyncReadStream, class DynamicBuffer> size_t read_until(SyncReadStream& s, DynamicBuffer&&b, string_view delim); template<class SyncReadStream, class DynamicBuffer> size_t read_until(SyncReadStream& s, DynamicBuffer&&b, string_view delim, error_code& ec); […]
Modify 17.10 [networking.ts::buffer.async.read.until], as indicated:
template<class AsyncReadStream, class DynamicBuffer, class CompletionToken> DEDUCED async_read_until(AsyncReadStream& s, DynamicBuffer&&b, char delim, CompletionToken&& token); template<class AsyncReadStream, class DynamicBuffer, class CompletionToken> DEDUCED async_read_until(AsyncReadStream& s, DynamicBuffer&&b, string_view delim, CompletionToken&& token); […]-6- The program shall ensure both the AsyncReadStream object stream and the DynamicBuffer object b are
isvalid until the completion handler for the asynchronous operation is invoked.