Doc. no: D1146R0 Audience: LEWG Date: 2018-07-09 Reply-To: Vinnie Falco (vinnie.falco@gmail.com)
This document proposes an HTTP protocol library for C++. The library is aimed at providing low-level abstractions only. The benefit is that HTTP programs built with this library will be easier and faster to implement, understand, and maintain, because the library takes care of the protocol details.
Problem areas addressed by this proposal include:
Features that are considered outside the scope of this proposal include:
The bulk of the library interface is intended for use by developers with an understanding of Networking TS concepts, as well as a good grasp of HTTP protocol semantics. Most end users will likely not interact with the library directly; instead, this library will be used to build higher level abstractions such as full-featured HTTP clients or HTTP servers. This document refers to abstractions built on top of this library as HTTP middleware (or just middleware).
The Boost.Beast library, from which this proposal is derived, has been deployed in a number of production systems, such as internet-facing HTTP servers, decentralized cryptocurrency networks, and finance applications. The Boost.Beast library has been used on the following platforms:
The interfaces used for transacting with streams are based on idioms established by the Networking TS. In particular this library adopts the Universal Asynchronous Model defined by the TS.
This is a pure library proposal. It does not add any new language features, nor does it alter any existing standard library headers. This library can be implemented using compilers that conform to the C++17 standard (or later). However, this library also requires the library features offered in Networking TS.
The overarching design principle of this library is "Don't make odd choices on behalf of the user." Important decisions such as when and how to allocate memory are left to the caller. Customization points allow all the "interesting" behaviors to be user-defined. It is important to recognize that this library is low-level. It is not a complete HTTP client nor does it provide a complete HTTP server. However, those things can be built with the library. Specifically, this library provides the following:
Most importantly, this library defines a universal, first class container for holding any type of HTTP message, including both HTTP/1 and HTTP/2. This allows authors to write functional algorithms which operate only on messages, independent of streams or networking, with the guarantee that these algorithms will interoperate seamlessly.
All stream algorithms use interfaces similar or identical to those found in the Networking TS. The advantage is that skill acquired at operating the TS will translate directly into skill with using HTTP stream algorithms. To prevent duplication of functionality, the library only provides algorithms which are strictly part of the low-level HTTP protocol. Other operations, such as perfoming DNS lookups or making outgoing TCP/IP connections, are left to the Networking TS.
Stream algorithms are expressed as function templates which operate on stream concepts. This allows any form of transport to be used, including TCP/IP sockets but also including properly configured SSL or TLS streams. Unconventional transports such as interprocess communication may also be used, if an implementation exists or can be written to meet the concept requirements.
Why no HTTP/2?
The message container described herein does support the modeling of HTTP/2 messages. Missing, however, are the stream algorithms. The simple, functional approach taken by the HTTP/1 stream algorithms in this library are insufficient, as the HTTP/2 specification mandates a stateful connection. Furthermore, due to the required flow control features, a synchronous implementation is not practical. Implementing a full featured low-level HTTP/2 library that is up to the quality needed for standardisation is a non-trivial endeavor.
We note that HTTP/2 is still controversial. Widespread adoption has not happened yet. As of this writing, no browser supports HTTP/2 unencrypted. Even if the C++ standard had networking and HTTP/2 today, it would not be capable of serving content to any browsers, as C++ networking does not yet have SSL/TLS support.
Why HTTP/1?
HTTP/1 is unlikely to ever go away, as the relatively small amount of code required to implement it makes it favored for many use-cases such as the Internet of Things or embedded devices. HTTP/1 is also needed when implementing the HTTP/1 Upgrade handshake mechanic used for switching protocols, WebSocket in particular. The HTTP/1 Upgrade mechanism is even needed to initiate an HTTP/2 connection.
Exact wording is to be determined after [networking.ts] is merged
The following definitions are adopted from [networking.ts] as-is:
namespace std {
namespace http {
// 5.3 enumerations and strings
enum class field : unsigned short;
enum class status : unsigned short;
enum class status_class : unsigned char;
auto field_to_string (field f) -> string_view;
auto string_to_field (string_view s) -> field;
auto operator<< (ostream& os, field f) -> ostream&
auto int_to_status (unsigned short v) -> status;
auto to_status_class (unsigned char v) -> status_class;
auto to_status_class (status v) -> status_class;
auto obsolete_reason (status v) -> string_view;
auto operator<< (ostream& os, status s) -> ostream&;
auto string_to_verb (string_view s) -> verb;
auto verb_to_string (verb v) -> string_view;
auto operator<< (ostream& os, verb v) -> ostream&
// 5.4 message container
template <class Allocator>
class basic_fields;
using fields = basic_fields <std::allocator<char>>;
template <bool isRequest, class Fields = fields>
class header;
template <class Fields = fields>
using request_header = header<true, Fields>
template <class Fields = fields>
using response_header = header<false, Fields>
template <bool isRequest, class Body, class Fields = fields>
class message;
template <class Body, class Fields = fields>
using request = message<true, Body, Fields>;
template <class Body, class Fields = fields>
using response = message<false, Body, Fields>;
template <bool isRequest, class Fields>
void swap (header<isRequest, Fields>& m1, header<isRequest, Fields>& m2);
template <bool isRequest, class Body, class Fields>
void swap (message<isRequest, Body, Fields>& m1, message<isRequest, Body, Fields>& m2);
// body types
template <class CharT, class Traits = std::char_traits<CharT>, class Allocator = std::allocator<CharT>>
class basic_string_body;
using string_body = basic_string_body<char>;
template <class T, class Allocator = std::allocator<T>>
struct vector_body;
// parsing and serialization
template <bool isRequest, class Derived>
class basic_parser;
template <bool isRequest, class Body, class Allocator = std::allocator<char>>
class parser;
template <class Body, class Allocator = std::allocator<char>>
using request_parser = parser<true, Body, Allocator<;
template <class Body, class Allocator = std::allocator<char>>
using response_parser = parser<false, Body, Allocator>;
template <bool isRequest, class Body, class Fields = fields>
class serializer;
template <class Body, class Fields = fields>
using request_serializer = serializer<true, Body, Fields>;
template <class Body, class Fields = fields>
using response_serializer = serializer<false, Body, Fields>;
// synchronous read operations
template <class SyncReadStream, class DynamicBuffer,
bool isRequest, class Derived>
auto read_some (SyncReadStream& stream, DynamicBuffer& buffer,
basic_parser<isRequest, Derived>& parser) -> size_t;
template <class SyncReadStream, class DynamicBuffer,
bool isRequest, class Derived>
auto read_some (SyncReadStream& stream, DynamicBuffer& buffer,
basic_parser<isRequest, Derived>& parser, error_code& ec) -> size_t;
template <class SyncReadStream, class DynamicBuffer,
bool isRequest, class Derived>
auto read_header (SyncReadStream& stream, DynamicBuffer& buffer,
basic_parser<isRequest, Derived>& parser) -> size_t;
template <class SyncReadStream, class DynamicBuffer,
bool isRequest, class Derived>
auto read_header (SyncReadStream& stream, DynamicBuffer& buffer,
basic_parser<isRequest, Derived>& parser, error_code& ec) -> size_t;
template <class SyncReadStream, class DynamicBuffer,
bool isRequest, class Derived>
auto read (SyncReadStream& stream, DynamicBuffer& buffer,
basic_parser<isRequest, Derived>& parser) -> size_t;
template <class SyncReadStream, class DynamicBuffer,
bool isRequest, class Derived>
auto read (SyncReadStream& stream, DynamicBuffer& buffer,
basic_parser<isRequest, Derived>& parser, error_code& ec) -> size_t;
// asynchronous read operations
template <class AsyncReadStream, class DynamicBuffer,
bool isRequest, class Derived, class CompletionToken>
DEDUCED async_read_some (AsyncReadStream& stream, DynamicBuffer& buffer,
basic_parser <isRequest, Derived>& parser, CompletionToken&& token);
template <class AsyncReadStream, class DynamicBuffer,
bool isRequest, class Derived, class CompletionToken>
DEDUCED async_read_header (AsyncReadStream& stream, DynamicBuffer& buffer,
basic_parser <isRequest, Derived>& parser, CompletionToken&& token);
template <class AsyncReadStream, class DynamicBuffer,
bool isRequest, class Derived, class CompletionToken>
DEDUCED async_read (AsyncReadStream& stream, DynamicBuffer& buffer,
basic_parser <isRequest, Derived>& parser, CompletionToken&& token);
// synchronous write operations
template <class SyncWriteStream, bool isRequest, class Body, class Fields>
auto write_some (SyncWriteStream& stream,
serializer<isRequest, Body, Fields>& sr) -> size_t;
template <class SyncWriteStream, bool isRequest, class Body, class Fields>
auto write_some (SyncWriteStream& stream,
serializer<isRequest, Body, Fields>& sr, error_code& ec) -> size_t;
template <class SyncWriteStream, bool isRequest, class Body, class Fields>
auto write_header (SyncWriteStream& stream,
serializer<isRequest, Body, Fields>& sr) -> size_t;
template <class SyncWriteStream, bool isRequest, class Body, class Fields>
auto write_header (SyncWriteStream& stream,
serializer<isRequest, Body, Fields>& sr, error_code& ec) -> size_t;
template <class SyncWriteStream, bool isRequest, class Body, class Fields>
auto write (SyncWriteStream& stream,
serializer<isRequest, Body, Fields>& sr) -> size_t;
template <class SyncWriteStream, bool isRequest, class Body, class Fields>
auto write (SyncWriteStream& stream,
serializer<isRequest, Body, Fields>& sr, error_code& ec) -> size_t;
// asynchronous write operations
template <class AsyncWriteStream,
bool isRequest, class Body, class Fields, class CompletionToken>
DEDUCED async_write_some (AsyncWriteStream& stream,
serializer<isRequest, Body, Fields>& sr, CompletionToken&& token);
template <class AsyncWriteStream,
bool isRequest, class Body, class Fields, class CompletionToken>
DEDUCED async_write_header (AsyncWriteStream& stream,
serializer<isRequest, Body, Fields>& sr, CompletionToken&& token);
template <class AsyncWriteStream,
bool isRequest, class Body, class Fields, class CompletionToken>
DEDUCED async_write (AsyncWriteStream& stream,
serializer<isRequest, Body, Fields>& sr, CompletionToken&& token);
template <bool isRequest, class Fields>
auto operator<< (ostream& os, header<isRequest, Fields> const& msg) -> ostream&;
template <bool isRequest, class Body, class Fields>
auto operator<< (ostream& os,
message<isRequest, Body, Fields> const& msg) -> ostream&;
} // std
These constants and functions are used to describe and identify metadata in HTTP messages.
enum class field : unsigned short
{
see-below
};
enum class status : unsigned short
{
see-below
};
enum class status_class : unsigned char
{
unknown = 0,
informational = 1,
successful = 2,
redirection = 3,
client_error = 4,
server_error = 5
};
auto field_to_string (field f) -> string_view;
auto string_to_field (string_view s) -> field;
auto operator<< (ostream& os, field f) -> ostream&
auto int_to_status (unsigned short v) -> status;
auto to_status_class (unsigned char v) -> status_class;
auto to_status_class (status v) -> status_class;
auto obsolete_reason (status v) -> string_view;
auto operator<< (ostream& os, status s) -> ostream&;
auto string_to_verb (string_view s) -> verb;
auto verb_to_string (verb v) -> string_view;
auto operator<< (ostream& os, verb v) -> ostream&
enum class field : unsigned short
{
see-below
};
auto field_to_string (field f) -> string_view
Returns:
auto string_to_field (string_view s) -> field;
Returns:
auto operator<< (ostream& os, field f) -> ostream&
Effects: os << field_to_string(f).
Returns: os.
auto int_to_status (unsigned short v) -> status;
Returns:
auto to_status_class (unsigned char v) -> status_class;
Returns:
auto to_status_class (status v) -> status_class;
Returns:
auto obsolete_reason (status v) -> string_view;
Returns:
auto operator<< (ostream& os, status s) -> ostream&;
Effects: os << obsolete_reason(s).
Returns: os.
auto string_to_verb (string_view s) -> verb;
Returns:
auto verb_to_string (verb v) -> string_view;
Returns:
auto operator<< (ostream& os, verb v) -> ostream&
Effects: os << verb_to_string(v).
Returns: os.
[TBD]
id string value unknown "" 0 a_im "" accept "" accept_additions "" accept_charset "" accept_datetime "" accept_encoding "" accept_features "" accept_language "" accept_patch "" accept_post "" accept_ranges "" access_control "" access_control_allow_credentials "" access_control_allow_headers "" access_control_allow_methods "" access_control_allow_origin "" access_control_expose_headers "" access_control_max_age "" access_control_request_headers "" access_control_request_method "" age "" allow "" alpn "" also_control "" alt_svc "" alt_used "" alternate_recipient "" alternates "" apparently_to "" apply_to_redirect_ref "" approved "" archive "" archived_at "" article_names "" article_updates "" authentication_control "" authentication_info "" authentication_results "" authorization "" auto_submitted "" autoforwarded "" autosubmitted "" base "" bcc "" body "" c_ext "" c_man "" c_opt "" c_pep "" c_pep_info "" cache_control "" caldav_timezones "" cancel_key "" cancel_lock "" cc "" close "" comments "" compliance "" connection "" content_alternative "" content_base "" content_description "" content_disposition "" content_duration "" content_encoding "" content_features "" content_id "" content_identifier "" content_language "" content_length "" content_location "" content_md5 "" content_range "" content_return "" content_script_type "" content_style_type "" content_transfer_encoding "" content_type "" content_version "" control "" conversion "" conversion_with_loss "" cookie "" cookie2 "" cost "" dasl "" date "" date_received "" dav "" default_style "" deferred_delivery "" delivery_date "" delta_base "" depth "" derived_from "" destination "" differential_id "" digest "" discarded_x400_ipms_extensions "" discarded_x400_mts_extensions "" disclose_recipients "" disposition_notification_options "" disposition_notification_to "" distribution "" dkim_signature "" dl_expansion_history "" downgraded_bcc "" downgraded_cc "" downgraded_disposition_notification_to "" downgraded_final_recipient "" downgraded_from "" downgraded_in_reply_to "" downgraded_mail_from "" downgraded_message_id "" downgraded_original_recipient "" downgraded_rcpt_to "" downgraded_references "" downgraded_reply_to "" downgraded_resent_bcc "" downgraded_resent_cc "" downgraded_resent_from "" downgraded_resent_reply_to "" downgraded_resent_sender "" downgraded_resent_to "" downgraded_return_path "" downgraded_sender "" downgraded_to "" ediint_features "" eesst_version "" encoding "" encrypted "" errors_to "" etag "" expect "" expires "" expiry_date "" ext "" followup_to "" forwarded "" from "" generate_delivery_report "" getprofile "" hobareg "" host "" http2_settings "" if_ "" if_match "" if_modified_since "" if_none_match "" if_range "" if_schedule_tag_match "" if_unmodified_since "" im "" importance "" in_reply_to "" incomplete_copy "" injection_date "" injection_info "" jabber_id "" keep_alive "" keywords "" label "" language "" last_modified "" latest_delivery_time "" lines "" link "" list_archive "" list_help "" list_id "" list_owner "" list_post "" list_subscribe "" list_unsubscribe "" list_unsubscribe_post "" location "" lock_token "" man "" max_forwards "" memento_datetime "" message_context "" message_id "" message_type "" meter "" method_check "" method_check_expires "" mime_version "" mmhs_acp127_message_identifier "" mmhs_authorizing_users "" mmhs_codress_message_indicator "" mmhs_copy_precedence "" mmhs_exempted_address "" mmhs_extended_authorisation_info "" mmhs_handling_instructions "" mmhs_message_instructions "" mmhs_message_type "" mmhs_originator_plad "" mmhs_originator_reference "" mmhs_other_recipients_indicator_cc "" mmhs_other_recipients_indicator_to "" mmhs_primary_precedence "" mmhs_subject_indicator_codes "" mt_priority "" negotiate "" newsgroups "" nntp_posting_date "" nntp_posting_host "" non_compliance "" obsoletes "" opt "" optional "" optional_www_authenticate "" ordering_type "" organization "" origin "" original_encoded_information_types "" original_from "" original_message_id "" original_recipient "" original_sender "" original_subject "" originator_return_address "" overwrite "" p3p "" path "" pep "" pep_info "" pics_label "" position "" posting_version "" pragma "" prefer "" preference_applied "" prevent_nondelivery_report "" priority "" privicon "" profileobject "" protocol "" protocol_info "" protocol_query "" protocol_request "" proxy_authenticate "" proxy_authentication_info "" proxy_authorization "" proxy_connection "" proxy_features "" proxy_instruction "" public_ "" public_key_pins "" public_key_pins_report_only "" range "" received "" received_spf "" redirect_ref "" references "" referer "" referer_root "" relay_version "" reply_by "" reply_to "" require_recipient_valid_since "" resent_bcc "" resent_cc "" resent_date "" resent_from "" resent_message_id "" resent_reply_to "" resent_sender "" resent_to "" resolution_hint "" resolver_location "" retry_after "" return_path "" safe "" schedule_reply "" schedule_tag "" sec_websocket_accept "" sec_websocket_extensions "" sec_websocket_key "" sec_websocket_protocol "" sec_websocket_version "" security_scheme "" see_also "" sender "" sensitivity "" server "" set_cookie "" set_cookie2 "" setprofile "" sio_label "" sio_label_history "" slug "" soapaction "" solicitation "" status_uri "" strict_transport_security "" subject "" subok "" subst "" summary "" supersedes "" surrogate_capability "" surrogate_control "" tcn "" te "" timeout "" title "" to "" topic "" trailer "" transfer_encoding "" ttl "" ua_color "" ua_media "" ua_pixels "" ua_resolution "" ua_windowpixels "" upgrade "" urgency "" uri "" user_agent "" variant_vary "" vary "" vbr_info "" version "" via "" want_digest "" warning "" www_authenticate "" x_archived_at "" x_device_accept "" x_device_accept_charset "" x_device_accept_encoding "" x_device_accept_language "" x_device_user_agent "" x_frame_options "" x_mittente "" x_pgp_sig "" x_ricevuta "" x_riferimento_message_id "" x_tiporicevuta "" x_trasporto "" x_verificasicurezza "" x400_content_identifier "" x400_content_return "" x400_content_type "" x400_mts_identifier "" x400_originator "" x400_received "" x400_recipients "" x400_trace "" xref ""