Foreword

Our small team is developing two OpenSource tools for C ++ developers - the actor framework SObjectizer and the embedded HTTP server RESTinio . At the same time, we regularly encounter a couple of non-trivial questions:

• What features to add to the library, and which ones to leave "overboard"?
• How to demonstrate the "ideologically correct" ways of using the library?

It is good when the answers to such questions appear in the course of using our developments in real projects, when the developers come to us with their complaints or wishes. At the expense of satisfying the help of users, we fill our tools with functionality that is dictated by life itself, and not “sucked from the finger”.

But information reaches us far from all the problems and difficulties faced by users. And not always we can use the information obtained, and especially code samples, in our public materials.

Therefore, sometimes we invent small problems for ourselves, solving which we are forced to turn from the developers of tools into users. This allows us to look at our own tools with different eyes and understand for ourselves what is good, what is not, what is missing and which is too much.

Today we want to tell just about one such “small” task, in which SObjectizer and RESTinio naturally combined.

Scaling and distribution of images. Why exactly this?

As a small demo task for ourselves, we chose an HTTP server that distributes scaled images on requests. You add images to a directory, start an HTTP server, make a request to it like this:

curl "http://localhost:8080/my_picture.jpg?op=resize&max=1920" 

and get in response to the image, scaled to 1920 pixels on the long side.

This choice was chosen because it perfectly demonstrates the scenarios for which we started developing RESTinio at one time: there is a long-running and debugged C or C ++ code for which you need to attach an HTTP input and start responding to incoming requests. At the same time, what is important, the application processing of the request can take considerable time and therefore it is not profitable to pull the application code directly on the IO context. The HTTP server must be asynchronous: accept and parse the HTTP request, send the parsed request somewhere for further application processing, go on to service the next HTTP request, return to the response to the HTTP request when this response is prepared by someone.

This is exactly what happens when processing requests for scaling images. An HTTP server is capable of performing its immediate work (i.e., reading data, parsing an HTTP request) in a fraction of milliseconds. But the image scaling can take tens, hundreds, or even thousands of milliseconds.

And since it can take a lot of time to scale one image, you need to make sure that the HTTP server can continue its work while the image is scaled. To do this, we need to spread the work of the HTTP server and scaling the images to different working contexts. In the simple case, these will be different worker threads. Well, since we live during multi-core processors, we will have several working threads. Some of them will serve HTTP requests, part - work with pictures.

It turns out that in order to distribute scaled images over HTTP, we need to both reuse long-written, working C / C ++ code (in this case, ImageMagic ++), and asynchronously serve HTTP requests, and perform application-oriented processing of requests in several worker threads. Excellent task for RESTinio and SObjectizer, as it seemed to us.

And we decided to call our demo project a shrimp.

Shrimp as it is

What does a shrimp do?

Shrimp runs as a console application, opens and listens to the specified port, accepts and processes HTTP GET requests of the form:

 /<image>.<ext> /<image>.<ext>?op=resize&<side>=<value> 

Where:

• image is the name of the file with the image to scale. For example, my_picture or DSCF0069;
• ext is one of the extensions supported by shrimp (jpg, jpeg, png or gif);
• side is an indication of the side for which the size is specified. It can be either the width value, in this case the image is scaled so that the resulting width is equal to the specified value, the height of the image is automatically selected with the proportions preserved. Either the height value, in this case, scaling occurs in height. Either max, in this case, the long side is limited, and the shrimp itself determines whether the long side is high or wide;
• value is the size under which the scaling occurs.

If only the file name is specified in the URL, without the resize operation, then shrimp simply returns the original image in the response. If the resize operation is specified, then shrimp changes the size of the requested image and returns the scaled version.

In this case, shrimp keeps in memory a cache of scaled images. If a picture with the same resize parameters that is already in the cache is re-requested, the value from the cache is returned. If there are no pictures in the cache, then the picture is read from the disk, scaled, stored in the cache and returned in the response.

The cache is periodically cleared. Pictures that have been cached for more than an hour since the last access were pushed out of it. The same old pictures are thrown from the cache if the cache exceeds its maximum size (in a demo project it is 100Mb).

We have prepared a page for which anyone can experiment with shrimp:



On this page you can set the size of the image and click "Resize". Two requests will be made to the shrimp server with the same parameters. Most likely, the first request will be unique (i.e. there will not be any images with such resize parameters in the cache), so the first request will take time to actually scale the image. And the second request will most likely find the already scaled picture in the cache and return it immediately.

It is possible to judge whether the picture is given from the cache or was actually scaled by the text under the picture. For example, the text “Transformed (114.0ms)” says that the image was scaled and the zoom operation took 114 milliseconds.

How does shrimp do it?

Shrimp is a multithreaded application that launches three groups of worker threads:

1. The pool of worker threads on which the HTTP server is running. On this pool, new connections are serviced, incoming requests are accepted and parsed, and responses are generated and sent. The HTTP server is implemented through the RESTinio library.
2. A separate working thread on which the transform_manager SObjectizer agent runs. This agent processes requests received from the HTTP server and maintains a cache of scaled images.
3. The pool of worker threads on which the SObjectizer agents are transformer. They perform the actual scaling of images using ImageMagic ++.

It turns out the following scheme of work:



The HTTP server accepts the incoming request, parses it, checks for correctness. If this request does not require a resize operation, then the HTTP server itself processes the request using the sendfile operation. If the request requires a resize operation, then the request is asynchronously sent to the transform_manager agent.

The transform_manager agent receives requests from the HTTP server, checks for the presence of already scaled images in the cache. If there is a picture in the cache, then transform_manager immediately forms the answer for the HTTP server. If there is no image, then transform_manager sends a request to scale the image to one of the transformer agents. When a scaling result comes from transformer, the result is stored in the cache and the response for the HTTP server is generated.

The transformer agent receives requests from transform_manager, processes them and returns the result of the transformation back to the transform_manager agent.

What is Shrimp under the hood?

The source code for the most minimalist version of the shrimp described in this article can be found here in this repository: the shrimp-demo on BitBucket or on GitHub .

There is quite a lot of code, although for the most part, in this version of shrimp, the code is quite trivial. However, it makes sense to focus on some aspects of the implementation.

Using C ++ 17 and the most recent compiler versions

In the implementation of shrimp, we decided to use C ++ 17 and the latest versions of compilers, in particular GCC 7.3 and 8.1. The project is largely research. Therefore, practical familiarity with C ++ 17 in such a project is natural and permissible. Whereas in more mundane developments oriented towards practical industrial applications here and now, we are forced to look at rather old compilers and use perhaps C ++ 14, or even just a subset of C ++ 11.

I must say that C ++ 17 makes a pleasant impression. It seems that in the shrimp code, we didn’t use so many innovations from the seventeenth standard, but it was possible to feel the positive effect: the [[nodiscard]] attribute, std :: optional / std :: variant / std :: filesystem right “ out of the box ”, and not from external dependencies, structured binding, if constexpr, the ability to collect visitor for std :: visit on lambdas ... Separately, these are all trifles, but collectively produce a powerful cumulative effect.

So the first useful result we got when developing a shrimp: C ++ 17 is worth it to go to.

HTTP server using RESTinio

Perhaps the easiest part of shrimp was the HTTP server and the HTTP GET request handler ( http_server.hpp and http_server.cpp ).

Receive and dispatch incoming requests

In fact, the entire core logic of the shrimp HTTP server is concentrated in this function:

 void add_transform_op_handler( const app_params_t & app_params, http_req_router_t & router, so_5::mbox_t req_handler_mbox ) { router.http_get( R"(/:path(.*)\.:ext(.{3,4}))", restinio::path2regex::options_t{}.strict( true ), [req_handler_mbox, &app_params]( auto req, auto params ) { if( has_illegal_path_components( req->header().path() ) ) { return do_400_response( std::move( req ) ); } const auto opt_image_format = image_format_from_extension( params[ "ext" ] ); if( !opt_image_format ) { return do_400_response( std::move( req ) ); } if( req->header().query().empty() ) { return serve_as_regular_file( app_params.m_storage.m_root_dir, std::move( req ), *opt_image_format ); } const auto qp = restinio::parse_query( req->header().query() ); if( "resize" != restinio::value_or( qp, "op"sv, ""sv ) ) { return do_400_response( std::move( req ) ); } handle_resize_op_request( req_handler_mbox, *opt_image_format, qp, std::move( req ) ); return restinio::request_accepted(); } ); } 

This function prepares the HTTP GET request handler using the ExpressJS router implemented in RESTinio. When the HTTP server receives a GET request whose URL matches a given regular expression, the specified lambda function is called.

This lambda function makes a few simple checks for the correctness of the request, but in the main, its work boils down to a simple choice: if the resize mode is not specified, then the requested picture will be returned in its original form using the effective system sendfile. If the resize mode is set, then a message is generated and sent to the transform_manager agent:

 void handle_resize_op_request( const so_5::mbox_t & req_handler_mbox, image_format_t image_format, const restinio::query_string_params_t & qp, restinio::request_handle_t req ) { try_to_handle_request( [&]{ auto op_params = transform::resize_params_t::make( restinio::opt_value< std::uint32_t >( qp, "width" ), restinio::opt_value< std::uint32_t >( qp, "height" ), restinio::opt_value< std::uint32_t >( qp, "max" ) ); transform::resize_params_constraints_t{}.check( op_params ); std::string image_path{ req->header().path() }; so_5::send< so_5::mutable_msg<a_transform_manager_t::resize_request_t>>( req_handler_mbox, std::move(req), std::move(image_path), image_format, op_params ); }, req ); } 

It turns out that the HTTP server, accepting a resize request, sends it to the transform_manager agent via an asynchronous message, while continuing to serve other requests.

Distributing files with sendfile

If the HTTP server detects the request for the original picture, without the resize operation, the server immediately sends this picture through the sendfile operation. The main associated code looks like this (the full code for this function can be found in the repository ):

 [[nodiscard]] restinio::request_handling_status_t serve_as_regular_file( const std::string & root_dir, restinio::request_handle_t req, image_format_t image_format ) { const auto full_path = make_full_path( root_dir, req->header().path() ); try { auto sf = restinio::sendfile( full_path ); ... return set_common_header_fields_for_image_resp( file_stat.st_mtim.tv_sec, resp ) .append_header( restinio::http_field::content_type, image_content_type_from_img_format( image_format ) ) .append_header( http_header::shrimp_image_src, image_src_to_str( http_header::image_src_t::sendfile ) ) .set_body( std::move( sf ) ) .done(); } catch(...) {} return do_404_response( std::move( req ) ); } 

The key point here is to call restinio :: sendfile () , and then passing the value returned by this function to set_body ().

The restinio :: sendfile () function creates a file upload operation using the system API. When this operation is passed to set_body (), RESTinio understands that the contents of the file specified in restinio :: sendfile () will be used for the HTTP response body. Then it uses the system API to write the contents of this file to a TCP socket.

Image cache implementation

The transform_manager agent stores a cache of transformed images where images are placed after scaling. This cache is a simple homemade container that provides access to its contents in two ways:

1. By searching for an item by key (similar to how it happens in standard containers std :: map and std :: unordered_map).
2. By referring to the oldest cache entry.

The first access method is used when we need to check if the image is in the cache. The second is when we delete the oldest images from the cache.

We did not search for something ready for this purpose on the Internet. Probably Boost.MultiIndex would be quite suitable for yourself. But I didn’t want to drag Boost just for the sake of MultiIndex, so we made our trivial implementation literally on the knee. It seems to be working;)

Waiting queue in transform_manager

The transform_manager agent, despite its pretty decent volume ( hpp-file is about 250 lines and cpp-file is about 270 lines), in the simplest implementation of shrimp turned out to be, in our opinion, rather trivial.

One of the moments that makes a significant contribution to the complexity and volume of the agent code is the presence in transform_manager of not only the cache of transformed pictures, but also the queues of pending requests.

We have a limited number of transformer agents (in principle, their number should approximately correspond to the number of available cores). If more requests come at one time than there are free transformers, then we can either immediately respond negatively to the request, or put the request in the queue. And then take from the queue when the free transformer appears.

In shrimp, we use a queue of waiting requests, which is defined as follows:

 struct pending_request_t { transform::resize_request_key_t m_key; sobj_shptr_t<resize_request_t> m_cmd; std::chrono::steady_clock::time_point m_stored_at; pending_request_t( transform::resize_request_key_t key, sobj_shptr_t<resize_request_t> cmd, std::chrono::steady_clock::time_point stored_at ) : m_key{ std::move(key) } , m_cmd{ std::move(cmd) } , m_stored_at{ stored_at } {} }; using pending_request_queue_t = std::queue<pending_request_t>; pending_request_queue_t m_pending_requests; static constexpr std::size_t max_pending_requests{ 64u }; 

When we receive a request, we put it in a queue with a fixation of the time of receipt of the request. Then we periodically check if the request has timed out. After all, in principle, it may happen that a bundle of “heavy” requests had previously arrived, the processing of which took too long. It is wrong to wait indefinitely for a free transformer to appear, it’s better after some time to send a negative response to the client, meaning that the service is now overloaded.

For the queue of pending requests there is also a size limit. If the queue has already reached its maximum size, then we immediately refuse to process the request and tell the client that we are overloaded.

One important point is related to the queue of pending requests, in which we will focus on the article.

Type sobj_shptr_t and reuse of message instances

In the definition of the queue type of pending requests, as well as in the signatures of some transform_manager methods, one can see the use of the sobj_shptr_t type. It makes sense to dwell in more detail on what type it is and why it is used.

The bottom line is that transform_manager receives a request from the HTTP server as a resize_request_t message:

 struct resize_request_t final : public so_5::message_t { restinio::request_handle_t m_http_req; std::string m_image; image_format_t m_image_format; transform::resize_params_t m_params; resize_request_t( restinio::request_handle_t http_req, std::string image, image_format_t image_format, transform::resize_params_t params ) : m_http_req{ std::move(http_req) } , m_image{ std::move(image) } , m_image_format{ image_format } , m_params{ params } {} }; 

and we have to do something to keep this information in the waiting queue. For example, you can create a new instance of resize_request_t and move the values ​​from the received message to it.

And you can recall that the message itself in SObjectizer is a dynamically created object. And not a simple object, but with a reference counter inside. And that in SObjectizer there is a special type of smart pointer for such objects - intrusive_ptr_t.

Those. we can not make a copy of resize_request_t for the queue of waiting requests, but we can simply place in this queue a smart pointer to an already existing instance of resize_request_t. What we are doing. And in order not to write the rather exotic name so_5 :: intrusive_ptr_t everywhere, we enter our own alias:

 template<typename T> using sobj_shptr_t = so_5::intrusive_ptr_t<T>; 

Asynchronous client responses

We said that HTTP requests are processed asynchronously. And they showed above how the HTTP server sends a request to the transform_manager agent with an asynchronous message. But what happens with responses to HTTP requests?

Responses are also served asynchronously. For example, in the transform_manager code, you can see the following:

 void a_transform_manager_t::on_failed_resize( failed_resize_t & /*result*/, sobj_shptr_t<resize_request_t> cmd ) { do_404_response( std::move(cmd->m_http_req) ); } 

This code generates a negative response to an HTTP request in the case when the image was not scaled for some reason. The response is generated in the auxiliary function do_404_response, the code of which can be represented as follows:

 auto do_404_response( restinio::request_handle_t req ) { auto resp = req->create_response( 404, "Not Found" ); resp.append_header( restinio::http_field_t::server, "Shrimp draft server" ); resp.append_header_date_field(); if( req->header().should_keep_alive() ) resp.connection_keep_alive(); else resp.connection_close(); return resp.done(); } 

The first key point with do_404_response () is that this function is called on the working context of the transform_manager agent, and not on the working context of the HTTP server.

The second key point is a call to the done () method on a fully formed object resp. All asynchronous magic with the HTTP response takes place right here. The done () method takes all the information prepared in resp and asynchronously sends it to an HTTP server. Those. the return from do_404_response () will occur immediately after the contents of the resp object have been placed on the HTTP server queue.

The HTTP server, in its working context, detects the presence of a new HTTP response and begins to perform the necessary actions to send the response to the appropriate client.

Type datasizable_blob_t

Another small point that makes sense to explain, because it is certainly incomprehensible without an understanding of the subtleties of RESTinio. Speech about the presence, at first glance, of a strange type of datasizeable_blob_t, defined as follows:

 struct datasizable_blob_t : public std::enable_shared_from_this< datasizable_blob_t > { const void * data() const noexcept { return m_blob.data(); } std::size_t size() const noexcept { return m_blob.length(); } Magick::Blob m_blob; //! Value for `Last-Modified` http header field. const std::time_t m_last_modified_at{ std::time( nullptr ) }; }; 

In order to clarify why this type is needed, you need to show how the HTTP response is formed with the transformed image:

 void serve_transformed_image( restinio::request_handle_t req, datasizable_blob_shared_ptr_t blob, image_format_t img_format, http_header::image_src_t image_src, header_fields_list_t header_fields ) { auto resp = req->create_response(); set_common_header_fields_for_image_resp( blob->m_last_modified_at, resp ) .append_header( restinio::http_field::content_type, image_content_type_from_img_format( img_format ) ) .append_header( http_header::shrimp_image_src, image_src_to_str( image_src ) ) .set_body( std::move( blob ) ); for( auto & hf : header_fields ) { resp.append_header( std::move( hf.m_name ), std::move( hf.m_value ) ); } resp.done(); } 

Pay attention to the set_body () call: a smart pointer to the datasizable_blob_t instance is sent directly there. What for?

The fact is that RESTinio supports several options for forming the body of the HTTP response . The easiest way is to pass an instance of type std :: string to set_body () and RESTinio will save the value of this string inside the object resp.

But there are cases when the value for set_body () should be reused in several answers at once. For example, in shrimp, this happens when shrimp receives several identical requests to transform the same image. . RESTinio set_body() :

 template<typename T> auto set_body(std::shared_ptr<T> body); 

T : data() size(), , RESTinio .

shrimp- Magick::Blob. Magic::Blob data, size(), length(). - datasizable_blob_t, RESTinio Magick::Blob.

transform_manager

transform_manager :

• , ;
• control the time spent by queries in the waiting queue of free transformers.

The transform_manager agent performs these actions through periodic messages. It looks like this.

First, the types of signals that will be used as periodic messages are defined:

 struct clear_cache_t final : public so_5::signal_t {}; struct check_pending_requests_t final : public so_5::signal_t {}; 

Then the agent is subscribed, including to these signals:

 void a_transform_manager_t::so_define_agent() { so_subscribe_self() .event( &a_transform_manager_t::on_resize_request ) .event( &a_transform_manager_t::on_resize_result ) .event( &a_transform_manager_t::on_clear_cache ) .event( &a_transform_manager_t::on_check_pending_requests ); } void a_transform_manager_t::on_clear_cache( mhood_t<clear_cache_t> ) {...} void a_transform_manager_t::on_check_pending_requests( mhood_t<check_pending_requests_t> ) {...} 

Thanks to the subscription, SObjectizer will call the required handler when the agent receives the corresponding signal.

And it remains only to run periodic messages when the agent starts:

 void a_transform_manager_t::so_evt_start() { m_clear_cache_timer = so_5::send_periodic<clear_cache_t>( *this, clear_cache_period, clear_cache_period ); m_check_pending_timer = so_5::send_periodic<check_pending_requests_t>( *this, check_pending_period, check_pending_period ); } 

— timer_id, send_periodic(). , timer_id. , send_periodic() , . a_transform_manager_t :

 so_5::timer_id_t m_clear_cache_timer; so_5::timer_id_t m_check_pending_timer; 

shrimp-. , , RESTinio SObjectizer - - , HelloWorld. .

, transform_manager . , . , . . , , .

transform_manager. , — .

shrimp- « », . , . , , shrimp .

shrimp- . stay tuned.

- shrimp-, RESTinio SObjectizer-, . , shrimp — , - shrimp- - , resize, , .

Continued ...