Read to the line separator in boost :: asio :: streambuf

Question

I would like to use the very convenient Boost async_read_until to read the message until I get the delimiter \r\n\r\n .

I like to use this separator because it is easily debugged with telnet and makes multi-line commands. I simply signal the completion of the command in two new lines.

I call async_read_until as follows:

 void do_read() { boost::asio::async_read_until(m_socket, m_input_buffer, "\r\n\r\n", std::bind(&player::handle_read, this, std::placeholders::_1, std::placeholders::_2)); } 

And my handler looks like this:

 void handle_read(boost::system::error_code ec, std::size_t nr) { std::cout << "handle_read: ec=" << ec << ", nr=" << nr << std::endl; if (ec) { std::cout << " -> emit on_disconnect\n"; } else { std::istream iss(&m_input_buffer); std::string msg; std::getline(iss, msg); std::cout << "dump:\n"; std::copy(msg.begin(), msg.end(), std::ostream_iterator<int>(std::cout, ", ")); std::cout << std::endl; do_read(); } } 

I wanted to use std::getline just like in the example, but on my system this saves the \r character. As you can see, if I connect to the server and write hello plus two CRLFs, I get this side of the dump server:

 handle_read: ec=system:0, nr=9 dump: 104, 101, 108, 108, 111, 13, ^^^ \r here 

By the way, this will also save the next new line in the buffer. So I think std::getline will not do the job for me.

I am looking for a convenient and efficient way to read from boost::asio::streambuf until I get this delimiter \r\n\r\n . Since I use async_read_until once at a time when the handler is called, it is assumed that the buffer has accurate and complete data, isn't it? What do you recommend reading until I get \r\n\r\n ?

Answer 1

The async_read_until() operation captures all the data read into the input sequence streambuf, and the value bytes_transferred will contain the number of bytes up including the first delimiter. Although an operation can read more data outside the delimiter, you can use the bytes_transferred size and delimiter to retrieve only the data you bytes_transferred . For example, if cmd1\r\n\r\ncmd2 is readable from a socket, and the operation async_read_until() starts with the delimiter \r\n\r\n , then the input sequence streambuf may contain cmd1\r\n\r\ncmd2 :

  ,--------------- buffer_begin(streambuf.data()) / ,------------ buffer_begin(streambuf.data()) + bytes_transferred / / - delimiter.size() / / ,------ buffer_begin(streambuf.data()) + bytes_transferred / / / ,-- buffer_end(streambud.data()) cmd1\r\n\r\ncmd2 

Thus, you can extract cmd1 to a string from streambuf via:

 // Extract up to the first delimiter. std::string command{ boost::asio::buffers_begin(streambuf.data(), boost::asio::buffers_begin(streambuf.data()) + bytes_transferred - delimiter.size()}; // Consume through the first delimiter. m_input_buffer.consume(bytes_transferred); 

---

Here is a complete example demonstrating building std::string directly from the streambuf input sequence:

 #include <functional> // std::bind #include <iostream> #include <boost/asio.hpp> const auto noop = std::bind([]{}); int main() { using boost::asio::ip::tcp; boost::asio::io_service io_service; // Create all I/O objects. tcp::acceptor acceptor(io_service, tcp::endpoint(tcp::v4(), 0)); tcp::socket socket1(io_service); tcp::socket socket2(io_service); // Connect sockets. acceptor.async_accept(socket1, noop); socket2.async_connect(acceptor.local_endpoint(), noop); io_service.run(); io_service.reset(); const std::string delimiter = "\r\n\r\n"; // Write two commands from socket1 to socket2. boost::asio::write(socket1, boost::asio::buffer("cmd1" + delimiter)); boost::asio::write(socket1, boost::asio::buffer("cmd2" + delimiter)); // Read a single command from socket2. boost::asio::streambuf streambuf; boost::asio::async_read_until(socket2, streambuf, delimiter, [delimiter, &streambuf]( const boost::system::error_code& error_code, std::size_t bytes_transferred) { // Verify streambuf contains more data beyond the delimiter. (eg // async_read_until read beyond the delimiter) assert(streambuf.size() > bytes_transferred); // Extract up to the first delimiter. std::string command{ buffers_begin(streambuf.data()), buffers_begin(streambuf.data()) + bytes_transferred - delimiter.size()}; // Consume through the first delimiter so that subsequent async_read_until // will not reiterate over the same data. streambuf.consume(bytes_transferred); assert(command == "cmd1"); std::cout << "received command: " << command << "\n" << "streambuf contains " << streambuf.size() << " bytes." << std::endl; } ); io_service.run(); } 

Conclusion:

 received command: cmd1 streambuf contains 8 bytes. 

Answer 2

To answer your questions first:

Is it expected that the buffer will contain accurate and complete data?

Yes, it will have all the data, including "\ r \ n \ r \ n"

What do you recommend reading until I get \ r \ n \ r \ n?

What you do is good enough. You just need to ignore the extra "\ r" at the end of each command. This can be done while reading from stream or allowing it to process the shell (or whatever processing the commands does for you). My recommendation would be to defer the removal of the extra "\ r" to the command processor.

You probably need something in the lines:

 #include <iostream> #include <string> #include <sstream> void handle_read() { std::stringstream oss; oss << "key : value\r\nkey2: value2\r\nkey3: value3\r\n\r\n"; std::string parsed; while (std::getline(oss, parsed)) { // Check if it'a an empty line. if (parsed == "\r") break; // Remove the additional '\r' here or at command processor code. if (parsed[parsed.length() - 1] == '\r') parsed.pop_back(); std::cout << parsed << std::endl; std::cout << parsed.length() << std::endl; } } int main() { handle_read(); return 0; } 

If your protocol allows you to send empty commands, you will have to change the logic and look for two consecutive empty new lines.

Answer 3

What do you really want to make out?

Of course, you could just use the knowledge from your domain and say

 std::getline(iss, msg, '\r'); 

At a higher level, consider parsing what you need:

 std::istringstream linestream(msg); std::string command; int arg; if (linestream >> command >> arg) { // ... } 

Better yet, consider a parser generator:

 std::string command; int arg; if (qi::phrase_parse(msg.begin(), msg.end(), command_ >> qi::int_, qi::space, command, arg)) { // ... } 

Where command_ could be like

 qi::rule<std::string::const_iterator> command_ = qi::no_case [ qi::lit("my_cmd1") | qi::lit("my_cmd2") ];