Splitting a string but saving empty C ++ tokens

Question

I am trying to split a string and put it in a vector

however, I also want to keep an empty token when there is a serial delimiter:

For instance:

string mystring = "::aa;;bb;cc;;c" 

I would like tokenize this line to :; delimiters but between delimiters such as :: and ;; I would like to click on my empty line vector;

 so my desired output for this string is: "" (empty) aa "" (empty) bb cc "" (empty) c 

Also my requirement is not to use boost library.

If anyone could lend me an idea.

thanks

which symbolizes a string but does not include empty tokens

 void Tokenize(const string& str,vector<string>& tokens, const string& delim) { // Skip delimiters at beginning. string::size_type lastPos = str.find_first_not_of(delimiters, 0); // Find first "non-delimiter". string::size_type pos = str.find_first_of(delimiters, lastPos); while (string::npos != pos || string::npos != lastPos) { // Found a token, add it to the vector. tokens.push_back(str.substr(lastPos, pos - lastPos)); // Skip delimiters. Note the "not_of" lastPos = str.find_first_not_of(delimiters, pos); // Find next "non-delimiter" pos = str.find_first_of(delimiters, lastPos); } } 

Answer 1

You can make your algorithm work with some simple changes. First, do not skip separators at the beginning, but instead of skipping separators in the middle of the line, simply increase the position by one. In addition, your npos check should ensure that both positions are not npos , so it should be && instead of || .

 void Tokenize(const string& str,vector<string>& tokens, const string& delimiters) { // Start at the beginning string::size_type lastPos = 0; // Find position of the first delimiter string::size_type pos = str.find_first_of(delimiters, lastPos); // While we still have string to read while (string::npos != pos && string::npos != lastPos) { // Found a token, add it to the vector tokens.push_back(str.substr(lastPos, pos - lastPos)); // Look at the next token instead of skipping delimiters lastPos = pos+1; // Find the position of the next delimiter pos = str.find_first_of(delimiters, lastPos); } // Push the last token tokens.push_back(str.substr(lastPos, pos - lastPos)); } 

Answer 2

I have a version using iterators:

 std::vector<std::string> split_from(const std::string& s , const std::string& d, unsigned r = 20) { std::vector<std::string> v; v.reserve(r); auto pos = s.begin(); auto end = pos; while(end != s.end()) { end = std::find_first_of(pos, s.end(), d.begin(), d.end()); v.emplace_back(pos, end); pos = end + 1; } return v; } 

Using your interface:

 void Tokenize(const std::string& s, std::vector<std::string>& tokens , const std::string& delims) { auto pos = s.begin(); auto end = pos; while(end != s.end()) { end = std::find_first_of(pos, s.end(), delims.begin(), delims.end()); tokens.emplace_back(pos, end); pos = end + 1; } }