In C++ STL, there is no thread function to split the string, but the function is very common. Therefore, this paper introduces several string segmentation methods.

Using the strtok function in C

The strtok function in C can split C-style strings. Strtok_ris the thread safe version of strtok.

Reference resources https://linux.die.net/man/3/strtok_r and On the key points and implementation principles of the functions strtok and strtok UUR.

#include <string.h>

// str: string to split
// delim: separator as a split point
// saveptr: in thread safe version, the pointer used to save the split point location

char* strtok(char* str, const char* delim);

char* strtok_r(char* str, const char* delim, char** saveptr);

When STR is not NULL, search the first legal delimiter from the beginning, then replace the delimiter with '\ 0', save the location of the delimiter with a static variable (strtok version) or an incoming variable (strtok \ u t version), and return str. Because C-style strings end with '\ 0', you can get the first split string after calling the function.

Later, if you need to continue to split STR, you don't need to pass in str again, just pass in NULL. Other parameters are the same as the first call.

Implementation of split function:

// str: string to split
// Result: a string array to hold the split result
// delim: delimited string
void split(const std::string& str, 
           std::vector<std::string>& tokens, 
           const std::string delim = " ") {
    tokens.clear();
    
    char* buffer = new char[str.size() + 1];
    std::strcpy(buffer, str.c_str());

    char* tmp;
    char* p = strtok_r(buffer, delim.c_str(), &tmp); // First split
    do {
        tokens.push_back(p); // If p is nullptr, the entire string is taken as the result
    } while ((p = strtok_r(nullptr, delim.c_str(), &tmp)) != nullptr);
    // Strtok? R is the thread safe version of strtok.
}

Note: strtok and strtok_r will modify the passed string str, so you need to make a backup before calling them.

Flow in C + +

With istringstream and getline, you can split a string. However, there is a limitation: getline can only accept a char delimiter.

Implementation of split function:

void split(const std::string& str, 
           std::vector<std::string>& tokens, 
           const char delim = ' ') {
    tokens.clear();
    
    std::istringstream iss(str);
    std::string tmp;
    while (std::getline(iss, tmp, delim)) {
        if (tmp != "") {
            // tmp = = "" if two separators are adjacent, ignored.
            tokens.emplace_back(std::move(tmp));
        }
    }
}

C + + stream iterator

If the separator is a whitespace defined by the flow, there is a simpler implementation. Stream iterator istream & iterator Each iteration splits a string (separated by a space character).

Implementation of split function:

// str: string to split
// tokens: a string array to hold the split result
void split(const std::string& str, 
           std::vector<std::string>& tokens) {
    tokens.clear();

    std::istringstream iss(str);
    std::copy(std::istream_iterator<std::string>(iss), 
              std::istream_iterator<std::string>(),
              std::back_inserter(tokens));
}

find_first_not_of,find_first_of

Use find first not of to find the starting point of the split string start, use find first of to find the position of the separator, then [start, position] is to split into a string.

Repeat the above process until position = = STD:: String:: NPOs & & start = = STD:: String:: NPOs, then the segmentation ends.

Implementation of split function:

void split(const std::string& str, 
           std::vector<std::string>& tokens, 
           const std::string delim = " ") {
    tokens.clear();
    
    auto start = str.find_first_not_of(delim, 0);       // The first character of the split string
    auto position = str.find_first_of(delim, start);    // Location of separator
    while (position != std::string::npos || start != std::string::npos) {
        // [start, position] is the split string
        tokens.emplace_back(std::move(str.substr(start, position - start)));
        start = str.find_first_not_of(delim, position);
        position = str.find_first_of(delim, start);
    }
}

More

You can also use C++ 11 regular expression The library or C++20 provides ranges with a special split view. You can split strings by writing STR split (delim). See in detail Why does C + + string not provide split function? -Answer to the wind from the forest dark grass - Zhihu.

Source code

Github

Reference resources

• https://stackoverflow.com/questions/236129/how-do-i-iterate-over-the-words-of-a-string/237280#237280
• https://stackoverflow.com/questions/26328793/how-to-split-string-with-delimiter-using-c
• Why does C + + string not provide split function? -Answer to the wind from the forest dark grass - Zhihu
• Split a string