Atoi in an array of characters with many integers

Question

I have a code in which an array of characters is populated with integers (converted to char arrays) and read by another function that returns it back to integers. I used the following function to convert to a char array:

char data[64]; int a = 10; std::string str = boost::lexical_cast<std::string>(a); memcpy(data + 8*k,str.c_str(),sizeof(str.c_str())); //k varies from 0 to 7 

and returning to the characters is done with:

 char temp[8]; memcpy(temp,data+8*k,8); int a = atoi(temp); 

This works well overall, but when I try to do this as part of a project involving qt (ver 4.7), it compiles fine and gives me segmentation errors when it tries to read using memcpy (). Please note that the segmentation error occurs only during the read cycle, and not when writing data. I do not know why this is happening, but I want to do it in any way.

So, are there any other other functions that I can use that can take in an array of characters, the first bit and the last bit and convert it to an integer. Then I would not need to use memcpy () at all. What I'm trying to do is something like this:

 new_atoi(data,8*k,8*(k+1)); // k varies from 0 to 7 

Thanks in advance.

Answer 1

You copy only 4 characters (depending on the width of your system pointer). This will cause numbers of 4+ non-zero characters to be completed, resulting in fluent lines at the input to atoi

  sizeof(str.c_str()) //ie sizeof(char*) = 4 (32 bit systems) 

it should be

  str.length() + 1 

Or characters will not be destroyed.

Only for STL:

^{make_testdata() : see all the way down}

Why aren't you using threads ...?

 #include <sstream> #include <iostream> #include <algorithm> #include <iterator> #include <string> #include <vector> int main() { std::vector<int> data = make_testdata(); std::ostringstream oss; std::copy(data.begin(), data.end(), std::ostream_iterator<int>(oss, "\t")); std::stringstream iss(oss.str()); std::vector<int> clone; std::copy(std::istream_iterator<int>(iss), std::istream_iterator<int>(), std::back_inserter(clone)); //verify that clone now contains the original random data: //bool ok = std::equal(data.begin(), data.end(), clone.begin()); return 0; } 

You can do it much faster in regular C with atoi / itoa and some settings, but I believe you should use binary transmission (see Boost Spirit Karma and protobuf for good libraries) if you need speed.

Boost Karma / Qi:

 #include <boost/spirit/include/qi.hpp> #include <boost/spirit/include/karma.hpp> namespace qi=::boost::spirit::qi; namespace karma=::boost::spirit::karma; static const char delimiter = '\0'; int main() { std::vector<int> data = make_testdata(); std::string astext; // astext.reserve(3 * sizeof(data[0]) * data.size()); // heuristic pre-alloc std::back_insert_iterator<std::string> out(astext); { using namespace karma; generate(out, delimit(delimiter) [ *int_ ], data); // generate_delimited(out, *int_, delimiter, data); // equivalent // generate(out, int_ % delimiter, data); // somehow much slower! } std::string::const_iterator begin(astext.begin()), end(astext.end()); std::vector<int> clone; qi::parse(begin, end, qi::int_ % delimiter, clone); //verify that clone now contains the original random data: //bool ok = std::equal(data.begin(), data.end(), clone.begin()); return 0; } 

If you wanted to do architecture-independent binary serialization, you would use this tiny adaptation that did things a million times faster (see table below ...):

 karma::generate(out, *karma::big_dword, data); // ... qi::parse(begin, end, *qi::big_dword, clone); 

Speed ​​Up Session

Best performance can be achieved by using Boost Serialization in binary mode:

 #include <sstream> #include <boost/archive/binary_oarchive.hpp> #include <boost/archive/binary_iarchive.hpp> #include <boost/serialization/vector.hpp> int main() { std::vector<int> data = make_testdata(); std::stringstream ss; { boost::archive::binary_oarchive oa(ss); oa << data; } std::vector<int> clone; { boost::archive::binary_iarchive ia(ss); ia >> clone; } //verify that clone now contains the original random data: //bool ok = std::equal(data.begin(), data.end(), clone.begin()); return 0; } 

Testdata h2>

(common to all versions above)

 #include <boost/random.hpp> // generates a deterministic pseudo-random vector of 32Mio ints std::vector<int> make_testdata() { std::vector<int> testdata; testdata.resize(2 << 24); std::generate(testdata.begin(), testdata.end(), boost::mt19937(0)); return testdata; } 

Benchmarks

I compared it

• using input 2<<24 (33554432) random integers
• do not display output (we do not want to measure the scroll performance of our terminal)
• rough timings were
  • Only the STL version is not so bad at 12.6s
  • The text version of Karma / Qi launched 5.1s in 18 seconds , thanks to Arlen's prompt generate_delimited :)
  • The binary version of Karma / Qi (big_dword) is only 1.4 seconds ( roughly 12x 3-4 times faster )
  • Boost Serialization takes a cake with 0.8 s (or when replacing text archives instead of binary files, about 13 seconds).

Answer 2

There is absolutely no reason for the text version of Karma / Qi to be slower than the STL version. I improved @ in the implementation of the text version of Karma / Qi to reflect this statement.

The next version of Boost Karma / Qi is more than twice as fast as the STL version:

 #include <boost/spirit/include/qi.hpp> #include <boost/spirit/include/karma.hpp> #include <boost/random.hpp> #include <boost/spirit/include/phoenix_core.hpp> #include <boost/spirit/include/phoenix_operator.hpp> #include <boost/spirit/include/phoenix_stl.hpp> namespace ascii = boost::spirit::ascii; namespace qi = boost::spirit::qi; namespace karma = boost::spirit::karma; namespace phoenix = boost::phoenix; template <typename OutputIterator> void generate_numbers(OutputIterator& sink, const std::vector<int>& v){ using karma::int_; using karma::generate_delimited; using ascii::space; generate_delimited(sink, *int_, space, v); } template <typename Iterator> void parse_numbers(Iterator first, Iterator last, std::vector<int>& v){ using qi::int_; using qi::phrase_parse; using ascii::space; using qi::_1; using phoenix::push_back; using phoenix::ref; phrase_parse(first, last, *int_[push_back(ref(v), _1)], space); } int main(int argc, char* argv[]){ static boost::mt19937 rng(0); // make test deterministic std::vector<int> data; data.resize(2 << 24); std::generate(data.begin(), data.end(), rng); std::string astext; std::back_insert_iterator<std::string> out(astext); generate_numbers(out, data); //std::cout << astext << std::endl; std::string::const_iterator begin(astext.begin()), end(astext.end()); std::vector<int> clone; parse_numbers(begin, end, clone); //verify that clone now contains the original random data: //std::copy(clone.begin(), clone.end(), std::ostream_iterator<int>(std::cout, ",")); return 0; }