Geek Answers Handbook

With stream in linux?

Does anyone have a simple example of streaming in c?

I want to create a small console application that will read the txt file file one by one, and then use the streams to process the entire txt. How can I do it? breaking txt into X, where X = N threads, this is the first thing that comes to my mind, is there a better way?

+7
c multithreading linux
source share
6 answers

Search for POSIX threads, also known as pthreads. The tutorial is here

+6
source share

The best IMHO option is to use POSIX streams. You can see more details HERE .

Also check out the link in James's answer.

+8
source share

Search pthreads. I'm new too. Here is a code snippet for a sum of 1 to 1,000,000,000 (also my first pthread work program).

#include <stdio.h> #include <pthread.h> struct arg { int a, b; int *rst; }; typedef struct arg arg; void* sum(void *); int main() { pthread_t sum1, sum2; int s1, s2; pthread_create(&sum1, NULL, sum, &(arg){1, 500000000, &s1}); pthread_create(&sum2, NULL, sum, &(arg){500000001, 1000000000, &s2}); pthread_join(sum1, NULL); pthread_join(sum2, NULL); printf("%d\n", s1 + s2); } void* sum(void *ptr) { int i, temp = 0; arg *x = ptr; for(i = x->a; i <= x->b; ++i) temp += i; *(x->rst) = temp; }
+7
source share

If you need an easy way, OpenMP is a powerful multi-threaded library supported by gcc.

  #omp parallel for for(i=0; i<1000; i++){ a[i] = b[i] + c[i]; }

This will result in a simple addition of two arrays and save the result in "a", but 4 threads will be created on the quad-core processor to process it (8 if hyper-threading is supported).

Easy multi-core Linux programming. :)

Finn travel guide: http://bisqwit.iki.fi/story/howto/openmp/

+3
source share

splitting txt into X, where X = N of threads, this is the first thing that comes to my mind, is there a better way?

It depends on your application.

  • Topics can help if data interpretation is a bottleneck, performance gains will be limited by file I / O speed.
  • Threads will not help if file reading is a bottleneck, disk I / O is limited by hardware and will only get worse if more threads are requesting data

If the interpretation of the information takes a lot of time, you can use something like the manufacturer’s manufacturer’s template and check yourself how many threads you need. (try a low number and see how many of them will give you maximum performance). Some examples can be found here and here.

As other answers point out, you can use pthreads to implement streaming.

+1
source share

First you need to ask yourself if you really need multithreading here. You need a general state between threads, for example. Does it analyze information from all URLs in the same data structure? If not, there may be enough processes (fork). Or you may not even go this far and simply use event-based programming (glib, libev).

Glib can be worth your while, even if you decide to use streams in the end, as it has a decent stream abstraction, including stream pools. This will make it easier to split your file, as you simply create the X thread pools, then add the dl / parse pools to one of them (line pool size is #%).

If it comes to speeding up downloads, your http library may already have related features. For curl there is a bunch of curl_multicalls with interesting here .

+1
source share

More articles:


All Articles