Reading data in MATLAB from a text file

Question

I have a text file with the following structure:

1999-01-04 1,100.00 1,060.00 1,092.50 0 6,225 1,336,605 37 1999-01-05 1,122.50 1,087.50 1,122.50 0 3,250 712,175 14 ... 

The file contains repeating sets of eight values ​​(a date followed by seven numbers, each in its own line).

I want to read it in MATLAB and get the values ​​in different vectors. I tried to accomplish this using several different methods, but no one worked - everything outputs some kind of error.

In case this is important, I do it on a Mac.

Answer 1

EDIT: this is a shorter version of the code I previously had in my answer ...

If you want to read your data file directly without first programming it like dstibbe , the following should work:

 fid = fopen('datafile.txt','rt'); data = textscan(fid,'%s %s %s %s %s %s %s %s','Delimiter','\n'); fclose(fid); data = [datenum(data{1}) cellfun(@str2double,[data{2:end}])]'; 

The code above puts each set of 8 values ​​in a matrix of size 8 by N, and N is the number of 8 sets of rows in the data file. The date is converted to a serial date number so that it can be included with other double-precision values ​​in the matrix. The following functions (used in the above code) are of interest : TEXTSCAN , DATENUM , CELLFUN , STR2DOUBLE .

Answer 2

I propose another solution. This code is the shortest in MATLAB code. Using sed first, we format the file as a CSV file (shared comma, each record on the same line):

 cat a.dat | sed -e 's/,//g ; s/[ \t]*$/,/g' -e '0~8 s/^\(.*\),$/\1\n/' | sed -e :a -e '/,$/N; s/,\n/,/; ta' -e '/^$/d' > file.csv 

The explanation . First, we get rid of thousands of comma delimiters and trim the spaces at the end of each line, adding a comma. But then we remove this trailing comma for every eighth line. Finally, we join the rows and delete the empty ones.

The result will look like this:

 1999-01-04,1100.00,1060.00,1092.50,0,6225,1336605,37 1999-01-05,1122.50,1087.50,1122.50,0,3250,712175,14 

Further in MATLAB, we simply use textscan to read each line: the first field as a string (to convert to num), and the rest as numbers:

 fid = fopen('file.csv', 'rt'); a = textscan(fid, '%s %f %f %f %f %f %f %f', 'Delimiter',',', 'CollectOutput',1); fclose(fid); M = [datenum(a{1}) a{2}] 

and the resulting matrix M is equal to:

  730124 1100 1060 1092.5 0 6225 1336605 37 730125 1122.5 1087.5 1122.5 0 3250 712175 14 

Answer 3

Use a script to change the text file to what Matlab can read.

eg. make it a matrix:

 M = [ 1999-01-04 1,100.00 1,060.00 1,092.50 0 6,225 1,336,605; <-- notice the ';' 37 1999-01-05 1,122.50 1,087.50 1,122.50 0 3,250; <-- notice the ';' 712,175 14 ... ] 

import this into matlab and read the various vectors from the matrix.

Note: my matlab is a little rusty. Errors are possible.

Answer 4

It is not clear in what form you want the data to be if you read it. The code below puts everything in one matrix, with each line representing a group of 8 lines in a text file. You can use different variables for different columns or (if you have access to the Statistics toolbar), use an array of the data set.

 % Read file as text text = fileread('c:/data.txt'); % Split by line x = regexp(text, '\n', 'split'); % Remove commas from numbers x = regexprep(x, ',', '') % Number of items per object n = 8; % Get dates index = 1:length(x); dates = datenum(x(rem(index, n) == 1)); % Get other numbers nums = str2double(x(rem(index, n) ~= 1)); nums = reshape(nums, (n-1), length(nums)/(n-1))'; % Combine dates and numbers thedata = [dates nums]; 

You can also learn the textscan function for alternative ways to solve the problem.

Answer 5

Looks like Richie. Using str2double to convert file strings to paired. This implementation is processed line by line rather than splitting the file into a regular expression. The output is an array of cells of individual vectors.

 function vectors = readdata(filename) fid=fopen(filename); tline = fgetl(fid); counter = 0; vectors = cell(7,1); while ischar(tline) disp(tline) if counter > 0 vectors{counter} = [vectors{counter} str2double(tline)]; end counter = counter + 1 if counter > 7 counter = 0; end tline = fgetl(fid); end fclose(fid); 

Answer 6

This is a regular expression check to make sure your data is formatted well.

  fid = fopen ('data.txt', 'rt');

 % these will be your 8 value arrays
 val1 = [];
 val2 = [];
 val3 = [];
 val4 = [];
 val5 = [];
 val6 = [];
 val7 = [];
 val8 = [];

 linenum = 0;  % line number in file
 valnum = 0;  % number of value (1-8)

 while 1
    line = fgetl (fid);
    linenum = linenum + 1;
    if valnum == 8
       valnum = 1;
    else
       valnum = valnum + 1;
    end

     % - if reached end of file, end
     if isempty (line) |  line == -1
       fclose (fid);
       break;
    end


    switch valnum
       case 1
          pat = '(? \ d {4}) - (? \ d {2}) - (? \ d {2})';  % val1 (eg 1999-01-04)
       case 2
          pat = '(? \ d * [,] * \ d * [,] * \ d * [.] \ d {2})';  % val2 (eg 1,100.00) [valid up to 1billion-1]
       case 3
          pat = '(? \ d * [,] * \ d * [,] * \ d * [.] \ d {2})';  % val3 (eg 1,060.00) [valid up to 1billion-1]
       case 4
          pat = '(? \ d * [,] * \ d * [,] * \ d * [.] \ d {2})';  % val4 (eg 1,092.50) [valid up to 1billion-1]
       case 5
          pat = '(? \ d +)';  % val5 (eg 0)
       case 6
          pat = '(? \ d * [,] * \ d * [,] * \ d +)';  % val6 (eg 6,225) [valid up to 1billion-1]
       case 7
          pat = '(? \ d * [,] * \ d * [,] * \ d +)';  % val7 (eg 1,336,605) [valid up to 1billion-1]
       case 8
          pat = '(? \ d +)';  % val8 (eg 37)
       otherwise
          error ('bad linenum')
    end

    l = regexp (line, pat, 'names');  % l is for line
     if length (l) == 1% match
       if valnum == 1
          serialtime = datenum (str2num (l.yr), str2num (l.mo), str2num (l.dy));  % convert to matlab serial date
          val1 = [val1; serialtime];
       else
          this_val = strrep (l.val, ',', '');  % strip out comma and convert to number
          eval (['val', num2str (valnum), '= [val', num2str (valnum), ';', this_val, '];'])% save this value into appropriate array
       end
    else
       warning (['line number', num2str (linenum), 'skipped! [didnt pass regexp]:', line]);
    end
 end