Geek Answers Handbook

How to provide username / password to access a web resource using Matlab urlread / urlwrite?

Following this question regarding access to a PDF file on a web page using Matlab, which was originally buried behind the Javascript function. Now I have a URL that allows me to directly access the page, this works well using the Matlab web browser object (the PDF appears on the screen), but to save the PDF file for further processing, it seems to me that I need to use the Matlab urlread functions / urlwrite; However, these functions do not provide a method for providing credentials.

How to specify username / password for matlab urlread / urlwrite functions?

+5
source share
5 answers

The Matlab urlread () function has an argument of "params", but these are CGI style parameters that are encoded in the URL. Authentication is performed using lower-level HTTP request parameters. Urlread does not support them, but you can use the code directly for the Java URL class to use them.

You can also use the Sun class sun.misc.BASE64Encoder to programmatically encode Base 64. This is a non-standard class that is not part of the standard Java library, but you know that JVM delivery using Matlab will have it, so you can do without encoding.

Here is a quick hack showing it in action.

function [s,info] = urlread_auth(url, user, password)
%URLREAD_AUTH Like URLREAD, with basic authentication
%
% [s,info] = urlread_auth(url, user, password)
%
% Returns bytes. Convert to char if you're retrieving text.
%
% Examples:
% sampleUrl = 'http://browserspy.dk/password-ok.php';
% [s,info] = urlread_auth(sampleUrl, 'test', 'test');
% txt = char(s)

% Matlab urlread() doesn't do HTTP Request params, so work directly with Java
jUrl = java.net.URL(url);
conn = jUrl.openConnection();
conn.setRequestProperty('Authorization', ['Basic ' base64encode([user ':' password])]);
conn.connect();
info.status = conn.getResponseCode();
info.errMsg = char(readstream(conn.getErrorStream()));
s = readstream(conn.getInputStream());

function out = base64encode(str)
% Uses Sun-specific class, but we know that is the JVM Matlab ships with
encoder = sun.misc.BASE64Encoder();
out = char(encoder.encode(java.lang.String(str).getBytes()));

%%
function out = readstream(inStream)
%READSTREAM Read all bytes from stream to uint8
try
    import com.mathworks.mlwidgets.io.InterruptibleStreamCopier;
    byteStream = java.io.ByteArrayOutputStream();
    isc = InterruptibleStreamCopier.getInterruptibleStreamCopier();
    isc.copyStream(inStream, byteStream);
    inStream.close();
    byteStream.close();
    out = typecast(byteStream.toByteArray', 'uint8'); %'
catch err
    out = []; %HACK: quash
end
+6
source

urlwrite_auth is the next step, so here it is ...

function  [output,status]=urlwrite_auth(url, user, password,location,wanted) 
%URLWRITE_AUTH Like URLWRITE, with basic authentication 
% 
% location is where you want the file saved
% wanted is the name of the file you want
% Returns the output file which is now saved to location. 
% 
% Examples: 
% sampleUrl = 'http://browserspy.dk/password-ok.php'; 
% [output,status] = urlwrite_auth(sampleUrl, 'user', 'password', location, wanted); 


% Matlab urlread() doesn't do HTTP Request params, so work directly with Java 
jUrl = java.net.URL(url); 
conn = jUrl.openConnection(); 
conn.setRequestProperty('Authorization', ['Basic ' base64encode([user ':' password])]); 
conn.connect()
%note this calls the function below

% Specify the full path to the file so that getAbsolutePath will work when the
% current directory is not the startup directory and urlwrite is given a
% relative path.
file = java.io.File(location);

% the path.
try
    file = file.getCanonicalFile;
catch
    error('MATLAB:urlwrite:InvalidOutputLocation','Could not resolve file    "%s".',char(file.getAbsolutePath));
end

% Open the output file.
pathy=strcat(location,'\',wanted);
try
    fileOutputStream = java.io.FileOutputStream(pathy);
catch
    error('MATLAB:urlwrite:InvalidOutputLocation','Could not open output file "%s".',char(file.getAbsolutePath));
end

% Read the data from the connection.
try
    inputStream = conn.getInputStream;
        import com.mathworks.mlwidgets.io.InterruptibleStreamCopier; 
    % This StreamCopier is unsupported and may change at any time.
    isc = InterruptibleStreamCopier.getInterruptibleStreamCopier;   
    isc.copyStream(inputStream,fileOutputStream);
    inputStream.close;
    fileOutputStream.close;
    output = char(file.getAbsolutePath);
catch
    fileOutputStream.close;
    delete(file);
    if catchErrors, return
    else error('MATLAB:urlwrite:ConnectionFailed','Error downloading URL. Your network     connection may be down or your proxy settings improperly configured.');
    end
end

status = 1;


function out = base64encode(str) 
% Uses Sun-specific class, but we know that is the JVM Matlab ships with 
encoder = sun.misc.BASE64Encoder(); 
out = char(encoder.encode(java.lang.String(str).getBytes())); 
%this is the bit of code that makes it connect!!!!

, http.

+1

, , Matlab , , Mathworks , . Firebug, Base64, , . PDF- - . ...

, get post , .

0

: - webread, .

options = weboptions('Username','user','Password','your password');
data = webread(url, options);

websave webwrite. weboptions

0

matlab, .

:

s = urlread('url','method','params')

, , , post.

// Params is supposed to be a "cell array of name/value pairs, I don't know matlab... 
s = urlread('http://whatever.com','post', {'username' 'ian'; 'password' 'awesomepass'})

You will need to look at the actual HTML request form or view the net tab in firebug to find out what the actual name / values ​​of its username and password parameters are.

-2
source

More articles:


All Articles