Failed to execute python request request: cookies?

Question

I try to execute this post , but I get server error 500 :

 import requests base_url = "https://www.assurland.com/ws/CarVehiculeSearch.asmx" url = "%s/%s"% (base_url,"GetCarBodyTypeListByCarAlim") pars ={"CarAlim":"DIES","CarType": "A7", "CodeMake": "AUDI", "FirstDrivingDate": "2015-09-22"} with requests.Session() as s: r = s.post(url,data=pars) print r.status_code ## 500 

I think I need to set cookies or something else.

Thanks in advance for any help.

Answer 1

Most of the API access points on this site seem to be broken. There is nothing wrong with the code. I can not get the site to respond to any requests in its own web interface, even when using a normal browser.

basic API documentation (in French) that you can also use the SOAP protocol to create queries; Python has several SOAP client libraries that you could choose from.

However, I was not lucky that it happened. Using the excellent zeep library I tried to access the simpler GetMainCarMakeListByFirstDrivingDate endpoint (note that I had to use the datetime() object to model the timestamp):

 >>> from zeep import Client >>> client = Client('https://www.assurland.com/ws/CarVehiculeSearch.asmx?WSDL') >>> client.service.GetMainCarMakeListByFirstDrivingDate(FirstDrivingDate=datetime(2015, 9, 22)) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/mjpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/site-packages/zeep/client.py", line 25, in __call__ self._op_name, args, kwargs) File "/Users/mjpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/site-packages/zeep/wsdl/bindings/soap.py", line 109, in send return self.process_reply(client, operation_obj, response) File "/Users/mjpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/site-packages/zeep/wsdl/bindings/soap.py", line 145, in process_reply return self.process_error(doc, operation) File "/Users/mjpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/site-packages/zeep/wsdl/bindings/soap.py", line 223, in process_error detail=fault_node.find('detail')) zeep.exceptions.Fault: <exception str() failed> 

a Fault is the SOAP equivalent of error 500.

When I turn on debug logging , we see that the server is having problems receiving a response:

 >>> client.service.GetMainCarMakeListByFirstDrivingDate(FirstDrivingDate=datetime(2015, 9, 22)) zeep.transports: HTTP Post to https://www.assurland.com/ws/CarVehiculeSearch.asmx: <?xml version='1.0' encoding='utf-8'?> <soap-env:Envelope xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/"> <soap-env:Body> <ns0:GetMainCarMakeListByFirstDrivingDate xmlns:ns0="http://tempuri.org/"> <ns0:FirstDrivingDate>2015-09-22T00:00:00</ns0:FirstDrivingDate> </ns0:GetMainCarMakeListByFirstDrivingDate> </soap-env:Body> </soap-env:Envelope> zeep.transports: HTTP Response from https://www.assurland.com/ws/CarVehiculeSearch.asmx (status: 500): <?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><soap:Fault><faultcode>soap:Server</faultcode><faultstring>Le serveur n'a pas pu traiter la demande. ---&gt; Erreur lors de la génération du document XML. ---&gt; Le type common.FormDataListItem n'était pas attendu. Utilisez l'attribut XmlInclude ou SoapInclude pour spécifier les types qui ne sont pas connus statiquement.</faultstring><detail /></soap:Fault></soap:Body></soap:Envelope> Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/mjpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/site-packages/zeep/client.py", line 25, in __call__ self._op_name, args, kwargs) File "/Users/mjpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/site-packages/zeep/wsdl/bindings/soap.py", line 109, in send return self.process_reply(client, operation_obj, response) File "/Users/mjpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/site-packages/zeep/wsdl/bindings/soap.py", line 145, in process_reply return self.process_error(doc, operation) File "/Users/mjpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/site-packages/zeep/wsdl/bindings/soap.py", line 223, in process_error detail=fault_node.find('detail')) zeep.exceptions.Fault: <exception str() failed> 

Hidden in error message:

Track. ---> Erreur lors de la génération du document XML. ---> Le type common.FormDataListItem n'était pas attendu. Utilisez l'attribut XmlInclude ou SoapInclude pour spécifier les types qui ne sont pas connus statiquement.

or, with any help from Google Translate, in English:

The server could not process the request. ---> Error creating XML document. ---> type common.FormDataListItem was not expected. Use the XmlInclude or SoapInclude attributes to indicate types that are not known statically.

Since we are not sending any types of common.FormDataListItem , and the message complains that the XML document could not be generated, it looks like a server-side programming error.

There are methods that work, either through SOAP:

 >>> client.service.GetCarTypeListByCodeMake(CodeMake='BMW', FirstDrivingDate=datetime(2016, 1, 1)) ['I3', 'I8', 'M2', 'M3', 'M4', 'M5', 'M6', 'SERIE 1 II', 'SERIE 2', 'SERIE 3 VI', 'SERIE 4', 'SERIE 5', 'SERIE 6', 'SERIE 7', 'X1', 'X3', 'X4', 'X5', 'X6', 'Z4'] >>> client.service.GetAllCarTypeListByCodeMake(CodeMake='BMW') ['1502', '1600', '1602', '1800', '1802', '2000', '2002', '2500', '3,0', '3,3', '315', '316', '318', '320', '323', '324', '325', '328', '330', '518', '520', '523', '524', '525', '528', '530', '535', '540', '545', '550', '628', '630', '633', '635', '645', '650', '725', '728', '730', '732', '733', '735', '740', '745', '750', '760', '840', '850', 'I3', 'I8', 'L7', 'M2', 'M3', 'M4', 'M5', 'M535', 'M6', 'M635', 'SERIE 1', 'SERIE 1 II', 'SERIE 2', 'SERIE 3', 'SERIE 3 (SUITE)', 'SERIE 3 VI', 'SERIE 4', 'SERIE 5', 'SERIE 6', 'SERIE 7', 'X1', 'X3', 'X4', 'X5', 'X6', 'Z1', 'Z3', 'Z4', 'Z8'] 

or by sending application/x-www-form-urlencoded data using requests :

 >>> response = requests.post('https://www.assurland.com/ws/CarVehiculeSearch.asmx/GetCarTypeListByCodeMake', data={'CodeMake': 'BMW', 'FirstDrivingDate': datetime(2016, 1, 1)}) >>> print(response.text) <?xml version="1.0" encoding="utf-8"?> <ArrayOfAnyType xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://tempuri.org/"> <anyType xsi:type="xsd:string">I3</anyType> <anyType xsi:type="xsd:string">I8</anyType> <anyType xsi:type="xsd:string">M2</anyType> <anyType xsi:type="xsd:string">M3</anyType> <anyType xsi:type="xsd:string">M4</anyType> <anyType xsi:type="xsd:string">M5</anyType> <anyType xsi:type="xsd:string">M6</anyType> <anyType xsi:type="xsd:string">SERIE 1 II</anyType> <anyType xsi:type="xsd:string">SERIE 2</anyType> <anyType xsi:type="xsd:string">SERIE 3 VI</anyType> <anyType xsi:type="xsd:string">SERIE 4</anyType> <anyType xsi:type="xsd:string">SERIE 5</anyType> <anyType xsi:type="xsd:string">SERIE 6</anyType> <anyType xsi:type="xsd:string">SERIE 7</anyType> <anyType xsi:type="xsd:string">X1</anyType> <anyType xsi:type="xsd:string">X3</anyType> <anyType xsi:type="xsd:string">X4</anyType> <anyType xsi:type="xsd:string">X5</anyType> <anyType xsi:type="xsd:string">X6</anyType> <anyType xsi:type="xsd:string">Z4</anyType> </ArrayOfAnyType> 

You will probably have to contact the developers of this API to figure it out, this is not something you can get to work on the Python side.

Answer 2

The problem is the date format, which is poorly processed on the site.

This 500 error carries this description:

 Erreur lors de la génération du document XML. ---> Le type common.FormDataListItem n'était pas attendu. Utilisez l'attribut XmlInclude ou SoapInclude pour spécifier les types qui ne sont pas connus statiquement. 

Try using it in your browser on your page for more convenient debugging: https://www.assurland.com/ws/CarVehiculeSearch.asmx?op=GetCarBodyTypeListByCarAlim

Note: " XmlInclude ou SoapInclude attribute . Pour spécifier les types qui ne sont pas connus statiquement"

So, I tried a little search to enter a date in a different format.

As an example:

 Fri, 19 Jan 2001 

or 19 Jan 2001

You get a 200 OK request - in this case, an empty result.

Unfortunately, after a certain year, it seems that the format is also not accepted: September 22, 2001 gives 200 OK, but September 22, 2015 no more - it goes until 2008-12-31.

Try a different datetime format ... But there seems to be something there ... take another site :)