Geek Answers Handbook

Big POST data is corrupted when using Django / PyISAPIe / IIS

I had a problem with big POST data (> 16384 bytes) when using Django 1.2.3, PyISAPIe v1.1.0-rc4 and IIS 7.5.

For example, when sending approx. 60 KB of form data using POST, the following happens:

  • The first 16 KB POST data block is correct.
  • The next 16kB block is a repetition of the first block
  • The next 16kB is another repeat of the first block
  • The rest (<16kB) are again correct.

Interestingly, when using content-type="multipart/form-data" it works fine.

Using this information, I found the likely location of the error in WSGIRequest._get_raw_post_data in django \ core \ handlers \ wsgi.py, which handles content-type="multipart/form-data" separately from the default case (no content).

Both cases self.environ['wsgi.input'] with self.environ['wsgi.input'] , which is installed in the PyISAPIe object. The difference is that the default case seems to be read in 16 kB chunks, while the multiprocessor handler seems to read the chunks a little less than 2 GB.

I don’t know enough about C and the Python interface for C to dig further, but I assume that the error is somewhere in PyISAPIe in the ReadClient function in ReadWrite.cpp.

My current solution is to add content-type="multipart/form-data" to forms that can handle more than 16 KB of data.

Does anyone come across this, or does anyone know how to determine if the error is really in PyISAPIe?

Thanks!

+2
source share
2 answers

Here is the author of PyISAPIe.

This has been fixed in revision 184 in the repository, but not in the downloadable release, as described on the mailing list .

He turned to a previously documented error, which, apparently, did not receive much attention, because many users check the source and not download the package. Or, that is my best guess anyway; regardless, I plan on providing a downloadable version of the fixed code.

Thank you for drawing attention to this, so I can remind you that this project is being released in a healthy state.

+3
source

I went a little deeper and I think I found the problem.

In PyISAPIe \ Readwrite.cpp:

 PyISAPIe_Func(DWORD) ReadClient( Context &Ctx, DWORD Length, void *const Data ) { if ( !Length ) Length = Ctx.ECB->cbTotalBytes; if ( !Data ) // Return the size of the the data that would be read return min(Length, Ctx.ECB->cbTotalBytes); DWORD Ret, Total = 0; if ( Length > Ctx.ECB->cbAvailable ) { [...snip...] } else { memcpy(Data, Ctx.ECB->lpbData, Length); Ctx.ECB->cbTotalBytes -= Length; Ctx.ECB->cbAvailable -= Length; return Length; }

If a method is called multiple times with a length <= Ctx.ECB-> cbAvailable, it always copies the beginning of the Ctx.ECB-> lpbData buffer to Data and does not delete this data from the buffer or the pointer does not advance. Only when the data is exhausted (cbAvailable == 0), are new data correctly read in Data later in the code.

I don’t know how to fix it, but at least I can get around it by reading large enough pieces of data so that one piece reads everything.

+1
source

More articles:


All Articles