Error "from memory" with mechanization

Question

I tried to clear some information from page by page, page by page, basically here, what I did:

import mechanize
MechBrowser = mechanize.Browser()

Counter = 0

while Counter < 5000:
    Response = MechBrowser.open("http://example.com/page" + str(Counter))
    Html = Response.read()
    Response.close()

    OutputFile = open("Output.txt", "a")
    OutputFile.write(Html)
    OutputFile.close()

    Counter = Counter + 1

Well, the above codes ended up throwing an "Out of memory" error and in the task manager it shows that the script used several minutes at 1 GB after several hours of work ... how did it happen ?!

Will someone tell me what went wrong?

Answer 1

This is not exactly a memory leak, but an undocumented function. In principle, it mechanize.Browser()collectively saves the entire browser history in memory as it appears.

If you add a call MechBrowser.clear_history()after Response.close(), it should solve the problem.