Description : Workbench very slowly exports large datasets through the CSV Export Wizard. Disproportionately slow compromise with a smaller set. However, this is what I came across with .NET before.
How to repeat : Get a table with 15 or so records or more and export through the wizard. Notice how long it takes, and then export a subset of this data and see how the time spent does not correlate linearly with the number of rows.
Recommended Bugfix : Something I noticed while creating CSV export applications is that the MS.NET environment cannot handle huge strings very well and as a result tends to work poorly.
I found a solution though. When you create a huge line in a dump in a file, when you generate it, instead of creating one huge line and writing it to a file immediately after the export is complete, I get much better performance by making only a few hundred CSV lines generated in one times, write them to a file and flush the buffer onto which you wrote the generated data.
I would recommend writing to a temporary file and then renaming / moving it to the specified user when this is done. Write to temp and then move / rename is the way Photoshop and some other applications save their data. Both the x string and flushing entries I discovered during development are much faster than trying to get .NET to manage the 20 MB string.
source share