MySQL workbench table data export extremely slow

I just downloaded the latest version of MySQL Workbench (6.3.6) and tried to export the remote table (to Google CloudSQL) to csv using the new table data export wizard. The table had about 600,000 rows and the final loaded size was about 75 MB. It took 7.5 hours.

I understand that I can use the Google Developer Console to perform this export (which I did and took about 15 seconds), but there seems to be something wrong with the MySQL Workbench. Could there be a problem with the configuration that causes this so slowly?

+6
source share
1 answer

Description : Workbench very slowly exports large datasets through the CSV Export Wizard. Disproportionately slow compromise with a smaller set. However, this is what I came across with .NET before.

How to repeat : Get a table with 15 or so records or more and export through the wizard. Notice how long it takes, and then export a subset of this data and see how the time spent does not correlate linearly with the number of rows.

Recommended Bugfix : Something I noticed while creating CSV export applications is that the MS.NET environment cannot handle huge strings very well and as a result tends to work poorly.

I found a solution though. When you create a huge line in a dump in a file, when you generate it, instead of creating one huge line and writing it to a file immediately after the export is complete, I get much better performance by making only a few hundred CSV lines generated in one times, write them to a file and flush the buffer onto which you wrote the generated data.

I would recommend writing to a temporary file and then renaming / moving it to the specified user when this is done. Write to temp and then move / rename is the way Photoshop and some other applications save their data. Both the x string and flushing entries I discovered during development are much faster than trying to get .NET to manage the 20 MB string.

+1
source

All Articles