What I'm really trying to accomplish is changing the data types for multiple columns. In particular, several columns that are stored as rows that must be dates. I tried to use the CREATE TABLE / SELECT command with similar problems. I / O errors or broken connections.
I realized that it would be more efficient to export and re-import the data than trying to abuse the cluster by reading / writing so much data in a circle more or less.
I tried both Aginy Workbench and SQL Workbench J with large timeouts (10 hours). SQL Workbench J was able to run for 2 hours and 12 minutes today before failing with the same error that I continue to see again and again.
An I / O error occurred while sending to the backend.
This is a fair piece of data ... 2,028,448,405 rows at present (I say "now" because we add about 70 million rows per day). But I would expect Redshift to handle this easily.
UNLOAD
(
'select
weekday,
day,
month,
year,
guid,
...,
colN
from actions a
where a.colN in (\ 'my \', \ 'penguin \', \ 'lolz \') '
)
TO 's3: //penguin-lolz-bucket/all.csv'
CREDENTIALS 'aws_access_key_id = $ id; aws_secret_access_key = $ key
ALLOWOVERWRITE; </code>
- .
1:
, 3 :
, / .
2:
. ( , 1)
PRO TIP. , AWS Redshift ( "", ). SQL Workbench J. 5 SQL Workbench J , . AWS Redshift , , .