To change or create an additional sort key, you will need to create a new table and reconfigure it, since both steps cannot be performed for existing tables.
DynamoDB streams allow us to migrate tables without downtime. I did it very efficiently, and the following steps:
- Create a new table (let's call it NewTable) with the desired key structure, LSI, GSI.
- Enable DynamoDB Streams on source table
- Associate the lambda stream with the stream that places the entry in NewTable. (This lambda should clip the migration flag in step 5)
- [Optional] Create a GSI on the source table to speed up object scanning. Make sure that this GSI has only attributes: primary key and migrated (see Step 5).
Scan the GSI created in the previous step (or the entire table) and use the following filter:
FilterExpression = "attribute_not_exists (Migrated)"
Update each item in the table with the carry flag (ie, "Migrated": {"S": "0"}, which sends it to DynamoDB streams (using the UpdateItem API to ensure there is no data loss).
NOTE You might want to increase the recording capacity units in the table during updates.
- The lambda will collect all the items, clip the Migrated flag and paste it into NewTable.
- After transferring all elements, reassign the code to a new table.
- Delete the original table and the lambda function is once happy, all is well.
These steps should ensure that you do not have data loss or downtime.
I documented this on my blog using the code: https://www.abhayachauhan.com/2018/01/dynamodb-changing-table-schema/
Abhaya chauhan
source share