Bulk moving rows from one table to another using SQLAlchemy

Question

I have two identical tables postand old_post. I have a request that checks old messages. I would like to move the rows returned by the query to the table old_postand remove the rows from the table post.

I could solve this problem by repeating the results returned by the original query and updating my results this way, however I am worried that it is very inefficient and will start to cause me problems when I have more than 1000 rows. How can I effectively "move an array" from one table to another?

Answer 1

Request for old Posts. Bulk insert OldPostswith old data Posts'. Mass removal of the old Posts.

keys = db.inspect(Post).columns.keys()
get_columns = lambda post: {key: getattr(post, key) for key in keys}

posts = Post.query.filter(Post.expiry > ts)
db.session.bulk_insert_mappings(OldPost, (get_columns(post) for post in posts))
posts.delete()
db.session.commit()

The function get_columnstakes an instance Postand creates a dictionary from the column keys and values. Read the docs and warnings about using bulk insertand delete.

Answer 2

You can use Common Table Expressions in PostgreSQL

WITH moved_posts AS (
    DELETE FROM post
    WHERE expiry > time_stamp
    RETURNING *
)
INSERT INTO old_post
SELECT * from moved_posts

CTE support for DELETE will be added in SQLAlchemy 1.1. In the current release, you can execute raw SQL

from sqlalchemy import text

sql = text('''
WITH moved_posts AS (
    DELETE FROM post
    WHERE expiry > ?
    RETURNING *
)
INSERT INTO old_post
SELECT * from moved_posts
''')
db.session.execute(sql, [time_stamp])
db.session.commit()

In SQLAlchemy 1.1, it will look like

posts = Post.__table__
old_posts = OldPost.__table__
moved_posts = (
    posts.delete()
    .where(posts.c.expiry > ts)
    .returning(*(posts.c._all_columns))
    .cte('moved_posts'))
insert = (
    old_posts.insert()
    .from_select(
        [c.name for c in moved_posts.columns],
        moved_posts.select()
    ))
db.session.execute(insert)
db.session.commit()