Changing Database Tables in Django

Question

I am considering using Django to launch a project (fyi, a browser-based game), and one of the features that I like most is to use syncdb to automatically create database tables based on the Django Model that I define (a feature that I can not find in any other structure). I already thought this was too good to be true when I saw this in the documentation :

Syncdb will not modify existing tables

syncdb will only create tables for models that are not yet installed. It never issues ALTER TABLE statements to match changes made to the model class after installation. Changes in model classes and database schemas are often associated with some form of ambiguity, and in these cases Django should have guessed the right changes. There is a risk that critical data will be lost in the process.

If you made changes to the model and want to modify the database tables to match, use the sql command to display the new SQL structure and compare it with the existing table schema to generate the changes.

It seems that modifying existing tables should be done "manually."

What I would like to know is the best way to do this. Two solutions come:

• As indicated in the documentation, make manual changes to the database:
• Make a backup of the database, wipe it, create the tables again (using syncdb, since now create tables from scratch) and import the backup data (this may take too much if the database is large)

Any ideas?

Answer 1

As indicated in other answers on the same topic, be sure to check out the DjangoCon 2008 schema evolution panel on YouTube.

In addition, two new projects on the map: Simplemigrations and Migratory .

Answer 2

In manual mode, SQL changes and dump / reload are both parameters, but you can also check out some of the schema evolution packages for Django. The most mature options are django-evolution and South .

EDIT : And here, dmigrations .

UPDATE . Since this answer was originally written, django-evolution and dmigrations ceased to evolve, and South became de facto to migrate the schema to Django. Parts of the South may even be integrated into Django in the next release or two.

UPDATE The Django 1.7+ schema is included in the South-based schema migration scheme (and by Andrew Godwin, author of the South).

Answer 3

One good way to do this is by using fixtures, especially initial_data fixtures.

A device is a collection of files containing serialized database content. So, it looks like backing up a database, but somehow Django knows about it easier to use and will have additional benefits when you come to do things like unit testing.

You can create a device from the data currently in your database using django-admin.py dumpdata . By default, the data is in JSON format, but other options, such as XML, are available. A good place to store fixtures is the fixtures subdirectory of application directories.

You can download the patch using django-admin.py loaddata , but more significantly, if your device has a name like initial_data.json , it will be automatically loaded when you run syncdb , saving the import problem yourself.

Another advantage is that when you run manage.py test to run tests per unit of time, the source data device will also be loaded in the time test database.

Of course, this will work when you add attributes to models and columns in the database. If you delete a column from the database, you will need to update your fixture to remove data for this column, which may be difficult.

This works best when many small database changes are made during development. To update production databases, a better manually generated SQL script may work.

Answer 4

I am using django-evolution. Cautions include:

• His automatic suggestions were evenly rotten; and
• The fingerprint function returns different values ​​for the same database on different platforms.

However, I found a convenient schema_evolution.py approach. To get around the fingerprint problem, I suggest code like:

 BEFORE = 'fv1:-436177719' # first fingerprint BEFORE64 = 'fv1:-108578349625146375' # same, but on 64-bit Linux AFTER = 'fv1:-2132605944' AFTER64 = 'fv1:-3559032165562222486' fingerprints = [ BEFORE, AFTER, BEFORE64, AFTER64, ] CHANGESQL = """ /* put your SQL code to make the changes here */ """ evolutions = [ ((BEFORE, AFTER), CHANGESQL), ((BEFORE64, AFTER64), CHANGESQL) ] 

If I had more fingerprints and changes, I would refactor it. Until then, making it cleaner, one could steal development time from something else.

EDIT: Given that I manually create my changes anyway, I will try dmigrations next time.

Answer 5

django-command-extensions is a django library that provides additional commands for manage.py. One of them is sqldiff, which should provide you with the sql needed to update your new model. This, however, is indicated as "very experimental."

Answer 6

So far in my company we have used a manual approach. What works best for you depends a lot on your development style.

Usually, we do not have many schema changes in production systems and several formalized deployments from development to production servers. Whenever we deploy (10-20 times a year), we make the difference between the current and the upcoming production industry, looking at all the code and noting what needs to be changed on the production server. The necessary changes may include additional dependencies, changes in the settings file and changes in the database.

This is very good for us. All this is automated - this is a niche vision, but difficult for us - maybe we could manage the migrations, but we still have to process an additional library, server and any dependencies.

Answer 7

The book Django explains how to do it manually.

http://www.djangobook.com/en/2.0/chapter10/

I have done this so many times, and it is very flexible, allowing you to leave data in tables, removing them from the model.

I recently started using the South. It seems a little less flexible (or maybe I just need to read the documents a little more.) But you can type a little text. Sometimes you manage to synchronize it with the database, which is a nightmare. Everything seems to be going well while you continue to use it. It seems to put models and the actual database together, which may or may not be a good thing depending on your situation.

Answer 8

Django 1.7 (currently under development) adds native support for schema migration using manage.py migrate and manage.py makemigrations ( migrate deprecates syncdb ).