Using subversion with a really really big site

Question

I am a big fan of subversive activities and am going to take on a large site (about 200 mb). I truncated the main site from the original size of 500 MB.

I am going to check this site in a new subversion repository. The problem is that my subversion repository is remotely hosted so that another colleague can also work on the site.

I worry that every time I have to do updates on the site, I need to register and output 200 MB.

Development is quite active, so there will be many changes on an ongoing basis.

Assuming I checked everything in order, will subversion guarantee that it only uploads new / fixed files / folders every time I do a new check, or will I wait 200 MB to download every time?

Answer 1

If I am mistaken after the first check / exit, you only process .diff files, so you only need to upload / download changes (and not the entire file, only the lines that were changed if the ASCII file) that exists between the files on the client / server.

The first comit / update will be terrible though.

Answer 2

Another thing to keep in mind is to make copies of your issued folders, and they will still be valid working copies:

svn checkout http://server/path/to/repos my_working_copy cp -a my_working_copy another_working_copy svn status another_working_copy 

This can save a lot of time / bandwidth if you need multiple working copies. It also makes branching and switching much faster.

 svn checkout http://server/path/to/trunk my_trunk cp -a my_trunk my_branch cd my_branch svn switch http://server/path/to/branches/stable 

As pointed out in other answers, you will only need to download the differences between the trunk and the branch.

Answer 3

As already mentioned, commit / update passes diff only quickly enough. Payouts are more time consuming - use the svn switch to quickly switch between branches.

In addition, the HTTP / WEBDAV transport protocol is not very efficient, especially when working with a large number of small files (for example, source code :)). You can use svnserve instead.

200 MB of data should not be too complicated for Subversion, but if disk space and efficient data transfer are really a problem, you can also learn git or mercurial. Especially git is much more efficient, but you probably need a little more time to wrap your head around the concepts of distributed source control, and you should live without visual graphical tools at the moment (also command line tools have become much more useful lately).

This link may be interesting: Website Update

Answer 4

I run sites about 5 GB or more in size. (and a build system that makes changes to many files for each build). Thus, the delta will be approximately 200 MB easily (and to the remote site). SVN does a great job of this. It also depends on how good your apache is (if you use apache).

Answer 5

If many changes often occur, why not include a cron entry that updates the subversion version to update the local copy, say, every 6 hours?

This way you get the latest differences (or not if it doesn't update after a few hours), and not the entire shebang.

EDIT : to clarify, if there are many changes, but only on a few pages at a time, any task / update will be small; if they will be created for all / most sites, it is often important to keep track of updates.

Answer 6

it will only post changes when updating or commenting. You must be fine.

Answer 7

Subversion only gets diffs / updates, so you only get a full check on first receive. You will receive the changes later.

To help with mergers, it would be nice to have two working copies - one pointed to the main code line, one pointed to your task thread. Thus, you do not need to switch the working copy from one Subversion subcategory to another, which can be expensive, for example, with code verification.

Answer 8

It will only download files that have been affected between this time. However, if you are going to branch out (as it should be), you can wait a long time.

What part of the project is really needed? I doubt there is a 200 mb source. If a lot of data is resources that change very rarely (i.e. Images), then you might consider splitting repositories into smaller projects.