How can I speed up SVN updates?

Question

We have a pretty large SVN repository. SVN updates take longer and longer, the more we add code. We added svn:externals to folders that were repeated in some projects, such as FCKeditor on different sites. It helped, but not so.

What is the best way to reduce upgrade time and increase SVN speed?

Answer 1

If this is an older SVN repository (or even a completely new one, but has not been optimally configured), it may use the older style of the BDB repository database. http://svn.apache.org/repos/asf/subversion/trunk/notes/fsfs contains notes for the new one. To go from one to another isn; It’s too difficult to dump the entire history, reinitialize it with a new svn file system file and re-import. It may also be useful to filter the repo dump at the same time to remove entire checks of useless information (for example, I deleted the files of size 20 MB + tarball that someone registered).

As for the overall speed - a high-quality (fast) hard drive and additional memory for OS-based caching will be difficult to eliminate in terms of increasing the speed of SVN.

On the client side, if you have tortoisesvn installed via PuttyAgent to access SSH to the external repository machine, you can also enable SSH compression, which can also help.

Edit: SVN v1.5 also has the fsfs-reshard.py tool, which can help split FSFS based on svn repositories into a series of directories, which themselves can be connected to various master spindles. If you have thousands of changes, this can also help - if not for any other reason than finding one file among thousands, it takes time (and you will tell if this is a problem by looking at IOwait time)

Answer 2

Disable virus scanning in folders containing working code. This made my updates twice as fast.

Answer 3

This is actually not an answer, but it may be interesting to know that one of the reasons svn is that I / O-heavy is the fact that it stores one additional copy of each file in the .svn / text-base directory. This makes local diff operations fast, but eats up a lot of hard disk space and I / O.

http://subversion.tigris.org/issues/show_bug.cgi?id=525 contains information.

Answer 4

It looks like you have several projects in one repository. Separating them as needed will give you a big boost.

Presumably Git is much faster than Subversion due to the way it stores / processes changes, but I have no direct experience with it.

Answer 5

There are some general performance tweaks. SVN is a very heavy I / O, so faster hard drives are an option (on both ends). Add more memory to your server. Make sure your clients have a defragmented hard drive (for Windows).

Which access method you use also matters. Repositories stored on remote file systems (using the file: /// access) will be much slower than svnserve or Apache with mod_svn. Consider using one of the latter if you have a repository on a file share.

Answer 6

Make sure your connection to the server is as fast as it can be (gigabit ethernet). Make sure the server has fast disks in the array. And of course, just check what you need.

Answer 7

TotoiseSVN by default considers file changes in the background, and I saw that it slows down my machine. I changed the configuration to exclude everything, and then only include directories in which I have statements. You can also disable background checking. Both of these settings are in the "Icon Overlay" node settings.

Answer 8

I found in my own experience (that is: not through any real tests) that, especially if the SVN repo server is deleted, using external resources slows down. If you have duplicate code (for example, your FCK editor) in several places, I would be inclined to stick to external resources, since saving these files is synchronized and manageable more important than the update speed. However, you could use symbolic links to cast in duplicate code. (If you are using Windows XP, you can use junction points ).

Answer 9

We split our codebase into several sibling modules and wrote Ant scripts so that one developer could work on one module at a time without worrying about what was happening in the other modules.

• top-level assembly script runs all module assembly scripts.
• external libraries are not stored in Subversion, but rather pulled out of a network drive using Apache Ivy. (think of it as your own Maven repository).
• dependencies between modules are also managed using Ivy.

As a rule, developers need to update their entire tree a couple of times a week, but this is easy to do before heading out for a lunch / coffee break.

Answer 10

Using read permissions (i.e. restricting read access to specific individuals / groups) will significantly slow down the repository. Especially when authentication is performed in a certain way, for example, against a Windows domain. The same is true for write permissions, but writing is less frequent than reading. Restricting write access may be more important than restricting read access.

Answer 11

If you have many folders in the root directory of the repository, and your local copy reflects the repository, try cutting the monolithic local copy into many shared download folders and updating these folders separately, it will be really faster than one large folder.

Answer 12

Sometimes svn slow, especially with many external ones, is related to DNS. It looks like svn is doing a DNS lookup for every svn: external, even for relative ones. Adding your svn server hostname to / etc / hosts or fixing resolv.conf may be helpful.