How slow is Python and str.join string concatenation?

Question

As a result of the comments in my answer to this thread, I wanted to know that the speed difference between the += operator and ''.join()

So what is the speed comparison between the two?

Answer 1

From: Effective string concatenation

Method 1:

 def method1(): out_str = '' for num in xrange(loop_count): out_str += 'num' return out_str 

Method 4:

 def method4(): str_list = [] for num in xrange(loop_count): str_list.append('num') return ''.join(str_list) 

Now I understand that they are not strictly representative, and the 4th method joins the list before repeating and combining each element, but this is a fair indication.

String binding is much faster than concatenation.

Why? Rows are unchangeable and cannot be changed in place. To change one, you need to create a new view (concatenation of two).

Answer 2

My source code was wrong, it seems that concatenation + usually faster (especially with newer versions of Python on newer hardware)

The time is as follows:

 Iterations: 1,000,000 

Python 3.3 for Windows 7, Core i7

 String of len: 1 took: 0.5710 0.2880 seconds String of len: 4 took: 0.9480 0.5830 seconds String of len: 6 took: 1.2770 0.8130 seconds String of len: 12 took: 2.0610 1.5930 seconds String of len: 80 took: 10.5140 37.8590 seconds String of len: 222 took: 27.3400 134.7440 seconds String of len: 443 took: 52.9640 170.6440 seconds 

Python 2.7 on Windows 7, Core i7

 String of len: 1 took: 0.7190 0.4960 seconds String of len: 4 took: 1.0660 0.6920 seconds String of len: 6 took: 1.3300 0.8560 seconds String of len: 12 took: 1.9980 1.5330 seconds String of len: 80 took: 9.0520 25.7190 seconds String of len: 222 took: 23.1620 71.3620 seconds String of len: 443 took: 44.3620 117.1510 seconds 

On Linux Mint, Python 2.7, a slower processor

 String of len: 1 took: 1.8840 1.2990 seconds String of len: 4 took: 2.8394 1.9663 seconds String of len: 6 took: 3.5177 2.4162 seconds String of len: 12 took: 5.5456 4.1695 seconds String of len: 80 took: 27.8813 19.2180 seconds String of len: 222 took: 69.5679 55.7790 seconds String of len: 443 took: 135.6101 153.8212 seconds 

And here is the code:

 from __future__ import print_function import time def strcat(string): newstr = '' for char in string: newstr += char return newstr def listcat(string): chars = [] for char in string: chars.append(char) return ''.join(chars) def test(fn, times, *args): start = time.time() for x in range(times): fn(*args) return "{:>10.4f}".format(time.time() - start) def testall(): strings = ['a', 'long', 'longer', 'a bit longer', '''adjkrsn widn fskejwoskemwkoskdfisdfasdfjiz oijewf sdkjjka dsf sdk siasjk dfwijs''', '''this is a really long string that so long it had to be triple quoted and contains lots of superflous characters for kicks and gigles @!#(*_#)(*$(*!#@&)(*E\xc4\x32\xff\x92\x23\xDF\xDFk^%#$!)%#^(*#''', '''I needed another long string but this one won't have any new lines or crazy characters in it, I'm just going to type normal characters that I would usually write blah blah blah blah this is some more text hey cool what crazy is that it looks that the str += is really close to the O(n^2) worst case performance, but it looks more like the other method increases in a perhaps linear scale? I don't know but I think this is enough text I hope.'''] for string in strings: print("String of len:", len(string), "took:", test(listcat, 1000000, string), test(strcat, 1000000, string), "seconds") testall() 

Answer 3

This is what stupid programs are for testing :)

Use plus

 import time if __name__ == '__main__': start = time.clock() for x in range (1, 10000000): dog = "a" + "b" end = time.clock() print "Time to run Plusser = ", end - start, "seconds" 

Output:

 Time to run Plusser = 1.16350010965 seconds 

Now with join ....

 import time if __name__ == '__main__': start = time.clock() for x in range (1, 10000000): dog = "a".join("b") end = time.clock() print "Time to run Joiner = ", end - start, "seconds" 

Exit:

 Time to run Joiner = 21.3877386651 seconds 

So, on python 2.6 on windows, I would say that + is about 18 times faster than join :)

Answer 4

I rewrote the last answer, can you please share your opinion on how I tested?

 import time start1 = time.clock() for x in range (10000000): dog1 = ' and '.join(['spam', 'eggs', 'spam', 'spam', 'eggs', 'spam','spam', 'eggs', 'spam', 'spam', 'eggs', 'spam']) end1 = time.clock() print("Time to run Joiner = ", end1 - start1, "seconds") start2 = time.clock() for x in range (10000000): dog2 = 'spam'+' and '+'eggs'+' and '+'spam'+' and '+'spam'+' and '+'eggs'+' and '+'spam'+' and '+'spam'+' and '+'eggs'+' and '+'spam'+' and '+'spam'+' and '+'eggs'+' and '+'spam' end2 = time.clock() print("Time to run + = ", end2 - start2, "seconds") 

NOTE. This example is written in Python 3.5, where range () acts like the old xrange ()

I got the output:

 Time to run Joiner = 27.086106206103153 seconds Time to run + = 69.79100515996426 seconds 

Personally, I prefer '.join ([]) along the Plusser path because it is cleaner and more readable.