Building a dendrogram in a Scipy error for a large dataset

Question

I use Scipy for hierarchical clustering. I manage to get flat clusters on the doorstep using fcluster. But I need to visualize the generated dendrogram. When I use the dendrogram method, it works great for custom vectors of 5-6 thousand. But my dataser consists of 16k custom vectors. When I run it for 16k users, the dendrogram function raises the following error:

File "/home/enthought/lib/python2.7/site-packages/scipy/cluster/hierarchy.py", line 2333, in _dendrogram_calculate_info leaf_label_func, i, labels) File "/home/enthought/lib/python2.7/site-packages/scipy/cluster/hierarchy.py", line 2205, in _append_singleton_leaf_node ivl.append(str(int(i))) RuntimeError: maximum recursion depth exceeded while getting the str of an object 

Any ideas on visualizing the dendrogram for a larger document?

Answer 1

It may be a little late, but if you are comfortable with increasing the recursion limit to undermine the recursion depth limit, you can do it. This is not recommended, and definitely not "pythonic", but most likely you will get the desired results.

 import sys sys.setrecursionlimit(10000)