Does the Guava Bloom filter not support large inserts?

Question

I used BloomFilter in guava v.11.0.1, and it seems that I get an exception when my insert is big. I tried 10 million with 0.001 fpp and it failed.

java.lang.IllegalArgumentException: Number of bits must be positive at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88) at com.google.common.hash.BloomFilterStrategies.checkPositiveAndMakeMultipleOf64(BloomFilterStrategies.java:72) at com.google.common.hash.BloomFilterStrategies.access$000(BloomFilterStrategies.java:18) at com.google.common.hash.BloomFilterStrategies$From128ToN.withBits(BloomFilterStrategies.java:37) at com.google.common.hash.BloomFilter.create(BloomFilter.java:192) at com.ipg.collection.BloomFilterWritable.impl(BloomFilterWritable.java:43) at com.ipg.collection.BloomFilterWritable.put(BloomFilterWritable.java:62) at com.ipg.prophet.twitter.twitflow.archive.UnzipTweetsProcessAndUpload$ProcessorConsumer.process(UnzipTweetsProcessAndUpload.java:107) at com.ipg.prophet.twitter.twitflow.archive.UnzipTweetsProcessAndUpload$ProcessorConsumer.run(UnzipTweetsProcessAndUpload.java:84) at java.lang.Thread.run(Thread.java:662) 

I think at least it should support a lot of attachments with such a high fpp value, right?

Answer 1

Sorry, I'm the culprit :)

We hope that we will be able to promote the next version soon. This is not the time to mention this, but there is the potential of this accident: this means that we can finally kill the current BF serial form and the supporting code associated with it (which was the error itself), which I have been trying to fix for a month - by the way, fixing this problems also fixes this problem.

Edit: more details here (and in the case of Louis)

Answer 2

This should probably be filed as a problem in Guava, not StackOverflow. (I confirm this, by the way, and I basically figured out what was going on.)

UPDATE: I registered issue and started the patch.