The Kalman filter is applied when all measurements (accelerations in this case) are equal to the true value plus the measurement error. Measurement error is the noise of the process. In order for the original Kalman filter to be applied, the noise must be normally distributed, that is, sometimes the error will be positive, sometimes negative and on average equal to zero.
If you sharply hold your Android phone back and forth, there will be great acceleration. I suggest recording the readings of the accelerometer in this kind of action and looking through the eyes to see if the readings seem to be really affected by some normally distributed process noise. I assume that the answer is no, i.e. I expect that their readings when plotting on the chart will be smooth. But if they are not smooth, a Kalman filter may be useful.
If you are trying to use the accelerometer readings to determine your location, I think your project is doomed to failure. Acceleration is a second derivative of position with respect to time, and I have never heard that anyone can integrate readings with sufficient accuracy to be useful.
I have successfully applied the Kalman filter to GPS readings on my Android phone to improve my location estimation. See Smooth GPS data for a code that uses the Kalman filter for this. Subsequently, I wondered if speed and possibly acceleration data could be used to improve location estimates. Although I never followed this idea, see https://dsp.stackexchange.com/questions/8860/more-on-kalman-filter-for-position-and-velocity for the math that I considered when using.
The optimal way to use all sensor inputs (GPS, accelerometer, gyroscope, etc.) to get a good location estimate is a very difficult (and interesting) problem. To find out more, the key phrase for your search is “Sensor Merging”. There is an old youtube video on this subject at http://www.youtube.com/watch?v=C7JQ7Rpwn2k .