I can not explain the behavior of the buggy. Obviously, firefox does not clear the drawing area when drawing a different border.
But here is a (dirty) solution
<a class = 'testButton' href = '#' onmouseover = "var button = this; button.style.borderBottom = '0px'; setTimeout( function() { button.style.borderBottom = '5px solid red'; }, 1 )"> Button! </a>
source share