Trouble is in Firefos the thumbnails shift about and load wrongly.. Especially if you mouse over the furthest righ one it does really weird things to the layout.
This is a problem with the default theme of Beta 2 and 2.1. The problem is that the :hover borders and padding go on the outside, affecting the floats on which the gallery relies. This doesn't happen in Internet Explorer because it incorrectly applies padding to the inside of the box; therefore, you don't see it, because it's (in essence) underneath the thumbnail.
The solution? Remove any :hover pseudo-class rules for .slides li and .photos, which should solve the problem.