It appears that your two-line captions beneath the thumbnails are breaking the flow of the layout. The CSS inline display method requires that each element within the list be the same height or else the flow on the next line will begin at the thumbnail with the largest width.
You can fix this by making your captions fit on the same line, or by adding explicit heights in your CSS. Note that you will need to change these values if you increase/decrease the size of your thumbnails.
Modifying gallery.css .tag declaration seems to do the trick, you may want to experiment with other methods as well.