How to ignore PushState history and return to the last page on the back button

Question

I change location.hashwhen the user views the slide show. Therefore, each slide has its own hash.

I would like to make the back button on the previous page, not the previous one. What is the best way to do this?

Note. The previous page is not always the same.