Money back if you cancel?

Good afternoon, here's what happened: I was recently on vacation in a city with some colleagues, and we wanted to go on a tour. Five minutes after booking, we changed our minds and canceled. Yesterday, the colleague who booked the tour received a call saying we wouldn't get a refund because it was too short notice. We booked the tour the day before. In my opinion, it's not too short notice, since the booking worked. Is there anything we can do?