Can I decline a roll?

Yes, you can always decline. You do not need to explain why. A simple "not this round, thanks" is sufficient. Reasons include fatigue, injury, or simply not wanting to roll with that person. Your safety and comfort come first.