Unfortunately I’ve dealt with similar buyers and I do think it’s a flaw, maybe an inherent one, with how sites like this work. Buyers can blackmail you and threaten you with bad ratings if you don’t bend to their will even if they are in the wrong.
The problem is there is no, and can’t really be accountability for buyers, because even if they end up being a terrible client, any ratings against them are useless because they can easily create a new account, which I’ve seen some of my clients do for technical reasons or to circumvent me blocking them when I no longer wanted to work with them.
My first piece of advice would be simple to try your best to remain professional so that way if fiverr support reads through the order they will see that the buyer is the problem and not you. If the buyer is continuing to dispute the order you might have to get CS involved and inform the buyer than TOS states they cannot simply cancel an order because they are dissatisfied with the quality of the service. I have on one occasion had fiverr complete an order on my behalf in just such a situation.
The other option is to basically continue to refuse a cancellation request and eventually hope they get the picture and stop trying, let the order auto complete, and then not rate you, which has also happened to me. If in the event they still do rate you, you can always post a reply to their negative rating where you can outline to any potential readers that this buyer was clearly crazy and malicious so they will essentially ignore the rating.
It really just boils down to you weighing the pros and cons against your particular account and situation. On some level it might be worth giving into him, cancelling the order to cut your loses, and avoid the negative rating. However, your completion rating may go down which could be a problem depending on the volume of work you do. If you put a substantial amount of time into this, are talking about a large sum of money, and feel CS will take your side, you can continue to fight it out and risk the bad rating with the assumption that one negative review among many positives won’t hurt you in terms of buyer perception and search algorithms more than the cancellation will.