Yes this would make sense.
Quote from “What methods can we use to obtain consent?”:
If you are asking for consent electronically, consent must be “not unnecessarily disruptive to the use of the service for which it is provided”. You need to ensure you adopt the most user-friendly method you can.
For a website, hiding rejection behind a link should class as “unnecessarily disruptive”. If you can provide consent with the press of a single button then rejecting should also be the press of a single button.
I encounter something similar to this often.
There’s a lot of cookie banners where “Accept All Cookies” is a single button but in order to reject cookies you have to press a “Manage Cookies” link which will have something similar to a “Reject All Cookies” button in it.
It’s very annoying.