You are both sort of right. Signal’s on device storage is AFAIK only weakly encrypted with your screen-lock password (although you can enable a second weak password in Signal specifically). So if your phone is unlocked and backdoored the attacker can pretty much do what ever they want including reading your Signal messages.

Edit: to make it more clear… someone with a backdoor can just start a hidden application that does the same as the Signal client and open the on device stored messages with out your knowledge. And it is also quite trivial to install a keylogger to capture all passwords including those inside Signal.