Alexa and Google Assistant (to my knowledge) do not have delays built-in. Apple’s Shortcuts do, but you would need to expose your TV to HomeKit in order to accomplish that and then of course have an Apple device to act as a Hub as well (not likely as you highlighted Alexa/Assistant). So scratch that idea.
My general answer for home automation tends to be Home Assistant. I’m quiet partial to it .
Assuming your TV integrates with Home Assistant (you could also integrate an IR blaster like Harmony or Broadlink) you could then setup a script in HA that could be triggered by Alexa or Assistant that has a delay in it before sending the call to turn off your TV.
@susandennis Roku is integrated as is what I use in my home.