Nitro-Net.com – Internet Marketing Services – A Global Marketing Group Company
John Mueller of Google said that your robots.txt content-type can be anything really. He said “any content type is fine,” adding “the important part is the content itself.”
He also said he has “seen html pages that included valid robots txt lines among a valid html page.”
If you are concerned, test it with Google’s robots.txt validator.
Oh, the Penguin runner asked this question:
Afaik any content type is fine. The important part is the content itself. I’ve seen html pages that included valid robots txt lines among a valid html page 🙂
— 🍌 John 🍌 (@JohnMu) June 1, 2020
Forum discussion at Twitter.