User-agent: AhrefsBot Disallow: / User-agent: Baiduspider User-agent: Baiduspider-video User-agent: Baiduspider-image Disallow: / User-agent: BLEXBot Disallow: / User-agent: DotBot Disallow: / User-agent: IstellaBot Disallow: / User-agent: MegaIndex.ru/2.0 User-agent: MegaIndex.ru Disallow: / User-agent: MJ12bot Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: Yandex Disallow: / User-agent: Qwantify Disallow: / User-agent: GPTBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: CCBot Disallow: / User-agent: Bytespider Disallow: / User-agent: PetalBot Disallow: / User-agent: * Disallow: /checkout.aspx Disallow: /logout.aspx # Twitter bot can't grab images when robots.txt blocks links with an "r" param, so remove those "r" restrictions. Perhaps other bots may have similar issue as well. # Disallow: /*?r= # Disallow: /*&r= # Disallow: /*?R= # Disallow: /*&R= Disallow: /*?ref= Disallow: /*&ref= Crawl-delay: 1 Sitemap: https://www.artpal.com/siteindex.xml