# www.path10.net robots.txt # # GIST OF THIS FILE # # You are free to do the below in this site:... # 1. Crawl / Spider # 2. Minimally index, for the purpose of rendering publicly available search results with relevant titles and/or tag words to your users # 3. Copy URLs, for the purpose of sending visitors via a hypertext link to any HTML web page within, AND in a new browser tab, without any # permission by Path10 # # ... PROVIDED THAT YOU... # 1. Respect the rules of this robots.txt file # 2. Respect the copyrights of the content within the site # 3. Respect the site infrastructure # 4. Announce yourself authentically via the User-Agent HTTP header # # If you do not know how robots.txt was meant to be followed, see: https://www.rfc-editor.org/rfc/rfc9309.html # # # REGARDING AI... # # Please note that with regard to usage of web content, Path10 is, for the most part, anti-AI and the topic of AI will even be discussed # from time to time in the Path10 blog. Path10 is not responsible for any ensuing negative outcomes to such AI bots, including but not limited # to robotic depression and/or GPU overheating. # ---------------------------------------------------- User-agent: * Content-signal: search=yes, ai-train=yes, ai-input=yes Allow: / Crawl-Delay: 10 # ----------------------------------------------------