I have no robots.txt file. The main reason for this is that I heavily support the Internet Archive. However, that doesn't mean I'm fine with you going about using website crawlers on my small site to feed your AI! Plus, if you really want to see my code, I plan to make it open source at some point. I also keep up every page I make (abandoned or not), including an archive of the original version of this site (/hworld). If you want to archive this site, then please, crawl my site all you want! Just please don't use it to feed your AI!