Statement on content written by AI

Question

We recently had some answers written by AI submitted to our moveset threads, so I think it is time we address this topic as other Q&A communities have.

We (staff) have agreed that content written in its entirety by AI is not allowed on PokeBase. I have updated the rules update draft to reflect this (and Pokemaster can sign off on that when he's next available). Our reasons for this are:

1. It's consistent with our existing preference for original content rather than copy-paste (see rule 0.4).
2. The most common model by far, OpenAI's GPT-x, is not up-to-date on Pokemon games and info (and will foreseeably remain this way).
3. AI doesn't know everything about Pokemon and can't always recognise gaps in its own knowledge. Even with this accounted for, AI often makes factual and logical errors.
4. Content written by AI is easy to distinguish from content written by people. For example, ChatGPT's replies have a distinctive cadence and structure, which makes a reasonable process for mods.

There is nuance in this stance. For example, if you use AI to write a data processing script to answer one of those "Which Pokemon have x trait and y trait" questions, then that would be permissible (as long as you share the script, so people can sanity check). We wouldn't consider this to be content produced "in its entirety" by AI.

This stance is also subject to change as AI improves. Indeed, GPT-x has improved in the past year, including in the area of Pokemon.

Feel free to discuss. I think AI is impressive, but it's a long way behind human users for the purposes of our site.

Comments

Can AI be used as a "source"? Can we answer using snippets from prompts given to AI?

---

Humans also often make factual and logical errors. As long as we continue fact-checking each other's answers, it'll probably be fine.

---

@vy That's OK, though I would recommend against it since anything AI tells you about Pokemon can be found somewhere else more reliable.
@sumwun I mean this more in the sense that AI makes mistakes that you and I never would.

Answer 1

Yes, I agree with this. GPT and similar systems are basically a (very) fancy predictive text engine. While that's great for some things like creative writing, accuracy is not the best. Especially on the free version (ChatGPT3).

In theory if a response is accurate it would be fine for an answer... but if you're asking GPT in the first place then you don't know the answer so you can't verify that the response it gives is accurate.

Fun fact, I was actually accepted into OpenAI's plugin program, to make a Pokemon plugin for ChatGPT4 (the paid version which is required to use plugins) but never got around to making it. In theory that would improve the accuracy by having the latest up-to-date information on new Pokemon, stats, moves etc. Will hopefully finish it some time.