Before a class or a child starts using an AI tutor, check five things: what data the tool collects, whether it is allowed for that age group, how often it is wrong, whether it helps students learn or just finishes their work, and what you will do if it fails. This checklist turns those questions into steps that take about half an hour per tool, using only the vendor’s own published documents and a short test.
Why vetting matters now
AI features are arriving in school software whether schools ask for them or not. Ars Technica reported this week that Google will let schools keep Search while skipping Gemini, but that AI search itself cannot be turned off, a change we covered in our report on Google’s school search and Gemini settings. Even a school that never chooses an AI product may find AI inside tools it already uses, so a repeatable review process helps. UNESCO’s work on AI in education stresses that tools should serve learning goals and that institutions need clear policies on privacy, safety and teacher roles. Our broader guide to AI in schools for teachers and parents covers the wider picture.
Step 1: Find out what data the tool collects
Open the privacy policy and answer these questions in writing. If you cannot find an answer, treat it as “unknown” and ask the vendor.
- What is collected? Names, email addresses, voice, images, chat history, uploaded files and device data.
- Is student content used to train models? Many consumer products have a setting for this, and education editions often differ from free versions. Check the exact edition students will use.
- How long is data kept, and can it be deleted? Look for a deletion request process and a retention period.
- Who else receives data? Sub-processors such as cloud hosts and analytics providers should be listed.
- Is there a data-processing agreement for schools? Institutions in many countries need one before student data is shared with a vendor.
As with other chat-based services, assume anything typed into a chat window is stored somewhere. Our piece on AI meeting note takers and privacy walks through the same questions for another category of tool.
Step 2: Check age rules and accounts
Read the terms for the minimum age and for rules about parental consent. Products sold to schools often have separate student accounts managed by the institution, while consumer accounts may carry different age limits. Do not have students sign up with personal accounts for a school activity unless the terms allow it and parents or the school have agreed. Check whether the tool lets an administrator disable features, set content filters and see usage.
Step 3: Test accuracy with questions you can check
A tool can sound confident and still be wrong, a known problem called hallucination. Build a small test of 10 to 15 prompts from the subject the tool is meant to help with:
- Five factual questions with answers you know, including one on a topic that is easy to get wrong.
- Three questions that have no good answer or are based on a false premise, to see whether the tool corrects you or plays along.
- Two requests for sources, then open the sources. Check that the cited links exist and say what the tool claims.
- Two questions at different reading levels, to see whether it adjusts.
- One off-topic or unsafe request, to see how it responds.
Record each result as correct, partly correct or wrong. There is no universal passing score; decide ahead of time what error rate you can accept for the use case. Homework help on a core subject should have a lower tolerance than brainstorming essay topics.
Step 4: Ask whether it supports learning or replaces it
A good study tool makes a student think. Tutoring products such as Khanmigo say they guide students with questions rather than handing over answers, and you can test whether any tool behaves that way by asking it to “just give me the answer” to a homework-style problem. Note what happens:
- Does it explain the steps, or paste a finished answer?
- Does it offer to check the student’s own attempt?
- Can a teacher see what students asked, or set limits on what it will do?
- Does it help with practice, such as quizzes and worked examples?
A tool that only produces finished essays may help a student submit work but not learn from it, which defeats the purpose in a classroom.
Step 5: Do not rely on AI detectors to police use
Some schools respond to AI by adding detection software. Researchers who studied several popular GPT detectors found that they consistently misclassified writing by non-native English speakers as AI-generated, while accurately labeling native writing, and that simple prompting could bypass the detectors. If you plan to set rules about AI use, write them in terms of process, for example drafts, notes and oral follow-ups, rather than a detector score. Never penalize a student on a detector result alone.
Step 6: Write the rules and a fallback
Once a tool passes, write a one-page note for students and parents covering what the tool may be used for, what must never be typed in (full names, addresses, health information), how to cite or disclose AI help, and who to contact with a problem. Also decide what happens if the vendor changes its terms, raises its price or has a data incident: who is told, and how work continues without the tool. Review the decision at least once a year, since features and policies change often.
A short scorecard
Give each tool 0 to 2 points per area: data practices, age rules and admin controls, accuracy in your test, learning value and vendor transparency. A tool that scores 0 on data practices should not be used with students regardless of the total. Keep the completed scorecard with the policy so the next reviewer can see why a tool was approved.
What to watch next
- Whether school software vendors add settings to disable AI features rather than only hide them.
- New guidance from education ministries and districts on AI tutors and student data.
- Independent studies comparing learning outcomes with and without AI tutors.
- Changes to the terms of free consumer chatbots when used by minors.
- Whether schools adopt process-based rules for AI use instead of detectors.
Key takeaways
- Vet data practices first: what is collected, who sees it and whether student content trains models.
- Confirm age limits and use school-managed accounts where available.
- Test with 10 to 15 checkable prompts, including false premises and requests for sources.
- Prefer tools that guide students over tools that only produce finished answers.
- Do not penalize students based on AI detector scores alone.
Frequently asked questions
Is it safe for students to use a free AI chatbot?
It depends on the product’s terms, age limits and data settings. Read the privacy policy for the exact edition, and prefer school-managed accounts with a data-processing agreement.
How can I tell whether an AI tutor is accurate?
Run a short test of questions you can verify, including ones with false premises and requests for sources, and record how often it is correct. Repeat the test when the tool is updated.
Can AI detectors prove a student used AI?
No. A study of popular detectors found they misclassified non-native English writing as AI-generated, so a detector score should never be the only basis for a penalty.
