Building your first version
How do I test an AI-built app if I can't read the code?
Lumos · Updated
Test what a person needs to accomplish. Start from the link they'll receive, enter fresh sample information, and follow the task to its result. Check that the result remains when you leave and return.
Write down what should happen before you try it. For the handbook's imaginary feedback app, that means a designer creates a request, a client responds, and the designer can read the response later.
Try the awkward cases too
- Leave a required field empty. Check that the app explains the problem and keeps the other information you entered.
- Refresh after saving, then open the app in another session with the intended account. Check that the correct record appears.
- Try the task on a phone and using a keyboard. Make sure you can reach and understand the controls.
- Use a separate test account that shouldn't have access. Check that it cannot read or change the private record.
The W3C's Easy Checks provides an introductory accessibility review. It can identify some barriers, but a complete accessibility assessment needs more than these initial checks.
Ask the AI to run the project's automated checks and explain what they cover. For important behaviour, ask for a test that would fail if that behaviour broke. A passing build alone doesn't demonstrate that the user can finish the task.
Record failures with the steps that caused them. These checks give you useful evidence; security-sensitive work can also need review by someone who understands the implementation.
In the Builders Handbook
Follow Test it, fix it, and keep going, then Give it to a friend to see someone else try the task.