Using Generative AI Without Rushing Decisions: Check Necessary Properties
Published:
Do not stop at the first convincing option
When you brainstorm with generative AI, obvious errors are not the only risk. A first option may receive an explanation that sounds convincing, and you may stop thinking too soon. You can then miss an option that fits the goal better and be unable to explain later why you chose the first one.
This risk exists with chat-based generative AI and with AI agents that work across files and materials. Treat their output as possible issues and options to check, not as a decision.
This article is for people who use generative AI to discuss a research question or method and want to avoid choosing an option too early.
What you will learn
- Why a remaining option is not, by itself, a reason to choose it
- Which assumptions, alternatives, and evidence to check with generative AI
- How to record a provisional decision and connect it to the next check
A remaining option is not necessarily the right choice
In Part 2, I explained why it helps to start from the necessary properties for answering a research question instead of from a method name. Necessary properties show what a candidate must be able to do.
Several methods may appear to fit a research goal. If you adopt the first method you find because it seems able to achieve the goal, you may simply be choosing the first candidate. Even if one candidate remains after checking the necessary properties, that does not by itself explain why it should be chosen.
Start by restating the goal and research question without naming a method. Then derive the necessary properties. Use them to examine existing methods, modifications or combinations, a new design, and a fundamentally different way to answer the question. This shifts the task from defending the first option to looking for one that fits the goal better.
For each property, check the following.
- Why is this property necessary for answering the question?
- Which paper, experiment record, or study material supports it?
- Is it essential, desirable but optional, or outside the scope of this work?
If you cannot explain a property or its support is weak, keep it open instead of rushing to a conclusion.
Do not narrow the candidates or assumptions too early
If you give generative AI only the first candidate you found, the comparison can remain limited to that candidate.
When considering a method or direction, ask whether you should also include these kinds of options.
- Use an existing method as it is.
- Combine existing methods or change part of one.
- Design a new method for the question.
- Consider a fundamentally different way to answer the question.
You do not need to implement every option. The important point is not to decide that the first option is enough without considering alternatives.
Check assumptions in the same way. A data format, comparison condition, or procedure used in prior work may have been chosen for that work. Consider whether it is necessary for your question and whether an option without that assumption is possible.
Ask generative AI for more than confirmation
If you ask generative AI only, “Is this option good?”, it is easy to receive an answer that reinforces it. Share the question and necessary properties, then include counterexamples, alternatives, assumptions, and evidence that still needs checking in the discussion.
Keep at least these points separate.
- Necessary properties for answering the question, and the evidence for them
- Existing methods, modifications or combinations, new designs, and fundamentally different methods
- Assumptions being made, and options without those assumptions
- Verified facts, inferences, and points that need further checking
Check papers, facts, and comparisons suggested by generative AI against the original sources and your own records. When an AI agent reads research materials, specify what it must read and which materials it must not read or send outside. In all other areas, it can follow related records and links. This may reveal evidence or issues you did not think of at first. The research foundation from Part 7 makes this checking easier.
If you cannot decide yet, record it as provisional
During research, you may not be able to choose one option yet. You do not need to make the evidence look more complete than it is. If you use an option for now, record that the decision is provisional.
Record at least the following.
- The option you are using now and why
- The sources you checked
- Options you have not compared and assumptions you have not verified
- What would make you reconsider the decision
This keeps options from an AI conversation from becoming conclusions automatically. Instead, they become the next things to investigate, test, or discuss.
Summary: Do not let generative AI settle a decision too soon
Generative AI can act as a research assistant that helps check options, counterexamples, assumptions, and evidence. But an option that does not conflict with your conditions, or an explanation that sounds convincing, is not enough to justify choosing it.
Start from the necessary properties for the question. Broaden the candidates and assumptions you examine, and check the evidence in the original sources. Record unresolved points as provisional decisions, then move to the next check. Part 9 will cover a day-to-day workflow for turning generative AI discussions into working notes and the next research question.