Research Does Not Start with a Method: How to Frame a Question
Published:
A method is not the starting point of research
It is easy to begin with a method or an experiment that looks feasible. I have done that too: I moved on to experiments before I had properly organized the purpose and problem of the research.
The experiment ran, but I could not tell what question its results answered. Even when a number improved, I could not tell whether it answered the question—or whether another method could do the same.
The main point of this article is simple: before choosing a method, identify what is needed to answer the research question. You do not need knowledge of a specific field to follow this process. It can help both people starting their first project and people whose research has become stuck.
What you will learn
- How to separate a social problem, a technical problem, and a research question
- Why methods should be compared against the properties needed to answer the question, rather than starting with a method name
- Where to return when the question is still unclear
Separate three different things
Research becomes hard to explain when the following three things are mixed together.
| Type | What to consider | Question to ask |
|---|---|---|
| Social problem | Why does this problem need to be solved? | Who faces what inconvenience, loss, or risk? |
| Technical problem | Why is the problem not adequately solved now? | What do current knowledge, techniques, and approaches still lack in order to address the social problem? |
| Research question | What will this research find out? | What would we need to know to make the next decision? |
For example, “I want to support students” is an interest, but it is not yet a research question.
- Social problem: If people who need support are not noticed early, their learning difficulties may continue for longer.
- Technical problem: Learning records may predict whether a student will pass or fail, but they may not show which factors matter or what kind of support would help.
- Research question: What factors can be explained from limited learning records to inform decisions about support?
This is a fictional example. It does not describe a real study or result. Still, separating the three parts makes it easier to see what to investigate next and what a method must be able to do.
The social problem explains why the work matters. The technical problem explains what is still missing. The research question sets the boundary of what this study will answer.
Think about necessary properties, not just usable methods
Here, a necessary property is similar to a necessary condition in mathematics: it is a property that cannot be left out if you are to answer the research question.
By contrast, it is not enough to choose a method only because “this method seems able to achieve the goal.” It may sound convincing, but if you have not compared other methods, it is difficult to answer: “Could another method do this too? Why did you choose this one?” In undergraduate thesis presentations I watched, it was not unusual for faculty members to ask questions of this kind.
Showing that a method may work is different from explaining why it should be chosen.
The following is a common type of choice in computer science research. The principle, however, applies across fields: start from the properties needed to answer the question, not the method’s name.
In the fictional example above, the necessary properties depend on the goal.
| What you want to know | Properties to consider first |
|---|---|
| Predict pass or fail as accurately as possible | Predictive accuracy, available data, and computational cost when running the method |
| Explain factors that could guide support | Interpretability, explainable assumptions, and reproducible results |
| Study changes in learning over time | Ability to handle change over time and a valid period for comparison |
Deep learning, a family of methods that learn complex patterns from data, may be able to handle complex predictions. On the other hand, when data is limited, interpretability matters, or the basis for comparison must be clear, there may be a reason to consider a simpler and easier-to-interpret method.
The point is not that one is always better than the other. First identify what is needed to answer the question, then compare methods against those requirements.
A small worksheet before choosing a method
Before looking for candidate methods, write a short answer to each of these four prompts.
- Social problem: Why does this problem need to be solved?
- Technical problem: What can current approaches not do well enough?
- Research question: What will this research find out?
- Necessary properties: What must the method or experiment be able to do to answer that question?
Necessary properties may include the following.
- The result can be explained.
- The idea can be tested with limited data.
- Changes over time can be handled.
- The comparison can be reproduced.
- Important assumptions are satisfied.
- The work can be tested within the available time and computing resources, such as hardware capacity and computing time.
You do not need a complete answer at this stage. You can write “not yet confirmed” for what you do not know. The important part is to make clear what needs to be checked before putting a method name first.
What happens when you start with a method
Starting with a method does not guarantee failure. It does, however, make the following questions harder to answer.
- Why did you choose this method?
- Why would other methods not be enough?
- What result would count as an answer to the research question?
- Why is this comparison or evaluation necessary?
“This method improved a result” is sometimes not enough for a research claim. You still need to explain what the improvement means and whether it actually connects to the research question.
It also becomes harder when you try to solve many problems at once. If you add several changes for several problems, it is difficult to show which change addressed which problem.
Start by narrowing the work to one central question. Think about the properties needed to answer it, and then choose the method and experiment. This order gives you a place to return to when you get lost.
A diagnostic order when the question is unclear
When research is not moving forward, check these points before adding another method or experiment.
-
Can you state what you want to find out in one sentence?
- If you cannot explain it without naming a method, the method may be taking priority over the question.
-
Can you explain what each possible result would let you say?
- If you cannot think about the meaning of a result in advance, the study may not answer the question.
-
Do you have a reason for each necessary property?
- Do not stop at “higher accuracy is better.” Connect the need for accuracy, interpretability, speed, reproducibility, or another property to the problem.
-
Are you trying to solve too many problems at once?
- Keep what is truly needed to answer the central question, and make the rest out of scope.
If you cannot answer these four questions, return to the social problem, technical problem, and research question. That is not moving backward. It is part of designing the next investigation so that your work is not wasted.
Generative AI can help check a question
Generative AI should not decide the research question for you. It can, however, be a useful partner for checking one.
For example, you can give it your four short answers and ask it to:
- check whether there is a jump between the problem and the question;
- identify necessary properties you may have missed;
- suggest alternative explanations or approaches; and
- list possible evidence or prior work to check.
A suggestion from generative AI is not evidence. You need to check its proposed assumptions, methods, and papers against primary sources, your research purpose, and the necessary properties you identified.
Generative AI does not remove the need to think. Used well, it helps you notice questions you might miss and keep your reasoning in order.
Part 8, Using AI Without Rushing Decisions, covers how to check assumptions, alternatives, and evidence so that a remaining option is not mistaken for a reason to choose it.
Summary
You can organize the order of research like this:
Social problem
→ Technical problem
→ Research question
→ Properties needed to answer the question
→ Method and experiment
When you find a method that looks useful, you do not need to adopt it immediately. First ask which property it satisfies for answering the question. If you cannot explain that, return to the question and the necessary properties.
Next, I will cover how to find and follow research papers so that you can place your question in the existing literature.