Before Using an LLM, Ask: Do We Really Need One?

Recently, while working on a project involving an LLM, I came across an interesting realization.

Today, when we face a new problem, our first thought is often:

“Can we use an LLM for this?”

But I think we should ask a different question first:

“Do we actually need an LLM for this problem?”

This small change in thinking can make a big difference in how we design AI systems.

Not Every Problem Needs an LLM

There are many problems where the answer is already clearly defined.

For example:

• Calculating a total price
• Checking whether a value is greater than a limit
• Validating an email address
• Applying a business rule
• Querying a database
• Calculating distance between two locations

These are deterministic problems.

If we know the rules, traditional programming is usually the better solution.

Why introduce an LLM when a simple function, SQL query, or if/else condition can give us the correct answer every time?

Using an LLM in such cases can add unnecessary cost, latency, complexity, and unpredictability.

So Where Does an LLM Become Useful?

The situation becomes different when the problem is not completely defined by fixed rules.

Consider a situation where one input can have many possible interpretations or outcomes.

For example, a user might describe a problem in natural language.

From that single description, we may need to understand:

• What is the user actually asking?
• What information is important?
• What could be the possible causes?
• What are the possible next actions?
• Which possibility is more likely?
• What similar situations exist?

Now the problem is no longer simply:

Input → Rule → Output

Instead, it becomes something closer to:

Input → Understanding → Possibilities → Reasoning → Possible Outcomes

This is where the power of LLMs becomes much more valuable.

Facts vs. Possibilities

I think one useful way to look at LLMs is through the difference between facts and possibilities.

If we already have a known fact, we don't necessarily need an LLM.

But when we have a large number of possible interpretations, scenarios, or outcomes, an LLM can help us work with that complexity.

For example, a traditional program can easily answer:

“Is this number greater than 100?”

But asking:

“Based on this description, what are the possible reasons for this situation and what should we consider next?”

is a very different type of problem.

The second problem involves language, context, ambiguity, and multiple possibilities.

That is where an LLM can add real value.

LLM Should Be Part of the Architecture, Not the Architecture

Another important lesson I took from my recent project is that an LLM does not have to be the entire solution.

A good system can combine traditional software with AI.

For example:

Traditional code can handle:

• Rules
• Calculations
• Validation
• Database operations
• Authentication
• Deterministic decisions

LLMs can handle:

• Understanding natural language
• Extracting meaning from unstructured information
• Generating possible explanations
• Handling ambiguous inputs
• Summarizing large amounts of text
• Producing contextual responses

The best architecture may therefore be a combination of both.

The Question We Should Be Asking

I don't think the important question in today's AI world is:

“Where can we use an LLM?”

Instead, we should ask:

“What part of this problem actually requires the capabilities of an LLM?”

That question helps us avoid using AI simply because it is available.

LLMs are powerful, but powerful does not mean necessary for every problem.

Sometimes the best AI solution is not using AI at all.

And when the problem involves ambiguity, context, probabilities, and a large number of possible outcomes, that's when we should seriously think about what an LLM can bring to the table.

Final Thought

My recent project made me realize that good AI engineering is not about putting an LLM everywhere.

It is about understanding the problem first.

Use traditional programming when the rules are clear.
Use LLMs when understanding and possibilities become the problem.
And when a problem contains both, combine them.

The future of AI engineering may not be about using more LLMs.

It may be about using them in the right places.