What actually drives the cost of an AI chatbot? The API bill is 5%

The question clients ask most often is: "Isn't AI expensive?"

Behind it is an assumption — that most of the money goes to calling the model. I ran the numbers on real API pricing and the conclusion is the opposite: model calls are under 5% of total project cost.

TL;DR

Cost item	Share	Note
Model calls (API)	< 5%	100 conversations/day = ¥1.4–23/month
Scoping and design	10%	Deciding what can and can't be automated
Implementation	55%	Front-end, back-end, context assembly, failure handling
Testing and acceptance	20%	Real conversations, edge cases
Maintenance	10%	Post-launch prompt iteration, log review

A $500 quote for an AI assistant means someone is cutting engineering. An ¥8,000 quote is paying for the other 95%.

---

1. What model calls actually cost

Current pricing (CNY per million tokens):

Model	Input	Output
GPT-4o mini	0.15	0.60
DeepSeek V4.1 Flash	0.30	1.20
DeepSeek V4 Pro	1.32	3.96
GPT-4o	2.50	10.00

A realistic scenario

• 100 conversations per day
• 1,500 input tokens each (system prompt + history + question)
• 400 output tokens each (the reply)

3,000 calls per month:

Input:  3,000 × 1,500 = 4.5M tokens
Output: 3,000 ×   400 = 1.2M tokens

Model	Input	Output	Monthly total
GPT-4o mini	¥0.68	¥0.72	¥1.40
DeepSeek V4.1 Flash	¥1.35	¥1.44	¥2.79
DeepSeek V4 Pro	¥5.94	¥4.75	¥10.69
GPT-4o	¥11.25	¥12.00	¥23.25

Even GPT-4o comes to ¥23 a month.

That surprises people. It's also true — the marginal cost of inference is low enough to be noise in the project budget.

Run your own numbers with the AI cost calculator.

---

2. What pushes the API bill up

The base is low, but four things can multiply it 10–50×.

Context length (biggest lever)

Input tokens are resent on every call. A 5,000-character system prompt goes in every time; costs triple.

I've seen entire product manuals pasted into the system prompt — 20,000 tokens per call, 13× the cost of a trimmed version.

The fix: retrieval. Pull only the relevant passage.

Call volume

100 vs 1,000 conversations a day is a 10× difference. Linear and predictable.

The real risk is unexpected calls: duplicated front-end triggers, users double-clicking, unbounded retries.

Retry storms

Network jitter, timeouts and rate limits all trigger retries. Retry logic written as "keep trying until it works" can burn a week's budget in one incident.

Cap retries at 2–3, then take the fallback path.

No caching

Repeated questions ("what are your hours?", "do you accept refunds?") shouldn't hit the model every time.

Caching frequent answers typically hits 30–50%, cutting both cost and latency.

---

3. Where the money actually goes

Scoping (~10%)

Before writing code, answer:

• Which questions should it handle, and which must escalate?
• What's the cost of a wrong answer? (A wrong price is not the same as a wrong opening time)
• What source material exists? (FAQ, tickets, product docs)
• What does "working" mean? (accuracy %, latency seconds)

Skip this and everything downstream is guesswork. I've seen projects where the assistant answered beautifully — questions nobody asked.

Implementation (~55%)

Part	Share	What it covers
Context assembly	20%	History trimming, retrieval, prompt structure
Front-end	20%	Streaming, loading states, errors, mobile
Back-end	25%	Rate limits, auth, logging, timeouts, retries
Failure handling	20%	Model down, no answer, degraded mode
Cost controls	15%	Token caps, caching, per-user limits

Note that failure handling plus cost controls is 35% — the two things a demo never has, and the first things production exposes.

Testing (~20%)

Real conversations, focusing on:

• Questions outside the knowledge base
• Adversarial input (prompt injection, abuse)
• Multi-turn memory ("do you still remember what I said first?")
• Concurrency (10 users at once — do their contexts bleed?)

Usually 2–3 rounds of prompt and logic changes.

Maintenance (~10%)

The first three months involve:

• Filling gaps found in logs
• Tuning tone
• Cost optimisation when one scenario spikes

---

4. When not to use AI

Three cases where I'll say no:

1. Answers must be 100% correct

Pricing, legal terms, medical guidance. One error is a loss. Rules engines plus human confirmation are more reliable.

2. Volume is tiny

Five enquiries a day: humans are faster. The build cost covers years of manual replies.

3. Questions are too dispersed

If every customer asks something different with no common pattern, there's nothing to learn. Fix your FAQ first.

---

5. How to tell if a quote is sane

Ask how the API cost is calculated. If the answer is vague, they haven't done the maths.

Ask what happens on timeout. "It won't time out" means no production experience.

Ask for acceptance criteria. Accuracy and latency in the contract, or expect an argument at delivery.

---

Bottom line

The cost of AI isn't the model. It's the engineering. Model calls cost a few yuan a month; making it work costs thousands — and that's what you're buying.

Estimate your own scenario with the calculator, or read what's included in the AI integration service.