A Smarter AI Agent: neo-pkg-llm-chat 2.0.0 Released

3 hours ago

5 min read

We have entered an era in which you can ask about data in natural language and let AI draw charts and even build dashboards for you.

Machbase Neo’s neo-pkg-llm-chat package is an LLM agent that makes this experience possible.

If you have used an earlier version, however, you may have encountered frustrations like these:

“I connected a local LLM through Ollama, but it keeps calling the wrong tools…”
“Why are the API costs higher than I expected?”
“When several people use it together, one person’s heavy analysis slows everyone else down.”

We at Machbase had the same concerns. That is why we completely redesigned the package for neo-pkg-llm-chat 2.0.0. In this article, we share the results.

Here are the key improvements over version 1.x.

A new harness architecture
A single package built on JSH
Up to 80% lower token usage
Other improvements
1.x vs. 2.0 at a glance

1. A harness architecture: Evolving how the AI agent thinks and acts

The previous version, 1.x, handled almost every user question in the same way.

Whether the request was as simple as “Show me yesterday’s data” or as complex as “Analyze anomalies in the past month’s vibration data and create a report,” the AI was shown the same 26 tools and left to choose the right ones.

This often confused the LLM. It might try to create a dashboard for a simple query, or draw a chart and stop when the user had asked for a report. Even a capable model like Claude sometimes went off course.

Version 2.0.0 introduces a harness architecture.

Simply put, it organizes the AI’s workspace in advance to match the user’s intent.

Eight skills organized by intent

The system first classifies what the user wants, then exposes only the tools needed for that intent.

User intent	What it does
Basic analysis	Query data, create charts, and build dashboards
Advanced analysis	Complex analysis such as FFT and statistics
Reports	Create HTML reports using four templates
Data queries	Run simple queries and return SQL results
Document search	Search Machbase manuals and documentation
Timers	Register and manage TQL timers
System information	Query system metrics and status
General conversation	Other open-ended conversation

For example, if the user asks, “Show me yesterday’s data from the BEARING table,”

the system recognizes a simple data query and exposes just three relevant tools to the AI.

That is a major change from asking it to choose among 26 tools in 1.x.

Stronger automatic correction and safeguards

LLMs sometimes make mistakes.

They may forget to draw a chart, send a tag name in lowercase when uppercase is required, or try to create a dashboard before checking the data.

Version 2.0.0 adds safeguards that automatically catch these common mistakes.

Analysis finished but the chart is missing? → Generate the chart automatically.
The LLM sent temperature, but the tag is actually TEMPERATURE? → Correct the letter case automatically.
The same tool keeps failing? → Stop it to prevent an endless loop.
A call uses a tag that does not exist? → Block it before execution to avoid unnecessary API usage.

In effect, we have created an environment that can accommodate LLM mistakes.

The quality of the results users see has become noticeably more consistent.

2. Now inside Machbase Neo: A single package built on JSH

If you installed 1.x, you may remember having to download the backend, neo-pkg-llm, and the frontend, neo-pkg-llm-chat, separately, then build the Go code.

You also needed a Go compiler and different binaries for different platforms. It was cumbersome.

For 2.0.0, we rebuilt the package to run in Machbase Neo’s internal JSH environment.

What has improved?

No Go build is required: it runs directly wherever Machbase Neo is installed.
One package is enough: the backend and frontend are now combined.
No platform concerns: it works the same way on Windows, Linux, and macOS.
Closer integration with Machbase Neo: direct access to Neo’s internal APIs makes data flow faster and more efficient.

This is a strong example of what the enhanced JSH environment in V8.5 makes possible.

Install Machbase Neo and copy the package once. That is all it takes.

Per-user settings are now supported

Another improvement is that multiple users can now work with their own settings.

User A can use Claude, user B can use GPT-4, and user C can use an internal Ollama deployment.
Each person can use their own API key, allowing costs to be billed separately by department.
Depending on security requirements, some users can be restricted to local Ollama models.

This should be particularly useful for team deployments in businesses.

3. Token usage is significantly lower—by up to 80%

This may be the change users notice most.

Why did 1.x use so many tokens?

The previous version sent the entire system prompt and all tool definitions—about 8,200 tokens—to the LLM for every question.

Even a simple request such as “Show me yesterday’s data” included descriptions of all 26 tools.

It was like reading out the entire restaurant menu every time a customer asked for a glass of water.

Version 2.0.0 sends only what the intent requires

Thanks to the harness architecture, the AI receives only the tools and descriptions relevant to the task. Token usage has consequently fallen as shown below.

Task	Reduction in token usage
Data queries	83%
System information	83%
Report creation	78%
Document search	77%
Basic analysis	68%
Timer tasks	59%
Advanced analysis	49%

Why does this matter?

1. Lower API costs

Claude and GPT services charge by token usage, so fewer tokens translate directly into lower costs. An 80% reduction makes a substantial difference for frequent tasks such as data queries.

2. Faster responses

The fewer tokens an LLM has to process, the sooner its response begins and the shorter the overall response time. The time to first token (TTFT) is especially noticeably reduced.

3. Better performance with local LLMs

Local LLMs used through Ollama often have smaller context windows or perform less well with long inputs. Reducing tokens helps smaller models operate reliably. In Ollama environments, the system also automatically uses compact English prompts, reducing token usage by a further 35–40% for the same tasks.

Other improvements

Sessions are isolated

Previously, when one user ran a heavy analysis, other users’ responses could slow down too.

In 2.0.0, each user has a separate processing space, so one person’s work does not affect others. If one user encounters a problem, others can continue working unaffected.

More natural reports

In 1.x, a report might show an exaggerated warning such as “Danger! Immediate training required!” even with a safety score of 95.

In 2.0.0, the tone automatically adjusts to the score.

90 or above → Positive, encouraging tone
70–89 → Balanced tone
Below 40 → Focus on serious problems

The vibration severity categories—Good, Warning, Danger, and Critical—have also been revised, making vibration analysis reports more trustworthy.

1.x vs. 2.0.0 at a glance

Area	1.x	2.0.0
Installation	Go build required; two packages	JSH-based; one package
Tools	All 26 tools exposed for every question	Tools selected according to intent
Token usage	About 8,200 tokens each time	49–83% reduction
Settings	One shared configuration	Per-user settings
Processing	One shared process	Independent processing for each user
Error handling	Limited correction	Automatic correction and safeguards
Report tone	Uniform	Automatically adjusted to results

Closing thoughts: Smarter, faster, and more efficient

When building neo-pkg-llm-chat 2.0.0, one goal mattered most:

“Enable customers to use Machbase Neo’s AI agent in real work.”

We needed more than a flashy demo. We needed an AI agent that multiple people could use simultaneously, reliably, and at a reasonable cost in a real production environment.

Version 2.0.0 is the first step in that direction.

We are excited about the possibilities opened up by an LLM agent integrated directly into Machbase Neo.