NL Tools Server

PreviousNext

Jev + Needle 3 + Gemini fallback for natural-language tool calls

Overview

@zooly/nl-tools-srv turns a natural-language query and a caller-supplied tool list into ranked tools, filled arguments, and executed results. Jev shortlists, Needle 3 fills arguments locally (WASM), and Gemini 3.8 Flash runs only when required args are missing or Needle cannot load.

Package Details

  • Package Name: @zooly/nl-tools-srv
  • Location: packages/nl-tools/srv
  • Type: Server-side NL → tools library
  • Playground: /admin/nl-tools in zooly-app (e-commerce fixture)

Runtime

query → Jev shortlist (top 5 + companions) → Needle 3 → Gemini fallback if needed

Needle 3 is about 35 MB of weights plus a 0.6 MB WASM engine, downloaded from Hugging Face into /tmp/cactus-needle on Vercel (or ~/.cache/cactus-needle locally). It does not run on the Edge runtime. The admin route uses runtime = "nodejs" and maxDuration = 60.

If WASM cannot instantiate, the result still includes a needle.error and Jev + Gemini continue.

Public API

import { handleNlQuery, isNlToolsConfigured } from "@zooly/nl-tools-srv";

const result = await handleNlQuery({
  query: "cancel order 4821",
  tools: [
    {
      name: "cancel_order",
      description: "Cancel an open order.",
      parameters: {
        type: "object",
        properties: { order_id: { type: "string" } },
        required: ["order_id"],
      },
      companion: "list_orders",
      execute: async ({ order_id }) => ({ cancelled: order_id }),
    },
  ],
});
ExportPurpose
handleNlQuery({ query, tools, ... })Full pipeline. Tools are passed in — there is no built-in catalogue.
isNlToolsConfigured()True when AI_GATEWAY_API_KEY is set.
@zooly/nl-tools-srv/fixtures/ecommerceOptional mock shopper catalogue and handleEcommerceQuery used by the admin playground.

Tool shape

  • name, description, parameters (JSON Schema) — what Jev and Needle see
  • companion — extra list tool given to Gemini only (e.g. cancel_order → list_orders). Needle never sees it.
  • execute — your function; the package never talks to product DBs

Options

  • system — Needle facts (date, locale, user)
  • noneDescription / jevInstructions — Jev refuse / ranking copy
  • topK — default 5
  • fallbackModel — default google/gemini-3.8-flash
  • isNeedleFailure — override the default empty/placeholder-id heuristic

Environment

VariableRequiredPurpose
AI_GATEWAY_API_KEYyesJev evaluate + Gemini fallback via Vercel AI Gateway
NEEDLE3_CACHEnoDirectory for WASM + weights. Defaults to /tmp/cactus-needle on Vercel
NEEDLE3_CACTnoPre-downloaded needle3.cact path, skips the Hugging Face fetch

Playground

Signed-in admins can hit /admin/nl-tools and run one shopper query against the e-commerce fixture. Do not batch the full 46-query demo on that page.

Out of scope

This package does not wire z-agent, merch, or stats. Those products import handleNlQuery and pass their own tools.