Calling an LLM from code

Why code, not chat

Real MDG work is volume: 3,000 failed IDocs after a cutover rehearsal, 40,000 suppliers to check. From code you send every item with the same prompt and get output a program can use. Every provider works the same way: messages and settings in over HTTPS, content plus token usage out.

Your first API call

Run pip install anthropic and set ANTHROPIC_API_KEY as an environment variable (never hard-code it). A call needs a model, max_tokens and messages; system is optional. The API is stateless: resend earlier messages to continue a conversation.

import anthropic

client = anthropic.Anthropic()   # reads ANTHROPIC_API_KEY
MODEL = "claude-sonnet-5-5"      # placeholder: check the docs for current names

resp = client.messages.create(
    model=MODEL,
    max_tokens=500,
    temperature=0,
    system="You are an SAP MDG support analyst. Be concise.",
    messages=[{"role": "user", "content": "Explain IDoc status 51 for CREMAS in two sentences."}],
)
print(resp.content[0].text)
print(resp.usage.input_tokens, resp.usage.output_tokens)
print(resp.stop_reason)          # "end_turn", "max_tokens", "tool_use", ...

Try it: complete the call

resp = client.______assistantchat.completions.createinput_tokenscontentusermax_tokensoutput_textmax_lengthprompt_tokensmessages.create(
    model=MODEL,
    ______assistantchat.completions.createinput_tokenscontentusermax_tokensoutput_textmax_lengthprompt_tokensmessages.create=300,
    system="You classify SAP IDoc errors.",
    messages=[{"role": "______assistantchat.completions.createinput_tokenscontentusermax_tokensoutput_textmax_lengthprompt_tokensmessages.create", "content": prompt}],
)
text = resp.______assistantchat.completions.createinput_tokenscontentusermax_tokensoutput_textmax_lengthprompt_tokensmessages.create[0].text
tokens_in = resp.usage.______assistantchat.completions.createinput_tokenscontentusermax_tokensoutput_textmax_lengthprompt_tokensmessages.create

Structured JSON output

Ask for JSON and show its exact shape. Then parse with json.loads, validate keys and allowed values, retry once, and route failures to a human. A stronger option: a tool whose input_schema is your output schema, forced with tool_choice. Some providers also offer structured-output modes. Validate either way.

import json
ALLOWED = {"MASTER_DATA", "CONFIG", "TECHNICAL", "UNKNOWN"}

def classify(error_text: str) -> dict:
    resp = client.messages.create(
        model=MODEL, max_tokens=300, temperature=0,
        messages=[{"role": "user", "content": PROMPT.replace("{error_text}", error_text)}],
    )
    if resp.stop_reason == "max_tokens":
        raise ValueError("truncated output")
    data = json.loads(resp.content[0].text)      # fails loudly if not JSON
    if data.get("category") not in ALLOWED:
        raise ValueError(f"bad category: {data.get('category')}")
    return data

Tool calling round trip

Pass tools=[...]. If the model wants one, stop_reason is "tool_use" and the reply holds a tool_use block with id, name and input. Run your function, then send a tool_result block with the matching tool_use_id. Loop with a maximum count and an allow-list of functions.

Evaluate with a small test set

Treat a prompt like a BRFplus rule: test before transport. Build 20–50 masked, real inputs with expected answers, including tricky ones. Score with code and rerun after every prompt or model change. Exact match works for categories; free text needs keyword checks or a rubric-graded model call you spot-check.

Try it: run an evaluation

Cost, latency and tokens

Cost = input tokens × input price + output tokens × output price; output usually costs several times more. Latency is driven mostly by output length. Levers: send less context, ask for compact JSON, use a smaller model for simple classification, cache repeats, batch non-urgent jobs. Log usage on every call.

Try it: bad JSON overnight

Check yourself