LLM API basics for .NET developers: tokens, context and structured output

Short answer: an LLM API takes text in and gives text out, priced and limited in tokens. Everything you send (system prompt, conversation, documents) has to fit in the context window; the model's reply is capped by max_tokens; and the stop reason tells you why it stopped. For anything your code consumes, use structured output so the reply is JSON that matches your schema. Master these five ideas and the rest of AI engineering is built on top of them.

A real example: triaging support email

A small software shop receives emails like: "I paid 750,000 Toman twice for DiskWatch yesterday (order AB12CD34EF). Please refund one." You want each one turned into a category, the order code if there is one, the amount, and a one-line summary, so that the right person sees it first.

Free-form text from the model is useless here. You need a typed object, and you need the model to leave the order code empty rather than invent one.

The five concepts

1. Tokens. Models read and write tokens, pieces of words. A short English word is often one token; Persian usually takes more tokens for the same meaning. You pay per input token and per output token, and both are reported on every response.

2. The context window. The maximum number of tokens the model can consider in one request: your system prompt, the whole conversation so far, any documents, and the reply. Current the AI agent models have very large windows (frontier models has one million tokens), but large isn't free: every token you send is billed and adds latency. Send what the task needs.

3. System prompt vs. user message. The system prompt holds the rules and the role ("you triage support email; extract facts only"). The user message holds the input. Keeping them apart makes prompts reusable and is the first defence against an input that tries to change the rules.

4. Stop reasons. Every response says why generation stopped: end_turn (finished), max_tokens (cut off, so the answer is incomplete), tool_use (the model wants to call a tool; more on that in the agents article), or refusal. Check it before you use the content.

5. Structured output. Instead of asking nicely for JSON, you send a JSON Schema and the API constrains the reply to it. Your code deserializes it like any other API response.

Runnable code

Create a console app (dotnet new console), add the official SDK (dotnet add package Anthropic), set ANTHROPIC_API_KEY, and replace Program.cs:

using System.Text.Json;
using Anthropic;
using Anthropic.Models.Messages;

// Reads ANTHROPIC_API_KEY from the environment.
AnthropicClient client = new();

var ticket = "Hi, I paid 750,000 Toman twice for DiskWatch yesterday (order AB12CD34EF). Please refund one. - Sara";

var response = await client.Messages.Create(new MessageCreateParams
{
    Model = "claude-opus-5",
    MaxTokens = 2048,
    System = "You triage support emails for a small software shop. Extract facts only; never invent an order number.",
    Messages = [new UserMessageParam { Content = ticket }],
    // Structured output: the reply is guaranteed to be JSON that matches this schema.
    OutputConfig = new OutputConfig
    {
        Format = new JsonOutputFormat
        {
            Schema = new Dictionary<string, JsonElement>
            {
                ["type"] = JsonSerializer.SerializeToElement("object"),
                ["properties"] = JsonSerializer.SerializeToElement(new
                {
                    category = new { type = "string", @enum = new[] { "refund", "bug", "license", "other" } },
                    orderCode = new { type = new[] { "string", "null" } },
                    amountToman = new { type = new[] { "integer", "null" } },
                    summary = new { type = "string" },
                }),
                ["required"] = JsonSerializer.SerializeToElement(new[] { "category", "orderCode", "amountToman", "summary" }),
                ["additionalProperties"] = JsonSerializer.SerializeToElement(false),
            },
        },
    },
});

var json = response.Content.Select(b => b.Value).OfType<TextBlock>().First().Text;
var triage = JsonSerializer.Deserialize<Triage>(json, new JsonSerializerOptions(JsonSerializerDefaults.Web))!;

Console.WriteLine($"{triage.Category}: {triage.Summary} (order {triage.OrderCode ?? "-"})");
Console.WriteLine($"stop: {response.StopReason}, tokens in/out: {response.Usage.InputTokens}/{response.Usage.OutputTokens}");

public record Triage(string Category, string? OrderCode, long? AmountToman, string Summary);

The lines that carry the weight:

• System states the rule that matters most: extract, don't invent. Instructions about what not to do belong here, not in each message.
• @enum for the category means your switch never meets a value it doesn't know.
• Nullable orderCode gives the model a correct way to say "there isn't one". Without a null option, a schema can push a model to fill the field with something plausible.
• additionalProperties: false keeps the object exactly the shape of your record.
• Usage is on every response. Log input and output tokens per feature; that is how you find the expensive one.

Things that bite in production

• max_tokens too low. The reply is cut mid-sentence and, with JSON, can't be parsed. Check StopReason and give enough room.
• Trusting the content blindly. A schema guarantees shape, not truth. Validate business rules (does order AB12CD34EF exist?) in code.
• Putting user input into the system prompt. Keep untrusted text in the user message, ideally inside clear delimiters, so it is read as data.
• No timeouts or retries. It is an external HTTP API. The SDK retries rate limits and server errors a couple of times by default; decide what your feature does when it still fails.

Pros and cons of structured output

Pros

• Your code gets typed data, so the AI call looks like any other API client.
• Enums and nullable fields make the allowed answers explicit, including "unknown".
• Fewer retries caused by malformed JSON.

Cons

• The schema is a contract you must version, like any API.
• It constrains shape only; facts still need checking.
• Very large or deeply nested schemas make prompts longer and harder for the model; keep them small and split big extractions.

Where to go next

With typed output from a single call, the next step is giving the model the right data: RAG in .NET. The full path is in the AI engineering roadmap.

Need an LLM feature that your .NET backend can actually rely on? See how I can help.