Building an LLM feature, part 2: design the system around the model

In part 1 we decided where the model belongs: after the parser and the rules, only for the descriptions nobody else can classify. Now we design the system around that decision.

Two paths: one the user waits for, one they don't

The request path does everything that's fast and deterministic:

1. Receive the file.
2. Pick a parser for the bank and file type.
3. Run the rules on each row.
4. Save the transactions, categorized or not.
5. Emit a statement.completed event.

The user sees their transactions right away. Most are already categorized.

The background path listens for that event, collects the transactions that are still uncategorized, sends them to the model in batches, validates the answers, saves them, and learns a rule from each answer. The dashed arrow in the diagram is the point of the whole design: every answer the model gives makes the next upload cheaper.

Why an event instead of calling the model inside the request?

• The upload stays fast even when the model is slow.
• A model outage can't break uploads.
• You can retry or re-run categorization later without re-uploading anything.

In Nora this is NestJS's event emitter. In a bigger system it would be a queue. The shape is the same.

Parsers behind one interface

Every bank parser implements the same small interface, so the rest of the pipeline doesn't care which bank a file came from:

export interface ParsedTransaction {
  date: Date;
  description: string;
  rawDescription: string;
  amount: number; // positive for income, negative for expense
  balance?: number;
  type: "INCOME" | "EXPENSE";
}

export interface ParseResult {
  transactions: ParsedTransaction[];
  errors: ParseError[]; // { row, rawContent, message }
  totalRows: number;
}

export interface IStatementParser {
  parse(fileBuffer: Buffer, fileName: string): Promise<ParseResult>;
  supports(bankType: string, fileType: string): boolean;
}

Two details matter here. Parsers return errors per row instead of throwing, so one broken line doesn't lose a whole statement. And the original description is kept as rawDescription, because that's what rules and the model work on.

The data model

Here are the parts of the Prisma schema that matter for categorization (trimmed):

model Transaction {
  id                      String   @id @default(uuid())
  account_id              String
  date                    DateTime @db.Date
  raw_description         String
  amount                  Int      // COP, no decimals
  category_id             String?
  confidence_score        Float?   // 0.0 - 1.0
  statement_upload_id     String?
  is_manually_categorized Boolean  @default(false)

  @@unique([account_id, date, amount, raw_description], name: "unique_transaction")
}

model CategoryRule {
  id          String @id @default(uuid())
  user_id     String
  category_id String
  keyword     String // normalized: lowercase, no accents
  match_count Int    @default(1)

  @@unique([user_id, category_id, keyword])
}

Design decisions hiding in those lines:

• category_id is nullable. "Uncategorized" is a real, valid state, as the plan requires.
• confidence_score is stored per transaction. Different sources get different scores (a learned rule starts at 0.80 and grows with each match up to 0.95; a keyword match is 0.75; an LLM answer is 0.85). The UI can use it, and so can your analysis later.
• is_manually_categorized marks human decisions, which should never be overwritten by automation.
• The unique constraint makes uploads idempotent. Upload the same statement twice, or two statements that overlap, and duplicates are skipped instead of double-counted.
• CategoryRule is per user. Your "Domicilios" isn't necessarily my "Restaurantes". match_count lets confidence grow as a rule proves itself.

The contract with the model

The model is a component with an interface, and the interface should be as strict as you can make it:

• Input: a numbered list of descriptions, plus the user's category names.
• Output: for each number, one category from that list, or "Sin categoria".
• Format: JSON that matches a schema, enforced by the API, not by asking nicely in the prompt.

Two choices make this contract robust:

1. Refer to items by index, not by echoing the text back. If the model has to repeat the description exactly, a single changed character breaks the mapping. An index can be validated in one line.
2. Make the category an enum built from the user's categories. The model physically can't invent a category, and your code still double-checks.

Batching

One call per transaction is simple and wasteful. One call for a whole statement can hit output limits and makes a single failure expensive. Nora sends batches of 50 descriptions. A failed batch only affects those 50, and they stay uncategorized until the next run.

Failure modes, decided up front

If this happens	The system does this
No API key configured	Skip the AI step. Uploads and rules keep working.
The model call fails	Log it. That batch stays uncategorized.
The model returns an unknown category	Ignore that item.
The model says "Sin categoria"	Leave it for the user.
The user corrects a category	Mark it as manual. Automation never touches it again.

Design checklist

• The request path never waits for the model.
• Parsers are deterministic and report errors per row.
• Uncategorized is a valid state in the schema.
• Every categorization stores who decided it and how sure they were.
• Uploads are idempotent.
• The model's output is constrained by a schema and validated again in code.
• Answers become rules.

In part 3 we write the code: the rules engine, the LLM call with structured output, and the background service.