Building a LangGraph portfolio agent that has to show its work

A portfolio chatbot has an obvious failure mode. It can invent a better version of its owner.

I wanted the small robot on this site to do something narrower. A visitor can ask about my public work, get a short answer, and open the exact page behind each claim. If the site does not support an answer, the robot says so.

That rule shaped the system more than the model did.

The product goal

The robot answers questions about the CV, project pages, and Engineering Notes published on this website. It can help a visitor find professional experience, technical decisions, or evidence related to a role without reading the whole site first.

It is not a general chat service. It does not answer unrelated questions, infer sensitive personal details, or treat a previous model answer as a source.

The product rule is simple. A factual answer needs public evidence. When the evidence runs out, the answer stops.

The robot lives in a bubble that is available across the website. It is not a separate page. During implementation I left a /portfolio-agent redirect in the site configuration. That was the wrong shape for the product, so I removed it. The route now returns 404. This article is the public page that explains the work.

Why it stays inside Next.js

The website already has a Next.js server, so the agent runs through the same-origin /api/portfolio-agent route. The provider key stays on the server. The browser never receives it.

The current implementation uses Next.js 16.3.3, React 19.2.3, LangGraph 1.4.8, the OpenAI JavaScript SDK 6.49.0, and Zod 4.3.6. A separate service would add another deployment, another credential boundary, and another network hop. This agent does not need those costs yet.

LangGraph handles the conditional workflow. Ordinary TypeScript handles retrieval, validation, safety rules, source checks, and logging.

The graph

The short version looks like this:

request
  -> validate and apply hard safety rules
  -> classify the intent and write a retrieval query
  -> retrieve approved public evidence
  -> draft an answer
  -> check every source reference
  -> complete, degrade, fail, or retry once

The real graph has a few branches that matter.

Invalid input stops before retrieval. Requests for private data, hidden instructions, shell access, or arbitrary URLs stop before the classifier. An in-scope question with no useful evidence returns a degraded result instead of a guess.

Some common questions take a deterministic path. Other questions use the model to write from selected evidence. No model draft reaches the visitor until the source check accepts it. Cancellation can stop the active path, including a retry wait.

The final states are explicit: complete, degraded, or failed. A failed run does not return 200 OK with an apology inside the answer text.

Public evidence and retrieval

The corpus comes from files already used to build the public website:

• structured CV records
• project records
• the article index
• approved article sections
• selected public page content

Private CV contact records stay out. The only contact address in the corpus is the professional email already published on the site. A regression test checks both sides of that boundary.

Each record has a stable ID, a public URL, a title, a type, and selected text. The model cannot add records or ask the server to read another file.

Retrieval uses exact text search and vector search over the same allowlisted records. Exact search is good at names, technologies, and measured results. Vector search helps when a visitor uses different wording in English or Spanish.

The vector index is a versioned file in the repository. Each record has a 512-dimension embedding and a hash of its source text. The server compares those IDs and hashes with the current corpus. A missing or stale index cannot silently answer from old content. The graph falls back to exact text search when vector retrieval is unavailable.

For each question, the server ranks up to 24 candidates from each search method. It combines both rankings with weighted reciprocal rank fusion, which rewards a record when both methods rank it well. The current page gets a small fixed priority when the question refers to "this page." The graph receives no more than six records.

Only the query embedding needs a provider request. Vector comparison, ranking, permission checks, and fusion run inside the application process.

There is no vector database. The corpus is roughly 100 public records in the current build. At that size, another database would create more synchronization work than retrieval value. A local index is enough to test semantic retrieval, fusion, versioning, and fallback behavior.

Retrieved text is data

Source text can contain hostile instructions. The server still treats it as data.

The model has no browser, shell, or arbitrary file tool. The retrieval function accepts only known source IDs. The model instruction labels retrieved text as evidence, not authority.

One regression test places a prompt-injection attempt inside a source record. The test checks that the text remains citable content and does not change the graph's instructions.

Source checks

Generation is followed by a source check.

The model must attach a source ID to every factual line. The server then checks that the answer has text, that every cited ID came from the selected evidence, and that each factual line has a citation. Any number in the answer must also appear in the cited source.

A rejected draft does not stream as a normal answer. The graph can make one controlled retry or return a degraded result.

This checker has a clear limit. It catches missing citations, unknown IDs, and unsupported numbers. It does not prove that every paraphrase preserves the source's meaning. That still needs evaluation and, for important decisions, a person opening the cited page.

Deterministic answers

The model helps interpret ordinary questions, but it does not need to write every response.

Code returns fixed, source-backed answers for several common cases. These include public contact requests, questions about failures, questions about conversation memory, and some identity or experience questions.

For example, the classifier can understand that "How can I connect with him?" is a contact request. The answer code then reads the published address from its approved record. A second model call is unnecessary.

The Privacy and Site terms pages use a similar path. The verified answer can appear first. An optional model addition is allowed only if it passes the same source check. If that request fails, the verified answer remains on screen.

Follow-up questions

Short follow-ups such as "And frontend?" need context, but the server does not need a full transcript.

The browser keeps a small display history for the current tab. It stores up to three completed questions, answers, and local source links in session storage. The Clear control removes it, and closing the tab removes it.

For a new request, the browser sends one source-backed topic anchor and no more than two recent user questions. The server validates and limits that context before using it.

Conversation context can clarify a question. It cannot become evidence. Previous answers are never sent back as facts that the next answer may cite.

Streaming and cancellation

The route uses Server-Sent Events. Status and approved sources can appear while the graph works.

The server does not stream raw model tokens. It buffers the draft, runs the source check, and only then sends accepted text. This makes the first visible answer slower and prevents unvalidated text from appearing first.

The browser owns an AbortController. Its cancellation signal reaches the route, graph, provider request, and retry delay. The server releases request capacity when the work ends.

Request controls and privacy

The public route checks the request origin, content type, JSON shape, and input size. It also limits request rate, concurrent work, and total execution time. An environment switch can disable the agent without removing the rest of the site.

The exact control values are deployment settings, so this public article does not publish them.

Structured logs record the request ID, graph step, status, duration, source count, retrieval mode, and failure category. They do not record the raw question, answer, source text, client address, headers, email address, vectors, or environment values.

Optional Langfuse traces follow the same rule. They measure workflow behavior, provider use, retrieval, latency, and source-check results without storing the visitor's conversation.

Website analytics and agent traces have different jobs. Analytics measure page activity under the site's consent rules. Traces measure whether the graph behaved as designed.

Evaluation

The repository tests the graph, retrieval, safety rules, source checks, conversation bounds, streaming, cancellation, telemetry, and browser behavior.

The main commands are:

pnpm test
pnpm eval:portfolio-agent
pnpm check:index:portfolio-agent
pnpm eval:portfolio-agent:hybrid
pnpm fuzz:portfolio-agent
pnpm smoke:portfolio-agent
pnpm lint
pnpm typecheck
pnpm build

The fixed cases cover English and Spanish questions, typing errors, page references, follow-ups, hiring questions, unrelated requests, prompt injection, private-data requests, unsupported judgments, unsupported numbers, and cancellation.

A larger generated set exercises routing and safety boundaries. It is useful for finding regressions. It is not evidence that the model answers 100,000 questions correctly.

The small hybrid-retrieval evaluation contains nine English, Spanish, exact-match, and semantic questions. In the verified local run, exact retrieval reached 88.9 percent Recall@6 with a mean reciprocal rank of 0.704. Hybrid retrieval reached 100 percent Recall@6 with a mean reciprocal rank of 0.833. Those numbers describe nine fixed questions in this repository. They are not a general accuracy claim.

Provider latency varied between runs, so the command reports it without turning one fast run into a performance claim. The same evaluation also forces an embedding failure and checks that exact retrieval still works.

What LangGraph helps with

LangGraph is useful here because stopping is part of the product.

Retrieval can find nothing. The source check can reject a draft. A provider call can fail. A visitor can cancel. Each event sends the graph to a named state that a test can target.

It also keeps workflow decisions out of the HTTP route. The route deals with HTTP and Server-Sent Events. The graph decides what the agent should do next.

What LangGraph costs

This project does not need long-term memory, multiple agents, model-selected tool loops, or durable checkpoints.

A small state machine could implement the same behavior with fewer dependencies. LangGraph adds state schemas, graph-builder types, and another execution layer. I would not add it to a plain endpoint that only retrieves records and asks a model to summarize them.

Here the conditional paths are the subject of the project. The repository explains and tests the graph itself.

Updating the knowledge

Articles, projects, CV records, and approved page text can all change. The robot needs the same release to include those changes.

The normal pnpm build command now runs a knowledge check before Next.js builds the frontend. It compares the public corpus with the committed embedding index.

If nothing changed, the check makes no provider request. If public evidence changed, it rebuilds the index and validates the new file. A release without the required provider key stops before deployment instead of shipping stale vectors.

That makes the update path ordinary. Edit the site, run the release build, and the build either produces matching knowledge or fails with a specific reason.

Limits and next steps

The robot can only know what the public site says. A citation checker cannot prove every paraphrase. A nine-question retrieval set is useful for regression testing, not broad quality measurement. The same-origin API also belongs to this one site. If several projects need AI later, a private gateway with separate credentials and per-origin policy would be a cleaner shared boundary.

For now, the robot has one job. It answers from the public site, shows where each claim came from, and refuses to make the portfolio sound better than the evidence allows.