Shipped an Agentic Sports Analytics Application in 5 weeks

The Problem

EdgeSon is a product for serious sports bettors. Users type a question in plain English and get an answer built from real data, with the data attached. Before this, their options were a dashboard that answers the questions its designer thought of, or a person with spreadsheets and a few free hours. Most bettors have neither, so most of their questions went unanswered. A typical session is a bettor preparing for the week ahead: ask, get the numbers, analyze them, then ask the next thing.

The Solution

The process begins by a user asking a question through the interface. The interface then sends the question to the agent. The agent reads the intent, makes a plan, and then takes action, whether that is requesting data from a backend or answering questions based on the history. Queries that go to the backend are validated and run under a read only role. The full results are pulled and sent back to the agent. The agent gets a small sample of the results along with any necessary highlights for analysis and then the agent response along with the completed table is sent back to the user through the interface.

Solution Highlights

The Most Critical Factor is Answer Correctness (Quality).

People use EdgeSon to decide where to put money. A wrong answer loses a user, in the good case, and loses them money in the bad case. So data quality and analysis accuracy must be prioritized. Security, speed and cost were constrained against them.

Correctness Begins with Data.

The MVP runs on open source data; as users scale, the same pipeline can pull paid feeds with real time streaming. Using multiple data feeds requires internal standardization so that internal applications can speak a unified language. This meant deciding what the internal database should look like before touching any source, then mapping each source into that shape and cross referencing values across sources without duplicating them everywhere. That became four Postgres schemas with separate roles:

raw (verbatim external data)
core (standardized tables, with sport specific schemas where structures do not generalize)
semantic (views over core, the only surface the agent sees)
app (users, chats, billing)

Swapping a free source for a paid one never touches the agent. This layer took two of the five weeks.

Analytics Data is most useful if you can slice it any way the question demands but this comes with security concerns

A serious bettor wants the ability to look at the data from multiple angles. Using REST or GraphQL would require answers to questions that someone predefined but there are an infinite number of questions that could be asked. Meaning the trade off was either to limit the question space, which would make this a narrow analytics product, or to provide maximum flexibility by allowing raw SQL to be created via an agent, at the cost of security.

Running raw SQL from a language model is dangerous, so everything around it exists to make that safe.

Because agents are pleasers, they often do things that are not good to do. In this instance, by giving an agent the ability to run SQL directly against your, ultimately, database, you are opening the door for your agent to do some very harmful things, or, and more realistically, whoever is using your agent. This means that we need to add protections against harmful activities. Here are some examples of what was done:

Agents only see a semantic view of the data
Queries pass a validation layer that rejects harmful patterns, then execute under a read only role.
Rate limits and timeouts stop one heavy query from starving everyone else. A bad or hostile query wastes fifteen seconds and nothing more. The tradeoff is that some legitimate queries get rejected and every query carries validation overhead. If we measure the time of our queries properly this can be mitigated.

Safe data does not mean good answers, so I made it easy for the agent to do the right thing and hard for the agent to do the wrong thing.

Every Phoenix trace is broken into four stages: intent, plan, action, result. The rule: anything the agent should never do becomes a code constraint; anything it does oddly becomes a prompt change. The agent once looped on a failing query and burned tokens, so a retry cap went into code. It also returned a fraction of the data because it treated its 20 row sample as the full answer, so the prompt now tells it to narrate the data the user is about to receive rather than return any of it. I then return the data through separate methods.

Correctness is expensive in tokens, so cost was engineered after quality was locked.

The system prompt is about 28k tokens; a question and answer are about 1k combined. Caching the system prompt, prior question, answer, and query results cuts token spend roughly 10x. The agent never sees a full result set, only a 5 row sample (users can ask for more) plus the highlights, enough to explain how to read the table and what to focus on. The full table goes to the user through the app for free. That lands at about $0.15 per question.

Speed took the back seat on purpose but it is still faster than the alternative.

Average response is about 15 seconds, longer for complex pulls. This can be improved in future versions. Relative to the real comparison of an analyst who has to source, slice and read the data first, or versus the question never getting answered at all, 15 seconds is pretty good.

Cost and build time

Running cost

Fixed costs today are about $40 a month, assuming early stage usage on shared infrastructure. AI costs is where the real expenses are.

Database: RDS db.t4g.micro, free for the first year, then about $14 a monthwith 20 GB. At 10x usage the 1 GiB of RAM runs out before CPU, so the move is db.t4g.small at about $28 a month with 30 GB.
Hosting: Railway and Vercel, about $20 a month each, not dedicated. Fine to 1k to 10k users; past that Railway needs a dedicated setup.
Auth: Auth0, free to 25,000 monthly active users. 1k to 10k stays free unless I need MFA or separate dev and prod environments, at which point Essentials runs$35 plus $0.07 per user above 500: roughly $70 a month at 1k and $700 at 10k.
Model calls: This is the real lever, and it scales with questions, not users. At $0.15 per question, a $25 subscription covers 166 questions a month before it is underwater on tokens alone, about 38 a week. Fixed costs excluded.

Build time

Five weeks, part time, one person, mid July to launch before football season.

Data layer: 2 weeks.
Agent: 2 weeks.
ELT pipeline, query backend, frontend, deploy: 1 week combined.

The time went where the correctness risk was.

Working on something like this?

I build full stack AI systems for companies that want to improve how they work or do things that were not possible before. EdgeSon is a sports product, but the pattern is not. The method behind it translates to any industry with data and questions to ask of it. If you are working on this kind of problem, message me on LinkedIn or shoot me an email. Let’s see what we can build.

Want to see the product first? Visit edgeson.com.

Syd Reynoldssydreynolds@merceng.com