Blog

Part 2 of 3 · AI agents, from scratch4 min read

Context Is Everything

Your AI agent wakes up every session with no memory. How you organize its world decides how well it works. Here's what a good setup looks like.

Brandon Lopez — Founder, Lyte Works

  • ai-agents
  • context
  • claude-md
  • agents-md

Thought experiment: imagine you wake up every day not knowing who you are or what you did yesterday. You still have to continue the life you've created for yourself.

You wake up with the capabilities, skills and knowledge you've gathered over your whole life. You just don't remember any of it being yours. You have to find a way to function each day as if there was no forgetting.

So, what would you do? Think about it... Whatever you come up with is likely how you should organize the project your AI agent works in.

That's not a made-up problem. As we covered in What Is an AI Agent?, it's exactly how an AI agent starts every session.

The page by the bed

Here's what I'd want: a simple page to read the moment I wake up. Just the basics.

"You are John Doe. You're 34, and you work for Acme as a project manager. This page is who you are and where to find the essentials. Yesterday you were in the middle of the Smith project. Everything about that project, and how to pick it back up, is in your journal."

One page. It doesn't tell me everything. It tells me who I am and where to look for the rest.

For an agent, that page is your CLAUDE.md or AGENTS.md. Here's what one might look like for a bakery:

# Sunrise Bakery

You're the assistant for Sunrise Bakery, a family bakery in Dover, NH.
Owner: Maria. Open Tuesday to Sunday, 6am to 2pm.

- Menu and prices: menu/
- Orders and customers: orders/
- How we write to customers: voice.md
- What happened last time: journal.md

Read journal.md before you start. Write in it before you finish.

Short. It points everywhere and doesn't try to hold everything itself.

The messy room

Now imagine you left yourself a note: "Wear this shirt, these pants and these shoes to the event today." You read it, look around, and your room is a mess. You can't find anything. After an hour of searching, you finally find what you need. The frustration and energy you wasted is not a good way to start the day, and it's likely to carry over into the rest of it.

Think of that wasted energy as the AI wasting tokens trying to find the right information for the task you gave it. If your project is a mess, the agent burns a lot of tokens just finding enough to do the job. More tokens, more money, more usage. More tokens also means more context in the conversation, and the more there is, the more likely the model is to lose track or make things up. That's AI hallucination.

A tidy room for an agent is simple:

  • Folders with obvious names. One place for each thing.
  • A map. One file that says what lives where.
  • No duplicates. Two copies of the menu means one of them is wrong.
sunrise-bakery/
  CLAUDE.md     ← the page by the bed
  map.md        ← what lives where
  journal.md    ← what happened, what we learned
  voice.md
  menu/
  orders/

The end of the day

Now the day is done. What do you do? Journal what you did. Then update the page by the bed so tomorrow-you knows where things stand.

That second part matters more than it sounds. If yesterday's note says "wear the blue shirt" and you gave that shirt away, the note is worse than no note at all. An agent following stale notes will confidently do the wrong thing. The brief only works if it's kept true.

You, as that forgetful person, have a waking process, work to do, and a routine to close out the day, so you can live the next day to the best of your abilities. This is a program you created for yourself to function better.

That's a memory loop

Wake up, read the brief, do the work, write it down, update the brief. Repeat. That's a memory loop. It's the difference between an agent that starts from zero every time and one that gets better every time.

At Lyte Works, our agents run this loop every session. Remember the reflexes from What Is an AI Agent? The harness hands the agent its brief when it wakes up, and checks that it wrote in its journal before it's allowed to finish. Over time, the best of those journal entries get reviewed and promoted into lessons the agent carries forward.

In the next post, I'll show you how to build a simple memory loop in your own setup, step by step, based on how we built ours: How to Build a Simple Memory Loop.

Frequently asked questions

Why does an AI agent need context?

Because it wakes up every session with no memory of your business. It knows a lot about everything and nothing about you or your work, so how you organize its world decides how well it works.

What should go in CLAUDE.md or AGENTS.md?

One short page: who the agent is, where things live and where to look for the rest. It points everywhere and doesn't try to hold everything itself. If it's getting long, something belongs in its own file.

How does a messy setup cost you?

The agent burns tokens searching for what it needs. More tokens means more money and more usage, and the more context piles up in the conversation, the more likely the model is to lose track or make things up.

What is a memory loop?

Wake up, read the brief, do the work, write it down, update the brief. Repeat. It's the difference between an agent that starts from zero every time and one that gets better every time.

Want this handled for you?

Tell us what’s heavy. We turn it into Lyte Work.