After studying Andrew and the entire Anthropic team, I’ve been obsessed with building my own self-improving knowledge base inside my Claude system. So finally, now we can have our knowledge base run on a schedule, self-improve by itself, and basically sync our context without us having to manually prompt it every single time. In this post I’m going to show exactly the four steps I used, the prompts that made it work, and the biggest lesson I learned from building this whole system myself.
The whole thing sits inside my Claude setup, but the principles apply to any AI agent that can read files and run scheduled tasks. I’m not just talking about dumping some documents into a folder. I mean a living, breathing knowledge base that pulls new data, looks at its own output, spots gaps, and cleans up stale info while you do something else.
1. Build the folder structure that actually makes sense
The first step is to set up the knowledge base folder. This is where we’ll store all our data and let the AI enhance it over time. The concept comes from Enricography over on X, who introduced the idea of a large language model knowledge base with a clean separation of concerns. The structure is stupidly simple, but the power is in how you customize it.
We start with a main knowledge base folder. Inside that, we have two subfolders: raw and wiki. The raw folder holds every unprocessed source you throw at it (transcripts, PDFs, plain text, Markdown notes, exported data, anything). The wiki folder is where the large language model synthesizes what’s in raw and writes its own indexed, cleaned-up reference pages. These wiki pages can still point back to specific raw files, so the model always knows exactly where to look.
Every time you ask a question about your knowledge base, the AI agent is going to look through the wiki folder to find the answer. It doesn’t need to re-read thousands of raw documents. It has already built a condensed, searchable layer. Next to those folders, you also keep a cloud.md or agents.md file (depending on whether you’re using Claude or another agent). That file contains the master instructions on how the AI should interact with your knowledge base.
Now, here’s the part that everyone else online doesn’t talk about: everyone’s knowledge base is really different. In my own system, I don’t just have one generic wiki. My main knowledge base folder is inside my Eric Tech OS workspace, a single master folder where I keep every project I’m working on (content creation, coding, business, personal, research). Inside the knowledge base folder, I still have the raw folder, but my wiki is split into subfolders like business, personal, and research. Each one of those wikis references its own slice of the raw folder.
Customization is not limited to just raw and wiki. You can structure it based on your exact needs. The concept stays the same: raw for unprocessed junk, wiki for model-indexed summaries. But you can name the wiki folders however you want, create per-domain agents.md files, and even spread the knowledge base across multiple projects. That flexibility is the real secret.
I use a prompt to set up the entire folder structure inside any existing project or a brand new one. It walks the model through creating the raw/wiki pairs, the agents.md, and linking everything so that future operations just work. I’ll make the prompt available in my community, but you can already guess the gist: it tells the AI “here’s the layout, now scaffold it and keep everything organized.”
2. Create the ingest skill so you never hand-prompt data again
Once the folder structure is ready, you need to fill it. That’s where the ingest skill comes in. If you open Claude Code pointed at your knowledge base, you can manually drop a file, tell it to “ingest that into my knowledge base,” and wait for it to put the raw file in the right place and update the wiki. But who wants to type that same instruction over and over?
Instead, you can package the entire ingestion routine into a skill called “ingest.” Here’s what the prompt does: you hand it any file, a block of text, a Markdown document, a PDF, or even an image. You trigger the skill, and it automatically figures out where to store the raw version, then processes it into the appropriate wiki. You don’t have to remind it about the raw folder, the wiki folder, or the agents.md instructions. The skill enforces all of that.
For example, if I have a transcript from a podcast I did, I just drop it into Claude Code, type /ingest, and the system files the raw audio notes and creates a wiki entry that summarises it, tags it, and cross-links it with anything relevant in the existing knowledge base. The same thing works for meeting notes, research paper summaries, or even exported emails.
This skill saves you from the repetitive prompting that would otherwise drive you nuts. It also guarantees that every ingestion follows the same rules. Your knowledge base stays consistent because the AI doesn’t get lazy and put things in the wrong place. The whole prompt for the ingest skill is one of the building blocks I’ll share in the full course, but the pattern is straightforward: define the skill name, tell it where raw goes, where wiki goes, what to do with different file types, and how to update the agents.md when new topics appear.
3. Feed the knowledge base with the right data sources
A knowledge base is only as good as the context you pour into it. The whole point is to make sure your AI agents help you make the right decisions and give you responses that actually match your world. The more accurate the context, the higher the accuracy of the agent.
So what can you actually add? I’m not talking about the same five Wikipedia articles everyone uses. I’m talking about the data you already generate every day that sits locked away in other tools.
Google Takeouts (and any email export). If you use Gmail, Google Calendar, or Outlook, you’ve got years of history sitting there. You can download a Google Takeout archive, feed it to your AI, and have it extract the knowledge. The AI can find patterns, recurring contacts, project references, and decisions that you probably forgot you made. That data doesn’t have to be cleaned. Just dump the archive into the raw folder and let the ingest skill process it. The model will figure out what matters.
Past session histories. This is one of my favourites. Every time you interact with Claude Code or Codex, your sessions are stored locally. In Claude Code, you can type /resume and see your past sessions. Those conversations contain real workflows, debugging patterns, and decisions you made over weeks of work. With the right prompt, you can have your agent analyse all those sessions, pull out the repetitive workflows, identify the patterns, and add them to your knowledge base.
I actually ran this on my own system. I pointed the agent at both my Claude and Codex session histories. After chewing through everything, it came back with a list of repeated workflows I hadn’t even noticed. Some of those turned into skills. Others became permanent wiki pages so the agent can reference them later without me re-explaining anything. The prompt for this is basically: “Go through all my session histories, identify the recurring tasks, patterns, and decisions, and synthesise them into the knowledge base. Only add things that are actually useful.”
MCPs and connectors. If you’re using Claude Code, you can connect it to a ton of data sources through MCPs. Inside Claude, just click “browse connectors.” You’ll see Slack, Microsoft 365, Gmail, Notion, Monday.com, and dozens more. Once connected, you can instruct your agent to pull data from those tools and ingest it. For instance, you could say “Pull the last 30 days of my Slack messages from the product channel, summarise the decisions, and add them to the business wiki.” The agent calls the connector, grabs the info, and runs the ingest skill. After a few days of scheduled runs, you’ve got a living reflection of what your teams are actually talking about.
Before we move on to the self-improving loop, I want to quickly shout out Verlo. Here’s a problem I ran into with Claude for content research: Claude can help you write scripts, plan videos, and brainstorm angles. But if you ask it what is performing right now in your niche, it needs access to real social data. That’s where Verlo fits into my workflow.
Verlo is a social listening and analytics platform for TikTok, Instagram Reels, YouTube Shorts, and Meta Ads. It tracks short-form trends, creators, competitors, outlier hooks, hashtags, and performance patterns so you’re not guessing from a blank prompt. The workflow I use is simple: I connect Verlo to Claude through MCP. Then I ask Claude something like “Using Verlo, find the top performing video ideas in the AI tools niche from the last 60 days. Give me the hook, angle, platform, view count, and why each one worked.”
Now Claude isn’t just making up generic content ideas. Verlo gives it live trend intelligence, and Claude turns that into useful things: video concepts, ad scripts, carousel ideas, creator briefs, or a full content plan. When I’m building a video about Claude Code or any AI productivity workflow, I can use Verlo to see which hooks are already breaking out across shorts, TikToks, and reels. Then I ask Claude to turn those patterns into a script that fits my style instead of copying the originals. That trend data also becomes part of my knowledge base, so my agent gets smarter about what content works over time.
The important thing is that all these data sources (takeouts, sessions, connectors, third-party tools like Verlo) flow into the same knowledge base through the same ingest skill. You don’t have five different processes. You have one pipe, and it works.
4. Set up the self-improving cron job
By now you have a working knowledge base, an ingest skill that can pull from multiple sources, and a growing collection of wiki pages. The last step is to make the whole thing self-improve without you touching it.
First, I create a skill called “improving system.” This skill is the brain of the self-improvement loop. The prompt tells the agent to, in a single pass, review the entire knowledge base, sync the existing session histories, run any available MCP connectors or data ingestion skills, look at recent outputs, and then figure out what needs to change.
The prompt I use is structured around a two-phase pass. Phase one: review everything. Look at the raw folder, the wiki folder, the agents.md, the current skills, and the latest data. This surfaces broken links, stale information, missing connections, and new opportunities. Phase two: refresh the available context by using the listed skills to pull fresh data from connectors, re-index anything that changed, and then produce a list of improvements.
Those improvements fall into three buckets:
- Auto-apply: small, low-risk fixes. The agent can just make them. Maybe a wiki page has a broken reference to a raw file that moved. Fixed automatically.
- Needs approval: anything that touches a skill, creates a new skill candidate, or significantly rewrites a wiki page. The agent flags it so I can review. I don’t want things changing behind my back.
- Needs context: ambiguous decisions where the agent can’t safely proceed alone. It then prompts me with a direct question.
This prevents the agent from going rogue and messing up weeks of careful curation. After I tested the skill manually a few times and saw it working, I moved on to scheduling.
In Claude Code’s co-work, there’s a “schedule task” option. I created a new task, gave it a name and description, and in the detail prompt I simply told it to trigger the self-improvement skill. You can set it to run daily, weekly, whatever suits you. The prompt looks something like: “Run the a self-improving pass across the system. Use the improving system skill. If any issues need approval, flag them. Do not create new connections without approval.” Then you attach it to the right project folder, set the frequency, and save.
The key is to test everything manually before you put it on a cron job. Make sure your ingest skill works from end to end, make sure the connectors pull data correctly, and make sure the agent doesn’t hallucinate file paths. Once you’ve seen it run cleanly at least twice by hand, schedule it and let it fly.
Key takeaways
- Folder structure is non-negotiable: raw for unprocessed sources, wiki for LLM-indexed knowledge, and an agents.md for master instructions. Customise the wiki subfolders to match your actual life (business, personal, research, whatever).
- Package ingestion as a skill: don’t prompt manually every time. A single “ingest” skill handles any file type and keeps the knowledge base consistent.
- Use the data you already have: Google Takeouts, session histories from Claude Code or Codex, and MCP connectors to Slack, Gmail, Notion, and other tools. More real context means more accurate AI responses.
- Self-improvement loop with a cron job: create a skill that reviews the whole knowledge base, pulls fresh data, and flags issues. Schedule it in Claude co-work only after manual testing proves it’s reliable.
- Customisation is the part nobody talks about: your knowledge base isn’t some generic wiki. It needs to mirror the way you work. The folder layout, the skills, the data sources, all of it should flex around you, not the other way around.
Right now I’m still building out the level three AI research and knowledge base course. What I’ve shown here is just the first layer. Once the full course drops, it’s going to cover everything from advanced data structures to full research pipelines. If you want early access, the community is where everything lands first. Pretty much that’s it, liking, subscribing, all that stuff helps. I’ll see you in the next piece.