Hi Reader,

In the past year, AI coding agents have made me feel everything from "wow, this is genuinely magic" to "this is the dumbest thing I've ever seen."

And a lot in between.

I've tried many things:

From full vibe coding (documented some of that on YouTube), to the more deliberate, step-by-step approaches.

I've built random web apps, Python projects, chatbots, data visualisations,...

I think right now, I have a much clearer sense of what actually works.

So today, I'm going to share my experience.

(Note that I'm not a professional software engineer, I'm working on projects for myself or prototypes of clients, mostly in Python.)

---

First, why is AI so good at coding?

Since the release of ChatGPT in late 2022, AI companies poured everything into scaling bigger models, chasing the AGI dream. That approach hit a wall somewhere in 2024.

And somewhere along the way, they discovered that the most valuable, most reliable use case for generative AI isn't even writing.

It isn't making videos or graphics either.

It's something much more... boring: Coding.

Coding tasks are structured. Text-based. The output can be verified and tested. All of that makes it the most tractable domain for AI to get genuinely good at.

But the models alone aren't the full story.

Even the most advanced LLMs like Claude Opus 4.8 are still prone to errors and hallucinations. What makes AI coding systems so powerful is the engineering wrapped around them: what people call coding harnesses.

Think of a harness as the giant scaffolding of hand-coded rules, if-else logic, regex routing, and tool orchestration that governs how the agent behaves, how it uses file systems, how it self-corrects.

The harness is the secret sauce.

Anthropic accidentally leaked 512,000 lines of Claude Code's source code earlier this year. I'm not the right person to do a deep analysis of it (there are people much smarter than me for that), but it does tell you something about how much engineering goes into making these agents actually useful.

---

How capable are they now?

In May 2026, Metr (an organisation that evaluates the capabilities and risks of frontier AI models) published a study on how long of a software task different LLMs can complete 50% of the time.

Their finding: Claude Opus 4.6 can successfully complete a 12-hour software task with a 50% success rate. Claude Mythos sits at around 16 hours.

(Btw, this graph sparked the usual wave of hot takes: "AI is eating everything", "No jobs are left", "AI is now improving itself recursively." These are wild extrapolations with no solid ground, so I'd take them with a generous handful of salt.)

This is to show that AI coding agents have actually gotten very much better in the past 2 years.

But they are certainly not perfect and godlike like how social media makes them to be. Remember, the Metr study above shows the task success rate of 50%, not 100%!

How to use them effectively in actual workflows, remains a massive challenge for most people.

---

Vibe coding: yes or no?

My simple rule of thumb:

If it's a tiny, throwaway tool → yes, vibe away.

I recently vibe-coded a small web app to help my Python students brainstorm ideas for what they want to build. It's literally just an HTML file. No sensitive data. Nothing stored or processed there.

For tiny tools like this, I don't need it to be secure or elegant. It just needs to work and kind of looks nice and fun.

If it's a real project with important logic → no.

Anything with a proper pipeline, data inputs, meaningful outputs, or people depending on it: I don't vibe-code the solution entirely.

Not because AI can't produce working code (it usually can). But because I don't want to face the consequence later: I won't be able to understand what's going on, spot mistakes, or fix things later.

Honestly, I hate fixing things I didn't write, and I hate trying to understand decisions I didn't make...

So for these projects, I work with the AI agent in small specific tasks.

The more important the task, the more I code myself.

---

My actual workflow

Here's how I work iteratively with Claude Code on my projects:

Step 1: I build the core logic myself.

I set up the project, install packages, initialise a Git repo, and write the core functionality, sometimes with Claude Code helping me autocomplete syntax for things like file loading or data transforming.

(If you read my last issue on turning an idea into a finished project, this maps to the same idea: build the core first, before anything else.)

This step is less about typing code and more about thinking: what's the input? What's the output? What does the shape of the whole pipeline look like?

I don't see the value in vibe-coding this part. You need to understand the foundation and know how it works before anything else.

Step 2: I ask Claude Code to generate a CLAUDE.md file.

Once the core logic works, I have Claude Code write a CLAUDE.md file for the project, and I adjust it if needed.

This file stores the full context of the project: what it does, how it works, the key frameworks, the file structure,...

This matters because Claude Code doesn't remember your project across sessions.

At the start of every new session, it reads from this file to get up to speed. Without it, you're re-explaining yourself every time.

Step 3: We work together on everything else.

From here, Claude Code does a lot of the heavy lifting on tasks that would cost me time but don't require me to make judgment calls:

• Scraping and data collection. For my GraphRAG project, I had Claude Code pull real article data with SerpApi. Saved me hours of reading documentation, cleaning and transforming data myself.
• New features. Things that are nice-to-haves, like adding a voice input mode or output summary.
• UI upgrades. Swapping a basic Streamlit interface for something like React.
• Visualisations. In my GraphRAG project, I let Claude Code build the full network graph. Something that would have taken me weeks on my own. (I made a similar app in my previous job, built over two months, yet honestly looked like it came out of the pre-Internet era. 🤣)
• Code cleanup. Adding documentation to functions, which I'm usually too lazy to do myself.
• Error handling and logging. So I can easily debug things later.
• README file. AI writes better documentation than I do. I've accepted this, lol.

---

A few things I've learned the hard way

1) Always read what it generates. It's tempting to just hit Approve. But coding agents aren't perfect. You need to catch issues early, before they compound into something messy.

Look for: overly complicated code, misunderstood intentions, hardcoded values that should be variables, API keys sitting right in your Python script...

2) Break the work into small specific tasks. Don't say "go scrape all articles about X and Y." Say "scrape a list of article URLs for these specific search terms using SerpApi." Then: "now extract the article text from those URLs and save to a dataframe."

Smaller tasks -> better output.

3) Start a fresh session when it gets dumb. As a session gets longer, the agent accumulates more context and can start making circular mistakes. It gets into something called "the dumb zone". I've found that shorter, focused sessions produce smarter output. If you're interested in more stuff like this, this talk is worth watching.

4) You need to know enough to read and understand the output. This is the bit people don't like to hear.

If you're new to coding, I'd advise agains leaning too heavily on AI agents, even "for learning." You learn the foundation by making your own mistakes. I'm afraid it's hard to outsource this to AI.

5) Beware of the cost. Even if you're a "power user", AI coding is not free lunch.

A data scientist friend told me recently she worries about losing her own coding instincts because she relies on AI so much. I get it.

Another challenge I face is, I also can't really get into a flow state with AI coding. You prompt, you wait 1-2 minutes, and in that gap you reach for your phone, check your email, get distracted.

I guess that's the cost we pay for working a little faster? 🤷‍♀️

---

My questions to you:

How are you using AI coding tools these days?

What are your challenges with these tools?

Let me know!

Chào,

Thu 😊

---