To tell this week's story we'll need to take a quick detour back to 2024.
Flashback
It was the summer and I was between jobs. I had a friend that was running a boutique startup specializing in building chatbots for non-profits (to help them get a better handle on their data). It was still early days in AI coding agents, so a fair amount of the work was still done by hand along with a lot of trial and error to dial the chatbot results. It was a tiny team, and my friend was the primary engineer. He needed help, so I joined part-time.
All manner of what he was doing was completely new to me, but it's all deemed old-hat nowadays.
E.g.,
Build a scraper & plumb it to the different places your documents are stored
Ingest said documents
Cut them up into relevant chunks
Run the chunks through a vector embedding model
Store the embeddings in a vector data store
Use the data store for semantic search as part of a question given to a model
Include the results as part of the model's answer to a user's question about the documents
a.k.a. Retrieval Augmented Generation (RAG)
I stuck around long enough to glean a fleeting understanding of the approach, helped him with the infrastructure build out to put it in front of customers, and then moved onto a different company - where I am today.
Questions Without Answers
While ramping at my current employer, I was getting up to speed on the product and kept coming hitting a point of friction. I'd have a question about some feature in the product, go look for the answer, and struggle to find it. I might eventually find it but only after a fair amount of struggle. Or I'd realize the answer wasn't in an obvious place (e.g., not in the user manual but in a knowledge base article, or a blog post). The information existed, but it had sprawl.
It became clear to me that we needed a better solution - something akin to what my buddy was building at his startup. But just as quickly as this idea came, it went. I knew conceptually what needed to be built but not enough of the fundamentals to feel confident that I could build it quickly (or well). We didn't have anyone else on my team with experience in this technology. We didn't have budget to hire out for it. There were more pressing priorities. So it got shelved.
Existential Dread
Now jump cut to January 2026. AI coding agents went from really good to freakishly effective. My job as a software engineer had fundamentally changed whether I wanted to acknowledge it or not. It was slowly happening leading up to this point. But now it was inevitable. Either I would need to change, or I would become irrelevant.
Like many of my peers I went through what felt like the stages of grieving. I was mourning the loss to a way of doing things, and with it, a sense of identity. I wasn't sure what to do and what would come next. On the other side of these feelings I realized a few crucial things:
It was clear to me that this wasn't simply a departure, but a revolution of the work
This would be true only if I learned the fundamentals of AI Engineering and the new paradigms of work that it creates
The thought of working with this technology was immensely exciting to me
All In
So I went all in. I signed up for an immersive multi-week hands-on cohort based course on AI Engineering, started building things as I learned, and I haven't looked back since.
Fast forward to this week. I did a speed run on building that chatbot for my employer. I deployed it this morning and rolled it out to a small group internally to start kicking the tires—and it's working (read: giving helpful results).
Building it wasn’t the hard part. Getting it to work reliably across a messy, unpredictable set of real questions—that was the trick. It turns out the gap between "it works" and "it’s actually useful" is where most of the work lives.
I’ll dig into that in a future post.