Let's say I've got whatever question, usually about AI; here are real examples of questions I've had

• How are people testing their coding agent prompts?
• Is there a tool similar to claude-dev-tools but for OpenCode?

Well, I have three main ways of getting an answer:

• I can ask AI
• I can google it
• I can go onto a web forum and ask.

I'm finding that for each of these approaches - chances are, I'm getting an AI generated response.

• Asking AI - is explicitly an AI response
• Googling - I'm going to be seeing the best SEO answers - the SEO being driven by AI
• Going on a web forum and asking - increasingly seems to be AI bots.

This makes it hard to assess whether people are actually getting value form that answer - or if it's just the convincing expert opinion of an AI.

Similarly - I can find various tools online, but again it's hard to assess whether anyone is getting value from them, or if it is a vibe coded side project with no real battle testing.

It's less of a problem for regular pre-AI software engineering concepts

Recently I've been doing more with telemetry and observability.

I use chat-based AI for this and generally I trust the answer I'm getting.

It's more a problem for new fields - such as AI itself

AI is a fast moving field - so it's fair to say 'of course you can't get a clear picture of what the consensus is - there is no consensus'.

It's a bit like bad old days of JavaScript web frameworks, where every three months or so it felt like there was a new framework, and if you backed the wrong horse you were stuck with a tool that the industry left behind. Like that, but on steroids.

And there has always been a bit of a problem of cargo cult programming - people adopting a technology without really knowing what problem it is solving.

But at least back then there were a handful or tools, and they were industry wide.

Now, the problem is:

ease of vibe coding a tool + lack of human connection = the same problem being solved repeatedly by ten thousand different people

The AI companies probably love this - rather than having one team use AI to vibe code the tool and the rest of the world use it - have one thousand teams each use AI to vibe the cool and each be using AI to maintain said tool.

The solution

I think in-person meetups are more important than ever.

Presenting at a meetup 'hey look at this tool I vibe coded, here's how I did it' I think is valuable signal, it gives us a grounded picture of what's actually happening in a fast moving industry.

Questions? Comments? Criticisms? Get in the comments! 👇