How to Vet Software Developers When Everyone Uses AI

Our coding exercises for developer candidates explicitly allow ChatGPT or any other AI tool. Many people expect that to make candidates harder to evaluate. It hasn't. The clearest signal wasn't candidates failing to produce code. It was candidates who produced working code but couldn't make a simple change to it later.

That matters well beyond our hiring. If you're paying a company to build or maintain your software, the people writing the code are using AI too. So how do you vet software developers when everyone uses AI? Here's what our interviews have taught us, how we check the developers who work on client projects, and what to ask any development partner before you sign.

How we interview developers

We don't use algorithm puzzles or trivia questions. We focus on core concepts and on whether a candidate understands the "why" behind their work. If someone hasn't used a particular technology, that's fine. What matters is how well they understand the things they have worked on.

So we use short practical exercises, which we vary from candidate to candidate, built on simple operations such as inserting data and querying it back. They show us how well a candidate reads requirements, uses online resources, writes working and maintainable code, and structures a project. Then we talk through the code with them.

What allowing AI revealed

1. Working code that they couldn't change

During the interview we ask candidates to make a small change to their own submission: add a column to a result set, or add an input field to a form. This is everyday work. Most of a developer's time goes into changing code that already exists.

Many candidates who submitted a working solution couldn't make that change. The code ran, but they didn't understand it well enough to extend it. That's the most telling result of the whole process, because changing existing code is exactly what a client project needs, week after week.

2. Submissions that look the same

Most submissions are strikingly similar: the same class names, function names, variable names, indentation and overall approach. The main difference between them is how much of the requirements they cover. Some include everything; others miss key parts.

That pattern points to AI-generated code submitted without a careful review against the requirements. The tool did the writing, and nobody checked that it did what was asked. On a real project, that's how a feature ships that looks finished but misses half of what the business needed.

3. Basic engineering practice is weaker than expected

Our exercises ask for source code only, without compiled output. A surprising number of candidates, including some with more than ten years of experience, submit the build output along with the source.

On its own that's a small mistake. But it points to something broader: the everyday practices around the code, such as source control, build artifacts, deployment outputs and project structure, are often weaker than you'd expect from an experienced developer. AI tools don't help much here, and these are the habits that keep a client's codebase clean and deployable.

4. Where the candidate ends and the tool begins

In one online interview, a candidate appeared to be getting help from tooling during the live session: speech-to-text was running, and code changes appeared without any visible typing. We couldn't tell exactly what was happening. But it showed how hard it's becoming to separate what a candidate can do from what their tools can do.

Why "they finished the exercise" isn't enough

Some will argue that a candidate who uses AI to complete the exercise has shown they can do the job. We don't agree. Completing a well-defined exercise with fixed requirements is very different from working on a real system: understanding code someone else wrote, working out what the business actually needs, and adapting when the requirements change halfway through.

The exercise is only the starting point. The conversation about the code afterwards is where we learn whether someone understands what they built.

We see the cost of that gap from the other side, too. When we take over an application another team built, the problem usually isn't that the code doesn't run. It's that nobody fully understands how it works, so even small changes become expensive and risky. Over the life of an application, the ability to explain, modify and maintain code is worth far more than the ability to generate it quickly.

What we look for now

If you want to know how to vet software developers today, this is where we've landed. Allowing AI didn't break our interviews. It changed what we pay attention to:

• The conversation, more than the submission. A working solution tells us little on its own.
• Live changes. A small modification to their own code shows understanding faster than any question.
• The reasoning. Why this structure, why this query, what would break if the requirements changed.

Most strong developers today use AI tools heavily, and they should. So the question is no longer whether a developer uses AI. Nearly everyone does. The differentiator is whether they can evaluate what the AI produces, adapt it to changing requirements, debug it when it fails, and explain the reasoning behind it. That's what we hire for.

It carries through to how we work on client projects. Code is reviewed before it's merged, every feature is tested against the agreed requirements before it goes live, and clients see working software in regular demos throughout the project. AI tools help our developers work faster, but the developer who submits the code is accountable for it.

What this means if you're hiring a development partner

The same gap shows up after hiring. A team that leans on AI without understanding the output can deliver a demo quickly, then slow down sharply when the first change requests arrive. Before you sign with any software company, ask:

• How do you vet your developers? Listen for practical exercises and a conversation about the code, not only a résumé screen.
• Who will actually work on my project? Ask to meet the technical lead and the developers, not only the sales team.
• Do you use your own employees or subcontractors? Every developer on a SharpQuest project is our own employee; we don't subcontract.
• How are code reviews done? Who reviews each change, and what do they check before it's merged?
• How is AI-generated code validated? A good answer covers review, testing against the requirements, and who is accountable for what gets merged.
• What happens when requirements change mid-project? Requirements always change once people see working software. Ask who writes up a change, how it's estimated and who approves it.

For the full list of questions to ask before you sign, see our guide How to Choose a Custom Software Development Partner. To see how our teams are set up and how reviews and demos run, read How We Work.

Frequently asked questions

Do you let candidates use ChatGPT in technical interviews?

Yes. Our coding exercises explicitly allow ChatGPT and other AI tools, because developers use them on real projects. We then ask candidates to explain their code and make small changes to it during the interview, which shows whether they understand what they submitted.

Is it a problem if a developer uses AI coding tools?

No. Most strong developers use AI tools heavily. The risk is a developer who can't evaluate the output: one who can't tell whether the code meets the requirements, can't debug it when it fails, or can't change it when the requirements change.

How do you vet software developers when everyone uses AI?

Give a short practical exercise, allow AI tools, and then talk through the code. Ask the developer to explain their choices and make a small change to their own code live. That shows whether they understand what they submitted, which a working solution alone doesn't.

How can I tell if a software company's developers really know their craft?

Ask how they vet developers and whether the people on your project are their own staff. Ask to meet the technical lead before you sign, and ask how code is reviewed and tested before it reaches you. Clear, specific answers are a good sign; vague ones are not.

Are SharpQuest developers employees or contractors?

Every developer on a SharpQuest project is our own employee. We don't use subcontractors.

---

Knowing how to vet software developers is one part of choosing a partner; the other is seeing how they work. If you're planning a new application or need a team for an existing one, talk to us. You'll meet the people who would work on your project, and you can ask them anything in this post. If you're a developer who likes being asked "why", see our open positions.