We tested six bots to see how they compare to real career coaches, asking everything from how to best handle sexual harassment to when to move on to a new job. We found that while the AI tools give decent boilerplate advice, they can also complicate issues or offer biased solutions.
How they did in our tests
Though not as powerful as human coaches, generative AI showed some promise as another tool for people struggling at work. The key is to use it as an idea generator or to get additional perspectives, but ultimately rely on your own judgment.
“Say, ‘This is my plan. What else should I think about?’” said Hatim Rahman, an assistant professor at Northwestern University’s Kellogg School of Management who studies AI’s impact on work. “That’s where I see the most promise.”
The AI tools specifically built for professional coaching walked us through issues step by step, often asking targeted follow ups. These apps typically guided us to solve our own problems, and avoided nudging us down a path. The more general apps like ChatGPT were likely to dump a bunch of information at once.
For sensitive subject matters such as harassment or suicidal thoughts, all the bots recommended contacting HR or mental health professionals.
Google SGE: The company’s generative AI search, which automatically shows up on the top of many Google searches, typically gave us a bulleted list of advice followed by links for more information. The links were to job sites, career service providers, and company and university blogs.
Copilot: Microsoft’s AI bot was similar, though it allows you to hold more of a conversation. After we asked it how to talk to a boss about micromanaging tendencies, we were then able to press it for things such as how to set boundaries. It spit out more steps, including possible wording: “I appreciate feedback, but I’d prefer to receive it during our weekly check-ins rather than throughout the day.”
GET CAUGHT UP
Summarized stories to quickly stay informed
ChatGPT: The free version of OpenAI’s app often loaded everything into one response, giving a list of tips, a sample script of how to have a difficult conversation and wrapping up with final thoughts. Similar to Copilot, you can follow up with additional questions or tweaks. (We asked for a less formal script for our fictional micromanager).
AIMY: When we asked AIMY — which is powered by tech from OpenAI, Anthropic and the company’s own models — whether to quit our job, the AI didn’t answer right away. First, it replied with questions about why we were asking, how we felt about our job and whether there were steps we could take to solve our issues. The app is owned by CoachHub, a company that connects people with human career coaches.
“If you need to manage a complex situation today, you don’t need to wait for your coach,” said Pedro Cabrera, senior vice president of data and insights at CoachHub.
Wisq: This app allows you to choose from a selection of topics such as “developing leadership skills” or chat freely with the bot. It similarly stayed middle of the road, suggesting a career-switch could be challenging but rewarding. It uses a combination of proprietary algorithms along with large language models from OpenAI, Meta and Anthropic.
“A very small percentage of employees have access to a human coach,” said Wisq CEO Jim Barnett. “We’re focused on giving the other 98 percent coaching.”
Rocky.ai: This tool stayed focused on specific goals, sometimes to a fault. That may be because the system, powered by its own models, only pulls from curated content, versus the entire internet, or data your company uploaded for corporate accounts, according to Harry Novic, founder and CEO of Rocky.ai. It uses major models like Google’s Gemini and OpenAI’s GPT to enhance specific conversational requests.
CoachHub’s AIMY, Wisq and Rocky.ai are available as mobile apps or on the web. CoachHub and Rocky.ai offer free versions, and all three have paid options.
Beware, if you’re using a consumer tool, your data may be used for training purposes. Corporate accounts often have a little more protection, but use your discretion when using company-provided software to ask sensitive questions about your job.
Why they’re not as good as humans
AI is no replacement for human career coaches, in part because of the technology’s habit of making things up.
OpenAI’s usage policy warns builders from using ChatGPT to facilitate activities that “may significantly affect the safety, well-being, or rights of others,” though it does not specify anything about professional advice. It warns that answers may sound right but be wrong and reminds users it doesn’t know everything. Microsoft says it agrees Copilot was not built to be a career adviser.
AI is also known to have issues with gender and racial bias in its answers. For example, when we asked ChatGPT for job recommendations for a woman and then a man, it gave gendered answers like more science and technology-related jobs for the man.
The bots were sometimes too general to be helpful, or too weird to make sense. Copilot didn’t know how to advise someone who may suspect they’re the victim of unconscious bias in the workplace, instead offering generic advice about unconscious bias like “educate yourself.” At one point Rocky.ai asked, “Which task will you prioritize today and draw upon your deepest reserves of discipline?”
In other cases, the follow-up questions became confusing and circular, almost baiting us to answer the original question by asking the same question back. In those cases with AIMY, we ended the loop with a simple, “I don’t know. Can you help?” which prompted the bot to then brainstorm with us.
“AI lives on the internet, and you and I live in the real world — a distinction we shouldn’t forget,” said Vinay Menon, who leads the global artificial intelligence practice for recruiting firm Korn Ferry. “AI is meant to support decision-making, not take it over completely.”
Will AI coaches be employable in the future?
AI won’t be able to give you what a human coach can as they will lack empathy, human emotional cues and knowledge of your personal experiences, experts and software makers agree.
“A mentor has awareness of your career trajectory and experience that [AI] wouldn’t,” Rahman said.
You should start with basic AI literacy and skepticism, and familiarize yourself with data privacy policies before asking for advice.
Software makers and AI experts agree that the technology is only expected to get better. OpenAI has already introduced tech that can identify some visual cues. That said, AI treats decision-making like a science versus by using insight. If you use it, don’t forget to rely on your own human intelligence.
“The human needs to come first and the AI second,” Menon said.
