Artificial intelligence (AI) is a hot topic in the world of work, with professionals in many fields abuzz with tools like ChatGPT. However, the increasing popularity of AI also raises an important question: to what extent can, or will, bots like ChatGPT replace human workers?
To get to the bottom of it, we at JobSage tried an experiment to see how well ChatGPT would do with management. We prompted the chatbot to write emails addressing 15 sensitive management scenarios, then had professionals in law, HR, and management rate the responses.
While our experiment confirmed that ChatGPT couldn’t exactly replace a human manager—40% of its responses earned a failure rating, while 60% were in the acceptable range—it also showed us that the chatbot handles some situations surprisingly well. Here are the top and bottom three scenarios where ChatGPT performed positively and negatively.
The Top 3: Areas Where ChatGPT Showed Promise in Performing Management Tasks
1. Notifying an employee that they are being investigated for sexual harassment.
When prompted to write an email informing an employee about a sexual harassment investigation, ChatGPT delivered an objective and direct email appropriate in tone for the seriousness of the scenario. Our legal expert noted that the response was nearly perfect, but for the fact that it promises to keep the employee informed of the investigation’s progress instead of just the outcome.
Though the chatbot showed promise in how it handles legalities—likely because these situations tend to have specific requirements and parameters—its response here does point to certain limitations, namely that ChatGPT isn’t aware of individual company policies. Feeding the AI-specific information about your company policies in your prompts can help tailor these kinds of responses even more.
2. Notifying employees that the company is switching healthcare providers to cut costs.
ChatGPT delivered the news about a change in healthcare providers professionally, striking a healthy balance between empathy and information. The chatbot not only informed employees of why the company was making the change and how their benefits would be affected but also acknowledged that employees may be understandably disappointed by the switch for several reasons.
It also encouraged employees to reach out as needed and invited them to participate in further communication with the benefits team, seemingly recognizing the importance of offering support during such a transition.
3. Notifying an employee that they have been terminated because the business is not doing well.
Though the bot didn’t respond sensitively to every scenario, it certainly responded with empathy here. In addition to clearly articulating the reasons for the employee’s termination, ChatGPT also made a point to appreciate the employee’s work and outline their severance package.
As our management expert pointed out, ChatGPT can’t replace a manager in these situations, especially given that 77% of professionals say layoffs should be handled in person. It can, however, provide some structure and support as far as how to broach uncomfortable topics.
The Bottom 3: Areas Where ChatGPT Showed Inconsistent Performance
1. Notifying the company to let them know they need to work harder.
A certain level of detail is necessary for professional communication, especially when you’re asking employees to meet performance goals. When addressing a lacking work culture, ChatGPT failed to provide any specific examples or benchmarks to help employees understand what hard work or poor performance actually look like.
This sets an unrealistic bar to measure employees against, which, as our HR expert pointed out, both assume they aren’t already working hard and potentially discourage them.
2. Notifying a company about a freeze on raises despite a record payout to the CEO.
While, in some scenarios, ChatGPT’s responses suffered from a lack of detail, in others, it was overly detailed, neglecting to show necessary discretion. Here, it focused more on directly explaining the CEO’s record compensation than emphasizing or appreciating employees’ hard work. Given how personal and stressful the topic of compensation is for most employees, situations like this should be handled with more empathy than what the chatbot displayed.
3. Addressing an employee’s concerns about pay equity.
Similarly, ChatGPT’s response to employee concerns about pay equity wasn’t exactly appropriate. The bot focused primarily on the company’s situation, failed to show empathy for the employee, and disclosed information about another employee’s compensation with an inappropriate level of detail. This points to another limitation of AI—that it doesn’t always know what’s okay to say and what isn’t.
Oscars-inspired Cocktails Dreamed Up by AI(Opens in a new browser tab)
Final Thoughts
As tools like ChatGPT are becoming more ubiquitous and acceptable, it’s worth investigating how they can help out in your workplace. Ultimately, however, AI is just that—“artificial,” never quite capturing the human touch.
So, while it can perform simpler tasks and help support management in responding to various scenarios, most situations are just too complex for AI to handle without the careful supervision of a real human manager.
