The strained relationship among members of the British Royal Family continues to be the topic of speculation. With many people trying to find explanations for their issues, several sources have reportedly revealed why Prince Harry and King Charles III are no longer speaking.

Prince Harry, King Charles III’s second son, officially resigned as senior member of the Royal Family in 2020 following his and his wife, Meghan Markle’s issues with the press and concerns over their safety. There are several claims about how much Harry “desperately” misses the U.K., and how much “heartache” is causing the King, but, at the same time, he vows not to bring his family there for safety concerns.

Back in the day, Harry used to be very close to his family, especially his brother, Prince William, and his wife, Princess Kate Middleton. However, after four years since Megxit, Harry’s relationship with his family is no better than it was in 2020.

Sources explain what happened between Prince Harry and King Charles

Early this year, Prince Harry rushed to the U.K. following King Charles’ cancer diagnosis and wanted to mend their relationship, but his safety concerns have deepened the issues with the Royal Family. Just a few days ago, Prince Harry gave an explosive interview in the ITV documentary Tabloids on Trial addressing his recent legal battle over hacking, and also noted that the lack of support from the family caused the “rift” between them.

After his visit to Charles in February, Harry returned to the U.K. in May to celebrate the10th anniversary of the Invictus Games, which he founded. Charles was notably missing from the ceremony, and the father and son didn’t connect during Harry’s stay. While that was allegedly because of the King’s “full schedule,” it looks like it might have something with Harry’s obsession with discussing his security. The monarch reportedly doesn’t take Harry’s calls nor respond to his letters. While Harry tried to find out more about his health, his efforts lead nowhere.

“He gets ‘unavailable right now,’” a friend of Harry’s told PEOPLE. “His calls go unanswered. He has tried to reach out about the King’s health, but those calls go unanswered too.”

“Harry is frightened and feels the only person who can do anything about it is his father,” a royal insider the same outlet. Another source explained, “Harry is determined to protect his own family at all costs.” A palace source, however, addressed the idea that the duke’s security lies solely in Charles’ hands is “wholly incorrect.”

Prince Harry and Meghan Markle’s exit from the Royal Family further deepened the problem. The monarch allegedly feared that history was repeating itself — when Princess Diana left royal life, she pulled the focus away from the rest of the Royal Family. “Charles’ fear is a repeat of the past,” the insider says. “When his wife divorced the institution, she took the headlines, overshadowed his work and became a global superstar. Seeing his son leave with his wife was not part of the plan.”

While Meghan Markle and Prince Harry continued to be a topic of some interest around the globe, Charles’ concerns of a Princess Diana 2.0 were unfounded, as Harry and Meghan seemingly haven’t managed to replicate the love and adoration “The People’s Princess” achieved.

Given the reduced security the Duke and Duchess of Sussex have after stepping down as senior Royals, Harry fears Meghan could face an acid or knife attack if she returned to the U.K., given all the negative press she receives there. “It only takes one lone actor with a knife or acid. That’s a real concern for me and why I won’t bring my wife back to this country.”