Kevin Costner is looking back at how the rumors about his Yellowstone exit were handled and feels “disappointed.”

The actor who portrayed patriarch John Dutton in the Taylor Sheridan-created series for Paramount Network left the show due to schedule conflicts with his four-part Western Horizon: An American Saga.

“I loved the show. I liked the people on the show. I liked what it was about. I love that world,” he told People.

Costner said that when they initially pitched the series to him, “it was one season and a long movie,” but they ultimately extended it, to which he “stepped up,” saying, “‘I’ll do it for three seasons, and I ended up doing it for five.”

When the show took an extended hiatus, Costner felt he could’ve used that time more productively, saying, “There was a moment where that show for me stopped for 14 months… That’s the fact. I could have done a lot of things in that time, but I wasn’t aware that that [hold-up] was going to happen.”

As rumors about his Yellowstone exit swirled, Costner opted to stay quiet.

“I read all the stories,” he said. “I was disappointed that nobody on their side… ever stepped up to defend what it was I actually did for them. There came a moment where I thought, ‘Wow, when is somebody going to say something about what I have done versus what I haven’t done?’”

Costner also said that he would be open to returning to finish up the Yellowstone saga, but it would have to be under certain conditions.

“I’ve always felt that…It might be an interesting moment to come back and finish the mythology of this modern-day family,” he said. “And if that happens, I would step into it if I agreed with how it was being done.”

He added, “In the very end, I couldn’t do any more for it than I had already done.”