Knuckles doesn't necessarily need to be in every story is what I have to say to it. If he's a limited character, you don't change his character, you use or create characters that can fit the narrative you want to shape.
Let's analyze how Knuckles has been used:
-In Sonic 3, Knuckles is on home territory and knows shortcuts and can set traps, and has only one mindset which is to stop threats to the master emerald. This is his initial character, and all he needs to be is clever, powerful, and focused on his ultimate duty.
-In Sonic Adventure 1 and 2, he's still remaining on emerald protection duty. It feels somewhat shoehorned into Adventure 2 and his subplot suffers for it, but for Adventure 1 it's done well as the master emerald is integral to the overall plot.
Starting with Heroes, they gave up. He's just...there, with no real explanation why. He's given basically no attention whatsoever and it destroys his established character.
After that, they did use him! Where it made sense. Unleashed skipped him (though honestly, he could had surely had a place in the plot considering the chaos emerald restoration goal), the storybook series gave him a cartoony drop in replacement, and generations he basically cameo'd.
Honestly, I think he's doing fine just considering those games. Keep him up on his island until he has a good reason to come down. He can be "put on a bus" until he's fresh or needed for a good point in the story, not there as a cheerleader. I think the solution isn't necessarily to remove knuckle's purpose, but to expand it logically for flexibility. Ideas:
-Reveal the master emerald's integrity is reliant on the chaos emeralds and you can safely place him in whenever Eggman is involved anyways.
-Sonic is now a chosen one entity of chaos, Knuckles needs to go bail him out if he's in danger since he's part of his agenda too because of this.
-Knuckles discovers a reason to go exploring that is more important to him than protecting the emerald. He'd check up on it frequently, but you could give him his own solo outing that only crosses paths with Sonic as he discovers some sort of truth about himself, etc. It could be the master emerald itself that gives him the quest, honestly.
-The master emerald is irreparably destroyed. He's now on a megaquest to go and make a new one, and theres currently no chaos emeralds. That should give him a reason to go adventuring with Sonic for a game or two at least. He still has his endgoal, but it doesn't keep him tied down as much since you can logically keep him at it for longer, and it would be able to impact the story more since the master emerald being gone would be a massive change to Sonic.
The only major exceptions I can see to Knuckles either skipping an adventure or having no reason to be there are 06 and Forces. Which, well, I say we can safely ignore those and just write better stories in the future.