What would happen to Saruman and Grima Wormtongune if they were not killed in The Shire and wandered Middle Earth as powerless vagabonds

If Saruman and Grima Wormtongune had not been killed after being defeated in The Shire and became drifters and vagabonds in Middle Earth what would their fate be?

Would they wander to Dunland and ask for food lodging? Go to live in Laketown and try to recruit friendly allies or just head East to lands like Rhun, Umbar, Khand or Harad hoping to manipulate men that were allied of Sauron?

You cannot imagine Saruman going to the Grey Havens and requesting passage to Valinor, he would be afraid of judgement awaiting him, he could go to Minas Morgul or Mordor and try to find allies there, can't imagine Saruman finding Gandalf and requesting forgiveness from him.