I used to do a lot of climbing. but long before my colostomy.Couldn't now if I wanted to. legs are bad. However I don"t see too much of problems. Keep the harness away from the appliance. Don't go skinnying into narrow gaps. Your waist line is biggerIt would be hell to be caught in a chimney half way up.
Like SAhaz says youy will find your limits. Maybe not as extreme as formerly but what you want to do you can do.