A lot of GIs like to "take a look" and, in my experience, it's especially those GIs who don't have a lot of IBD expertise or those who are technicians rather than clinicians. I've happily walked away from such GIs.
Many of the professional associations that GIs belong to (ASGE among them) caution against or outright contraindicate endoscopy/ colonoscopy to "visualize disease activity." It's poor practice and doesn't improve patient outcomes. Post diagnosis and barring any major shift in treatment, scopes should be limited to cancer surveillance. Most guidelines recommend every 1-2 years after 8-10 years of disease activity and even then, the risk of cancer is low.
At some point, you have to put your foot down.