I would keep the appt with the breast specialist. Personally, I wouldn't want to bet my life against a "suspected" benign report. I will say that, in order to report that, they are probably pretty sure, but mammograms are not "fool-proof." I had two palpable lumps, and my report showed no evidence of cancer. When my family doctor insisted on a biopsy, I thought he was being overly cautious, but I followed through at his insistence (he had me scheduled to see a surgeon the next day). On biopsy, my lumps were invasive cancer of two different types. They say that the younger you are and the more dense your breast tissue, the more likely you are to have a bad reading. Something like 30% of all breast cancers are missed on a mammogram. I didn't know that then, or I would have been more concerned. Now, I'm not saying I'm the norm, but...
If you already have the appt, I would keep it. If nothing else, you will have a second opinion from someone who specializes. That would certainly make me feel more comfortable with the report.
Good luck! And let us know how it goes.