Remove Custom Visuals from Report

I've added some custom visuals to my report but then decided they weren't what I wanted, so I removed them from the Report. However I still get the warning that the report is using custom visuals, since I imported the visualization file. The program should either:

1) Allow me to remove custom visualizations from the palette so that I don't get this warning if I'm not actively using the visualizations in the report

OR

2) Only display the warning if one of the custom visualizations is actually utilized in the report (not just if it's been imported to the visualization palette)