UIStoryboardPopoverSegue在按钮触摸上打开多个窗口

I'm using a UIStoryboardPopoverSegue to present a popover for an iOS 5 iPad app. The Segue works great, but it seems like the toolbar that contains the button is a passthrough view for the popover controller so if you keep pressing the button, more popovers appear. As I'm not creating and keeping track of the to present