We are using Summary Views that filter down to the team ( Internal) and the work product Type (Article, Web, etc).

This results in a summary view where the label for the work product type is bringing in work tagged with other team labels. In the Label View there is not a way to filter out the other team labels. It only allows you to filter by the label selected on the Summary View configuration page.