If Button Pay is set to "Always" enabled, only your default card is exposed. You should call that credit card provider and cancel that card. You don't have to cancel your other (non-default cards). Your data is securely encrypted and stored on our servers so there's no risk for the other cards within your account.
If Button Pay is not set to either to "Never" or set to a timer, then your LoopPay device will timeout and be unusable. You should still contact your card issuer and cancel your default card if your timeout has not expired.
You should however, also change your password and PIN, just as an additional security step.