About 20 percent of people said they had been frustrated with a financial institution. Consumers Union, the advocacy arm of Consumer Reports, is asking the Consumer Financial Protection Bureau to support new regulations that would make it easier for customers to move their money to a different bank.

Bank of America Corp. announced last September that it would start charging a $5 fee to customers each month for using a debit card. Bank of America later got rid of the fee due to public outcry, according to Reuters. 

Although many have considered moving their money, the majority of people have not.

“Many consumers don’t follow through because moving your money takes a lot of time and money and some bank policies make it harder than it should be,” said Suzanna Martindale, a staff attorney for Consumers Union. “We need to make it easier for consumers to switch banks so they have a real choice when it comes to where to keep their money.”