Uncle Sam has a reminder for some people who took advantage of the first-time homebuyer tax credit three years ago: He wants his money back.

Americans who bought homes in 2008 and took the government tax credit will be required to start repaying the credit beginning with their 2010 tax return, according to the Internal Revenue Service.

In an odd twist, those who took advantage of a nearly identical tax credit in 2009 or 2010 will not be required to pay it back, MSNBC reported. Under the terms of the 2008 tax credit, the credit must be paid back over a 15-year period, beginning with this year’s return.

That means anyone who took the maximum $7,500 credit will have to add $500 to their annual income tax liability for 15 years. For those who sell their homes before the 15 years are up, the entire tax credit bill will be due the year the house is sold.

The IRS describes the 2008 program as “like an interest-free loan.” This may come as a shock to some people, who may have forgotten the terms of the so-called credit, which was really more of a tax deferral. The IRS said it is sending reminders.

“There will definitely be people that are going to be surprised by it,” said Sean Dowling, vice president of The Dowling Group in Stamford, Conn., and a certified financial planner.

For others, it paid to be late. If you bought a home in 2009 or 2010 and took the same tax credit, you don’t have to pay it back, as long as you stay in your new home for at least three years. That’s because the government changed the rules regarding the tax credit after the first year, allowing people to take the credit without any requirement that they pay it back.