Looking for games, but on a tighter budget? Don't know where to shop? Look no further! These are my personal resources for inexpensive gaming.
Just watch out! too many sources of cheap games can become a bad thing! ("It's only $2.50!" .. 20 times in a week....)

Good Old Games, my personal favorite resource for gaming on a budget. GOG has games everywhere from the early days of PC gaming, all the way up to just a couple years ago and their catalog is ever growing. Games here will cost you either $5.99 or $9.99 (and there's a few free games), and usually include some random extras. All games from GOG are also completely DRM free (so you can make a disc of installers, and have your games no matter what happens), have all been tested to work on XP and Vista, and most work on 7.
As if all of that wasn't good enough though, every weekend they put an assortment of games 30-50% off, and every Wednesday they have a single game for 60% off.

GOG is a great site if you're looking for a nostalgia trip of replaying games from your childhood, or if you missed some great games of the past and can deal with highly dated graphics.
GOG is a dangerous site if you are prone to impulse purchases. *eyes 'shelf' of 50 games from GOG*
GOG should be passed if you have no taste in games, or if you simply cannot play games that don't have pretty shiny graphics.

Another digital distribution source, my second favorite. Impulse is a lot more modern games, and gets a lot of slightly less known about games. If you're familiar with Steam, Impulse is basically Steam-light, it does have an external program that runs, but it's mostly just the game catalog and your list of games, as opposed to a full-on gaming network. Good or bad? That's up to you to decide, I personally prefer Impulse over Steam.
Impulse has games ranging everywhere from under $10, to brand new games selling for $60, so they're not always cheap. What makes them a really good source for budget gaming though, is their Daily Deals (every day there's a single game 20-80% off) and Weekend Impulse Buys (an assortment of games, some newer, some older, usually 25-50% off). Plus, there's the occasional longer running sale. Impulse also has some game bundles, which can be good deals, great if they end up on a sale.

Impulse is a great site if you're looking for a bit newer games, maybe games you've never heard of, and are willing to spend a bit more.
Impulse is very dangerous if you're prone to "It's on sale!" because there's a game on sale every day!
Impulse should be passed if you.. uhm.. wow I can't think of a reason to pass up Impulse.

I'm not as big of a fan of Steam personally. It's a great system, basically like XBox Live or PSN for your PC, but it's not my cup of tea. Because of how Steam works, Steam is DRM and can also drain the system resources on slower computers. Occasionally, you might also have problems playing your games if you lack internet at that moment. These are the reasons Steam doesn't rank higher on my list. That being said, they do have a good catalog of games, ranging from brand new to old gems. And there's also a lot of indy games, some of which are exclusive to Steam (and usually inexpensive).
When it comes to gaming on a budget, Steam has all the same perks as Impulse: daily sales, weekend sales, random sales and game bundles. Add to that though, their assortment of indy games, and a fairly large selection of demos too, and you have a great resource.

Steam is a great system if you are looking for indy games and/or newer games, or if you really like the idea of having a gaming network with your single player games (if you like that sort of thing, you can also tether non-Steam games to Steam allowing you to hang with your Steam buddies when playing a game you bought from say.. GOG, without leaving your game).
Steam is a dangerous system if you're prone to "But.. but.. it's on sale.."
Steam should be passed if you have a really old computer that barely runs most games as is, or a very unreliable internet connection.

Amazon is a good source for gaming on a budget for 2 reasons. Primarily is it's still possible to get hard copies of a lot of games you'll never find in stores, and for fairly inexpensive (like I got Freedom Force vs. the 3rd Reich from Amazon, before Steam or GOG had it, for like $5). As well, Amazon has recently gotten in to the digital distribution market, their selection isn't as good as any of the above, nor do they have sales as often, but sometimes you can find great deals.

Amazon is a great site if you want hard copies of your games, or want to shop for your game fix at the same time as you shop for your mom's birthday present.
Amazon is a dangerous site if you're the type to think "If I spend $25, I can get free shipping." despite it being a $5 purchase.
Amazon should be passed if you don't want to support an evil giant corporation of doom, that is trying to take over the world. (Though Google is doing better at that right now.)

While not a sight that sells games GOG user Motoki pointed me to this site:
This site lists the deals on games from X company, giving you one place to look for your sale prices from Amazon, Impulse, Etc. Think a games a deal, check here to see if it's better somewhere else!

So that's the list of where I buy my games from.. at the moment, more may be added later. In the mean time, here's a bonus link for you:

Armor Games
Can't get more budget-friendly than free flash games in your browser! I've gone through a lot of flash game sites, and so far Armor Games seems to be the best, like all flash game sites it's got a LOT of crap, but if you're willing to sort through it, you will find some decent games.