Hooray for "junk" bins and used gear
For musicians/producers wanting old cheap gear in Japan, your options are sadly limited. The biggest auction site here is Yahoo! Auctions and due to the 10 minute clock reset at the end of acutions bargains are pretty darn rare. The idea is to prevent sniping by giving bidders a chance to increase their top bid before the clock runs out by resetting the clock to 10 minutes if a new high bid comes in at the last 10 minutes of an auction. The effect is that closing prices end up much higher because people end up bidding more than they thought their max was at first. The unintended effect is that the prices of used stuff in stores goes up to match auction closing bids. Also if you are new to auctions here, dont forget that you have to pay the bank transfer fee and shipping fees which can add up to 2,000 Yen onto your real costs.
In the US I used to buy used gear at pawnshops, garage sales and flea markets. Japan has pawnshops but they never really have music gear. There are used shops called "recycle shops" but music gear besides accoustic guitars or the occaisional DJ mixer is almost never to be found. Even when you do find something dont expect a bargain, there is a nation wide price guide published for these shops telling them the "correct" prices for things. There is a chain store which sells used stuff called "Hard Off" which often has instruments, FX, mixers, speakers, etc. but prices are again high. Forget flea markets for the most part, I've never seen a piece of music gear at one in the greater Tokyo area.
This basically leaves actual music gear shops. In Tokyo they are concentrated around the hill from Ochanomizu station. Some kind soul created a map here. Or theres some around the south exit of Shibuya station. Ishibashi mentioned above is a chain of music instruments and related stuff. You can check their used pricing here. AFAIK they have stores in many cities of Japan. I've been to their Kobe shop and it was pretty large by local standards. Dont expect great prices or much selection on used guear.
Generally do not expect many shop clerks to speak anything besides Japanese and in Tokyo anyways dont bother trying to haggle. In this part of Japan prices are almost always carved in stone.