I'll say what every geologist already knows: rock yards are the best! Rock yards are dealers of raw earth materials. You can save a lot of money with hobbies like gardening and terrariums if you skip the chain stores and go straight to the source. If you don't know about rock yards yet, let me explain them at an autistic level of detail. (Cue "A Whole New World," alright go)
The basics
I use the generic "rock yard" because they are literally big open yards with piles of different rock products, for example:

See? "Gravel lot" is also acceptable, as it clearly is a big open lot with piles of different gravel products. They can go by many names, so try a variety of search terms. Their biggest clients are landscapers, so business names might involve words such as landscape, gravel, sand, decorative rock, or all-encompassing terms like 'supply' and 'materials.'
What do rock yards have?
That's the beautiful thing, each one is different. It is worth exploring options around your area to see what is available.

I am able to get a variety of types of gravel, pumice, decomposed granite, lava sand and rock, flagstone, and screened soils from my local rock yards. My cactus and succulent soil mixes are made entirely from materials from my local rock yards; I also enjoy picking through the flagstone remnants for nice pieces to use in the garden, and as terrarium decor.
Some locations additionally offer a variety of composts, wood chips, firewood, used pallets, railroad ties (square timbers), weed-blocking cloth, tarps, and other landscaping supplies for a convenient one-stop shop.

Check the website
The business's website will have a list of materials, and with luck, pictures and price charts. Websites aren't necessarily super up-to-date: there might be additional varieties not listed, and some materials can be sold out. If it's a long drive it is worth calling in advance to check whether a material is in stock.
Why can't I find (specific material)?
Rock is heavy and expensive to transport, but most rock is also relatively inexpensive. This means the selection will be heavily based on what is regionally available, and you'll get the best prices for local materials. It is common for the business to be located directly at a source like a quarry or a river, trucking in additional materials from quarries they own and other sellers in the region.
Tile shops are okay too, I guess
When you start looking for rock yards, a lot of search results will be tile shops that have 'natural stone product.' It can take some research to find a shop with a good selection of natural materials though, as the tile market has trended toward "landlord remodel" styles in recent years.
Ignore big chain stores; you might find a deal on damaged product, but you still generally get an even better deal at your local tile specialty stores. Just ask them if they have scrap bins of broken tiles and old samples for sale cheap; exercise caution digging around broken edges which can be extremely sharp.
I've scavenged lovely pieces of inexpensive fossil-filled travertine, shale, slate, and marble in sizes from counter slabs to small tiles. A large piece works well as a sturdy table for outdoor plants, with pots layered using stacked tiles.
Volumes and pricing
A lot of rock yards offer self-fill: the option to fill up 5-gallon buckets yourself. This is great for small volumes, like ingredients for your potting mix. I also find it to be a good way to get inexpensive top dressing—I keep a bucketful next to my potting bench for convenience.
Self-fill versus delivery
Delivery is best if you want a large volume, i.e. several tons, or an entire truckload. Be warned that whatever you calculate probably isn't enough! You will most likely be underwhelmed when you see a one-ton pile; it's the equivalent of a cubic yard of material, or about forty 5-gallon buckets. (40 buckets still makes it sound relatively large, poured out it makes a small mound at best).
For medium-sized volumes, there is also the option to have them fill the bed of a pickup truck or a trailer.
Price
Prices will be listed on a per-ton or per-yard basis, with the best value when you buy larger volumes. The per-bucket rate is the highest price-per-unit but still inexpensive compared to box store prices; I pay somewhere around $3 to $10 per bucket depending on the material.

Larger decorative rocks like flagstone and boulders will typically be priced per pound instead. This means you pick out the rock(s) you want, weigh them, and then pay. The scale will look like a large metal pad, or some rock yards use a vehicle scale instead. In the latter case they'll ask you to drive onto their scale for a weigh-in when you arrive, and you'll weigh up again when you are done selecting rock.
The cheapass method
I don't have a pickup or trailer and so far this is the best approach I've come up with to purchase at bulk prices. I purchased one of the reusable heavy-duty bags they have available for the transport and storage of a cubic yard of material. I set that up at home, buy in bulk, and make several trips back and forth to shuttle the cubic-yard pile home in bucketfuls. Understandably, busy yards may not be able to accommodate this!
I fill my vehicle up to the weight limit each trip, keeping the material in buckets for a quick unload and turnaround once I'm home. Do the math yourself though; it's worth it for me because I have close access to rock yards with quality product here in Tucson.
How to use a rock yard
Here's how to act like you know what you're doing so you don't embarrass yourself in front of the professionals.
Visit in person prior to purchasing
Visit the rock yard to visually inspect their products. Materials typically look better in person compared to online images—gravel just isn't very photogenic!—so it is definitely worth visiting.
Two businesses may have the same material in name, but slightly different sourcing can lead to noticeable differences in quality, size, or color. There will be inconsistencies even when the material comes from the exact same location. Natural materials have natural fluctuations! It is common for the quality of a quarry to decline over a long period as the best material is mined out, or for the color to change—I haven't been able to exactly match the gravel color in my back yard when I need to fill patches.
At the yard

The main office at the yard will have price sheets and a display showing the palette of materials available. This palette can be indoors or outdoors, or sometimes both. Outdoor samples can get somewhat dusty or dirty, whereas samples indoors may have a sign indicating they have been washed to show their true color. This is in case you are trying to match a specific gravel.
When you are ready to buy, go to the office—for safety reasons you can't enter the main yard without checking in first. The main desk will (should) have a map showing where each material is since piles generally aren't labeled.
Don't forget to bring...

- Buckets you would not be the first person to forget the buckets at home, they usually have some for sale at the office.
-
Shovel although with a lot of materials it is just as easy to scoop into the side of a pile with the bucket.
- Protective gear closed-toed shoes, pants, gloves. It's easy to get scuffed up, I usually skin a knuckle or pinch a finger every visit.
-
Heat protection hat, gloves, water. There probably won't be shade cover in the summer so stay cool and watch out for rocks that are too hot to handle.
- Car liner makes it a lot easier to protect your vehicle and clean up spills. I use a car liner made for pets. A tarp works, but they tear easily under weight so go with a hardy one.
- Tarp If you are loading an open pickup truck or trailer bed, bring a tarp to cover the material. Yards often REQUIRE this, depending on the material, otherwise the blowing dust can create hazard for other drivers on the way home.
- Cash is nice in case you need help lifting. The yard-hands are happy to help and appreciate tips. Some business will also give you a deal when you pay cash, since it saves them on card fees.
- Recommended: N-95 or respirator designed for small particles, and safety goggles.
Rock yard safety
If you tell your local rock yard I sent you, for the love of god try to respect their rules. They may have safety rules posted on their website or in their office; familiarize yourself before your first visit.
Weight
Rock gets heavy fast—heavier than you might expect. Some materials like decomposed granite inexplicably have a density rivaling a star's core (I also might be enfeebled but, not to brag, I have been to the gym three times this year). Be easy on your back; if you need to, fill buckets part-way and top them up once they are in your vehicle. You can also fill two half-buckets instead of one full one.
It is important to check your vehicle's weight limit, because rock yards assume no liability nor responsibility for vehicle damage caused by overloading. The weight limit is typically on a sticker on the inside of the door frame of the car's front driver or passenger doors.
Maintain situational awareness
Read signs and take cautions seriously. Stay aware of and away from large vehicles and heavy machinery. Don't make employees have to watch out for you, and don't trust other customers to notice you once you are out of your vehicle. This is exactly why larger yards won't allow the general public do the self-fill method; the busier things get, the greater the risk of accidents!
Do NOT climb the rock piles
Have I climbed the piles? Yes. Have I injured myself? Also yes and I deserved it. The rock yard makes no guarantees about the stability of piles, so keep an eye out for pieces dislodging and tumbling down from the top as you dig. With a little momentum, a falling rock can easily break a bone! I was lucky to learn my lesson with a relatively lightweight lava rock that hit me in the ankle.
Be cautious for critters
Rock piles are habitat so you might have competition for those sweet, sweet rocks. Large rocks provide excellent basking and hiding spots for reptiles which, in Arizona, can include venomous ones. Not to mention black widow spiders! Finer gravels and sands... well have you ever seen a nature documentary where a creature wriggles under the sand and disappears in a matter of seconds?
All I'm saying is, it is risky to put your fingers places you can't see! I've gotten a few jump scares from lizards, so I try to be careful disturbing piles where wildlife might inadvertently get crushed.
Dust
You'll kick up plenty of dust on your own when you start digging, so keep car doors closed while you are working to help keep the interior of your vehicle clean. You should also be prepared for freak winds that might sandblast you and your vehicle—I've seen a dust devil more than once!—so this is a place for working vehicles, not appropriate for anything you expect to keep pristine.

I know I already mentioned masks but we are talking about ROCK dust which poses a serious health hazard with severe and repeated exposures. Pumice, for example, is used as an industrial abrasive and can likewise cause abrasive damage to your lungs, skin, and eyeballs. It's important to take precautions seriously because the health effects are both cumulative and irreversible, causing diseases like silicosis in advanced cases. Some US regions also have the soil-borne fungal infection Valley Fever to contend with, which can make people and pets very ill.
If you feel dusty or gritty after your visit to the rock yard, you really ought to upgrade to better protective gear. Some signs that you are getting high dust exposure include:
- Slight grittiness between teeth
- Eyes feel dry, lightly irritated (do NOT rub!)
- Fine layer of dust on your glasses or dark glasses (rinse off, do NOT rub!)
- Debris in tissue after blowing nose, gritty/sandy and discolored boogers, irritated sinuses
- Exposed skin easily chafing
In conclusion
Go get those rocks! Go on, go!