Shards

For every complete collection from you home island you will gain 4 crystal shards
Blacktrees - (Halloween Event) - Arcane Crystallization 1 shard
Shiny gemstone 560 magic beensprouts (available from level 46) pays 6 shards
Deer tracker 1060 magic beansprouts (available from level 56) pays 11 shards
Lost puppy 1770 magic beensprouts (available from level 66) pays 16 shards
Tributes - gained from Achievements from these adventures - El Chupacabra & The People Of The Mountain 1 shard per tribute (if placed by a building that gives an area buff - this number can be increased)
Use provision house to convert 10 shards into a Crystal

Crystals

Daily login give crystals
Daily quest give crystals (2x if running prem time)
Completing pathfinder will give 15 crystals (level 70 and above)
Prospector workshop - make 'Prospect Granite' buff - sometimes this has crystals and sometime not - worth taking the chance (10 crystals min and 30 crystals max)
Use excelsior to spend crystals - short letter quests

Gems

A Strange Idol give 50 Gems
An Elaborate Cat Collar gives 50 Gems
The Annolholics Tour give 50 Gems
A Really Sharp Claw give 50 gems
Log in every day for a week and collect 100 Gems
Once every 7 days in Merchant - Epic Items 100 Gems for 600 star coins

Happy Farming.