I've heard snippets of Berkeley's non-choral music here and there. It's generally very accessible and feels (to the ear) well written. A recent biography I read of Benjamin Britten mentioned Berkeley numerous times.
There's a terrific Naxos CD from St. John's College of the choral music. With the usual Naxos quality and budget pricing, it's an ideal introduction to Berkeley's music.
His anthem 'The Lord is my Shepherd' is done frequently in church, but truth to tell, I find it a bit cloying.