After 5 years of trial and error, I finally figured out the key to memoir. It sounds obvious in hindsight but: don't compare yourself to experts.

I was doing it completely wrong before and getting frustrated. Once I made this change, everything clicked.

Hope this helps someone else avoid the same mistake I made.