I came across the following slide of a presentation at the Faculty of Philosophy lecture by Robin Dunbar, Director of the Institute of Cognitive and Evolutionary Anthropology at the University of Oxford.
Using 19th century 'utopian cult data' (whatever that is), he determined the optimum group sizes for both religious and secular communities. Not surprisingly, for religious communities the number is about 150. The group size that is considered to be that throughout the majority of human evolution. But for secular communities, that number drops to about 50. It seems that a supernatural component to a group (whether it's true or not) provides increased social adhesion.
What we can deduce from that is to not worry so much when you see attendance at your secular events in onesie-twosie numbers while people are pouring into churches by the dozens. It's natural. Secular organizations should be more numerous, but smaller in size. From a membership of one, (like me), to around 50.