I think it is the other way around. In the config file you first add more specific policy, and then the more general one. Because the first policy that the request matches will apply and next rules will not apply. Please refer to the docs, everything is there. There is also a specific section for rule matching.

https://www.authelia.com/configuration/security/access-control/