I have a main application class, which contains a logger, plus some general app configurations, etc.
Now I will display a lot of GUI windows and so on (that will use the logger and configs), and I don't want to pass the logger and configurations to every single constructor.
I have seen some variants, like declaring the main class extern everywhere, but that doesn't feel very object oriented. What is the "standard" C++ way to make elements in the main class accessible to all (or most) other classes?
Use the singleton design pattern.
Basically you return a static instance of an object and use that for all of your work.
Please see this link about how to use a singleton and also this stackoverflow link about when you should not use it
Warning: The singleton pattern involves promoting global state. Global state is bad for many reasons.
For example: unit testing.
It is not so bad idea to pass the logger and config to all the constructors if your logger and config is abstract enough.
Singleton can be a problem in the future. But it seams like a right choice in the project begin. Your choice. If your project is small enough - go with singleton. If not - dependency injection.