To alter the Exchange owa policies you can access them Using the Office365 administration site and navigate to the Exchange section. In the default policy editor are only limited options available.
In case you would like f.e. disable the outlook wep app light (which is not available in the web editor), you have to use the almighty PowerShell console.
- Connect to you ExchangeOnline server
- Execute this snippet:
Get-OwaMailboxPolicy -Identity "OwaMailboxPolicy-Default" | Set-OwaMailboxPolicy -OWALightEnabled $false