There is no equivalent command for Advanced Mode button.
The advanced mode allows you to use the E5071C standard functions (Softkeys in the right side of screen).
To use standard functions, you need to send the E5071C standard command to the E5071C address (e.g. GPIB0::17:INSTR).