The CPU contains an "Executive" program that tells the PLC how to:

Execute the control Instructions from User's Programs
Communicate with other devices through other PLCs, Programming devices, I/O devices, etc.
Perform Housekeeping activities such as Diagnostics, etc

This program is stored in "nonvolatile" memory
Meaning that the program will not be lost if power is removed