As of now the app needs to running. Which is a pain. The Pokemon Go Plus accessory does the buzzing for this if you can get one.
Right now, you can use the power saving option, which allows you to turn your phone upside down and it dims the screen and turns off the main game screen and just shows a logo.