I did some googling and followed the advice on the Epson website, which essentially says go to Control Panel > Hardware > Devices and Printers and the right click on the printer which brings up a menu next to the cursor.
I was just trying to check the ink levels in the printer and couldn't figure out how to. My mother-in-law has just bought a new laptop that has Windows 8.1 and it is connected to her printer, an Epson stylus sx125.