In the advanced section of Excel Options, try checking the Show All Windows In The Task bar

Show All Windows In Taskbar.jpg