The best way to tell is to look at who publishes it. If it's published by a university press or by several other academic presses (e.g., Blackwell, Routledge, Palgrave, Ashgate) it is scholarly. There are other presses that publish scholarly books as well, depending on the discipline. Another way to decide is to look at the book's intended audience and purpose.