When I took it junior year, I don't remember the math/physics section being anything particularly difficult. I don't remember any specific derivative/integral calculations nor any specific physics calculations; as far as I can remember everything concerning physics and calculus was all conceptual on my particular version. I could be wrong, but that's just the best I can recall. Really, the ASTB just seemed like a Navy version of the SAT to me.
To the OP; I would seriously recommend getting a test prep book from your advisor, especially one that covers the spacial orientation and study the spacial orientation and instrument sections like crazy. The book I used is called "Military Flight Aptitude Tests" by Thomson*Arco.
You are correct, but due to competition and only being able to take the test 3 times, I highly recommended that they had some math and physics under their belts before taking the test.
-ea6bflyr