Your hunch is exactly right. Gravity is the only important force affecting Earth's motion, but scientists do not know if changes in the Earth's orbit is what causes Ice Ages.

Earth's orbit around the Sun is affected by small tugs from the other planets, particularly Venus and Jupiter. Earth's own spin, though not its motion around the Sun, is affected by tugs from our Moon, causing its spin axis to wander around over time. These tugs are produced by, you guessed it -- gravity. Newton first observed that all things in the Universe seemed to attract all other things based on their masses and their distances -- most of the time these effects are very tiny, because distances are great or masses are small, but even small pulls over time can move mountains or planets.

Combinations of these changes in orbit and spin are thought to be significant factors that may play important parts in the development of ice ages, but there is a great deal that we still don't understand about this. Perhaps if you study this field you may play a part in increasing our knowledge about this area in which we have some significant answers but many questions remaining.