I don't want to get involved in a fight with your teacher because I don't truly know the context of your argument, but the first thing to do is to look up the exact definition of "physical" and "chemical" change in your textbook. According to current understanding, boiling the water is a physical change, putting the teabag in with the result of making tea may also be physical, since no compounds are actually changing their molecular structure. Nothing is turning from one chemical into a different chemical. The only thing happening is some of the chemicals in the tea bag are moving from their dry state in the tea leaves to dissolve (and perhaps be suspended to some extent) in the hot water. The water is not causing anything in the tea to change to a different chemical. It is simply leaching out some chemicals that are phyically in the tea leaves, and now they are physically in the water. But chemically, nothing has changed.
From Discovery.com's Homework Help: "Physical change is a change of matter from one form to another without the production of any new substances. All chemical compounds present before the physical change are present afterward. Because no change in chemical structure occurs, there is no change in solubility, taste, odor, or ability to react chemically. For example, when wood is turned into sawdust, the change is physical. But if the wood were burned, new substances, ash and gas, would result, so the change would be chemical. Another example of a physical change is the melting of ice to water."
It looks like your teacher won the argument.
making a small donation to science.ca.