I prefer only to add nonfat milk to my coffee.

I prefer to add only nonfat milk to my coffee.

In the second sentence, the limiting modifier only is placed in front of the words it modifies: nonfat milk.