Did you mean to say yes or no?

Participial phrase

Gerund phrase

Infinitive phrase

Prepositional phrase

To say yes or no is an infinitive phrase.