Did you mean to say yes or no?

Participial phrase

Gerund phrase

Infinitive phrase

Prepositional phrase

To say yes or no is an infinitive phrase.

For more help. see section 48b.