STARTTLS for SMTP
STARTTLS is an SMTP extension in RFC 3207. After the client sends EHLO, the server may advertise STARTTLS, and both sides upgrade the connection to TLS before MAIL and RCPT.
RFC 8461 MTA-STS is a different control: a published policy that tells sending systems to require TLS to the recipient's MX hosts. STARTTLS is the upgrade command on the wire; MTA-STS is the DNS plus HTTPS policy that mandates it.
Opportunistic TLS
Without MTA-STS or DANE, many paths use opportunistic TLS: the client tries STARTTLS if advertised, but may fall back to plaintext if negotiation fails. RFC 3207 defines the extension; regional policy and receiver requirements decide whether fallback is acceptable.
Google's sender guidelines state that SMTP to Gmail must use TLS. The 530 5.7.0 row on Google's SMTP error list names STARTTLS when a sender attempts plaintext delivery.
After the upgrade
Once TLS is negotiated, the SMTP dialogue continues with MAIL and RCPT on the encrypted channel. Certificate validation is between the SMTP peers unless DANE publishes TLSA records for the MX host.
Google's sender guidelines require TLS for mail to Gmail. The 530 5.7.0 row on Google's SMTP error list rejects sessions that do not upgrade when STARTTLS is required.