The MUA (Mail User Agent, RFC term for mail client) will add most headers that you can see in email clients, in particular From:
and To:
. (You can sometimes see this in the To:
header when someone has "funny" nicknames for someone in their address book. Or when "From:" and SMTP envelope from (as in the MAIL FROM:
command) disagree, even though may servers will reject such messages, as it's often a spam indicator.) The MUA will set the Reply-To:
header as well, especially if the MUA is really a customer management system. The MTA (Mail Transfer Agent, actual SMTP server) may include bounce information (VERP).
The MUA then submits the message to the MTA. Every program (mail server, firewall, etc) that touches the email can add headers. For example, the MTA would normally do the DKIM signing (and add the relevant headers). It must prepend Received:
header (i.e., put it at the top) and it must not mess with other Received:
headers. Still some programs like firewalls may mess with headers.
You can see multiple DKIM signatures (see DKIM selectors), for example, when an email gets resent in the context of a mailing list. You could see a DKIM signature from the original sender to the list, then more headers including an additional DKIM header when the mailing list server sends its to the final recipients.
As for X-...
headers: these are non-standard headers (hence the X-
prefix). Everything is off here. Some sending MTAs insert them for tracking purposes (e.g., to catch spammers among their customer), some mail receivers put their spam assessments in a special x-header. Even the MUA may put some x-headers in the emails, for example, Claws Mail puts in a header for the account the mail came from, when it downloaded it, and so on. You may or may not trust these, and in fact, MUAs may have settings to indicate which x-headers they should trust.
To that regards, even the Authentication results header and such can come form anywhere along the road. Even the initial sender could add one, like some anti virus programs do, to indicate that they scanned the outgoing mail. Again, it's up to the receiver to decide whether to trust these headers. Clearly you want the mail server closest to you to do DKIM checks and anything related to authentication, as you can hopefully trust that server's verdict.
So, does order indicate which mailer inserted a header? Yes, but... mail servers may or may not conform to all parts of the RFCs, and many mail servers follow Postel's law and be somewhat lenient in what they accept.