The way to hook events in Lotus Notes is with the Notes C API's Extension Manager routines. The general strategy for hooking mail send operations is to set up a hook on EM_NSFNoteUpdateMailbox. This can be done on either the Notes client or the Domino Server. On the server side, if your processing is non-trivial, you run the risk of slowing down Domino if you try to do all of the work in your hook, so the strategy most people adopt is to have the hook code change the status of the message to "On Hold" and then signal a separate server task to take over the work. That way the hook can return and the separate server task can take whatever time is needed before resetting the status of the message so that normal routing will take over. You can find documentation here.
For server-side processing only, there are two options that will allow you to write your code in Java instead of C. One is an OpenNTF project called TriggerHappy that allows you to write Java Agents that are fired when some boilerplate hook code is triggered, and the other is the OpenNTF version of the Domino OSGi Tasklet Service (aka DOTS), which allows taskets to be fired in response to EM events. Note that there is an IBM-supported version of DOTS that is shipped in Domino 9 "Social Edition", but my understanding is that it does not include the EM event support.
Updated for additional info requested...
In order to add a flag to each message that can be read by the code that runs in your hook, you need to modify the mail template that is used in the users' mail files. You will need permission from the Domino system administrators to do this, and to be blunt: this is not something that Domino adminisrators like to do because it makes their job more difficult when they are upgrading Domino versions. There are many ways you could modify the template, but the point of all of them would be to add a new hidden field to messages which would serve as a flag for your hook code. If the field exists, your code would do the changes to the content that you need, otherwise it would leave the content alone. One possible way to do this would be by adding a shared action containing a formula to set your hidden field (that formula is just something like FIELD myFlagFieldName := "1";) and then you can add that shared action to the Memo form. Users would press the action button any time they want to trigger your code. The admin would need to maintain backup database containing your shared action and re-add it to the mail template after every upgrade and then re-add it to the new template version's Memo form. Of course there are other ways as well, some that are more friendly to users but harder for admins to deal with, and vice versa.
