Pipe filter for DC core

Can I use RPC on deltachat-rpc-server (of DC desktop) to get full email text from core (preferably already decrypted, but with headers), or just headers, and maybe conditionally delete the message before it gets to GUI?

My goal is to get flexible message filtering, like in ‘normal’ mail clients.

Update: original topic title was ‘Message filtering with RPC script’.

No, the messages are written into the database without any headers even before you get an event saying there is a new message. Generally raw messages with headers are not written into the database. Full payload is written for large HTML messages, but it is not exposed and cannot be relied on.

So, it’s turns out to be a feature request in the wrong subforum…

Can core pipe full decrypted email through stdin/stdout of external filter process, like procmail/courier-maildrop/renattach/etc. ?

I would like to add a few details here.

  1. Such stdin/stdout piping (filtering) is usual practice in mail handling, supported by many mail servers. It used not only for filtering incoming mail (with ready-mad and custom anti-spam filters and antivirus software), but also for performing the whole bunch of magic tricks, like starting print jobs, converting documents to different formats, signing outgoing corporate mail, etc.

  2. The primary goal is to give power users easy and powerful tool to handle DC messages in any ways they want, using any programming language, with minimal distraction of DC dev team.

  3. The primary target is desktop DC apps (and standalone core). Should be no problem to support it in Delta Touch and Arcane TUI. Android implementation will probably need some tinkering. IOS land is completely unknown to me.

  4. There are two parts of implementation:

  • add piping to the core
  • add some way to configure the script and it’s arguments (envvar is probably the most natural way, maybe not even needs modifications to UI)
  1. For the reference, here are the documents for such interface of two popular mail servers:
    Postfix manual - pipe(8)
    3. Exim filter files (‘pipe’ section)
    but there are many options that are not needed in this case.

There are additional uses.

This could serve as an API to allow other software on the system to be added to a DC group and get data from it (or send data to it). For instance:
1.Mangadownloader bot and Peruse, a manga reader (I have not yet used either, but I assume they do what it says on the tin)
2. a (currently hypothetical) bookbot and an Epub reader.
3. Calendar WebXDC apps and an ical calendar program

This would not necessarily mean modifying the software; a rule like “any Epub file sent by Bookbot should be saved in this folder” would in many cases be quite adequate. And a lot of software can send mails anyway. Your system can mail you, why not mail stuff to it?

This should probably be turned off by default, and chats where messages are leaving the client flagged.

There list of additional uses is actually endless.

Comparing to RPC interface, pipe filter is much easier to learn and program, and more powerfull.
Currently even power user needs to study core RPC, to probably find it can’t help in particular case (see the start of the topic).

Also RPC is domain-specific, only useful for DC.
As of piped full-text mail - it’s the most generic interface I can imagine. And learning how it works (which takes much less time) will help to deal not only with DC, but with classic mail too.
There are plenty of articles on it, targeted on classic mail admins and power users.

Any chat messages are leaving the client.
WDYM?

Ok, exampe: DC client on home router checks for messages and pipe them to
grep ^magnet: | xargs -n1 transmission-remote -a
When it will receive a new message with Torrent magnet link(s), it will start the torrent download.

Do you really want me to create the bot for it? :-/

unless you really need headers you could make one generic bot that does what you propose: send all incoming messages to a stdio pipe.

And if you need the headers then you need to fork core as this will be a somewhat bigger task. Maybe you find an elegant solution that has very minimal maintenance cost, which you could then propose in an pr or just publish as a standalone tool, it wouldn’t change too much so I think it could be a soft-fork that is easy to maintain.

And now I’m imagining a contact writing “I tried retrieving it with a magnet: it didn’t work. Do I really need to dismantle the drain? If so I need to borrow your plumbing box.” :slight_smile:

Sorry, that was very poorly phrased. Chats where some of the messages are split off to an end destination other than the chat client. They should be flagged to avoid covert use. I don’t want incoming e-mails messing with my system except in very well-defined ways, especially if they come from a bot I don’t control.

A separate client for machine-readable messages seems like a sensible idea.

… without conversation established via invite link ?
(BTW note ^ - so ‘magnet:’ should be at the start of the line.)
(Sure, additional checks could be implemented.)

I still do not understand.
In your DC client only you can setup (proposed) filtering. It can’t be forced by anyone else.

In general case it’s not ‘machine-readable only’. Human readers will also benefit.
Another use case: all incoming messages from specified correspondent (which uses foreign language) are going through local language translation software, like Apertium.org , and you can read translated version appended to the original message.

IRL, people are going to share filtering software, and might not even always have custody of their phone. I don’t know how this would integrate with the various platform-specific tools for locking the app (none of which run on mobile Linux).

I’m not saying this is a bad idea, but it does have security risks and would need some fierce input-sanitizing, and good manual oversight capabilities.

Translation is a neat idea. Personally I would find it more useful for sending mails to multilingual mailing lists, where each mail conventionally contains several language versions of the same text. Typing a text over again in multiple languages without skipping or duplicating languages or content is tedious. Easier to rewrite machine translations. That could be a WebXDC app, actually.

Sure, sanitizing 100% needed in case of ‘usual’ accounts (not ‘single-purpose’ ones).
But I think it’s known problem, and there are known solutions.
(Actually, at first I begin to write much longer sed pattern for that torrent example, but dropped it for brevity.)

In the same (great) Postfix, as far as I can remember, there is a similar interface but using network sockets, not pipes. It may be more universal, but will take significantly more effort to implement.
And I don’t know will it help or not on platforms like IOS.