From goffi at goffi.org  Tue May 24 22:26:05 2016
From: goffi at goffi.org (Goffi)
Date: Tue, 24 May 2016 22:26:05 +0200
Subject: =?utf-8?b?W1PDoFQgZGV2XQ==?= [en] messages refactoring
Message-ID: <3537397.2hQFBlS6HE@tazar2>

Hi there,

I have started a refactoring of messages handling, it's not finished yet but 
the basis is functionnal.

Message handling was here since the beginning of SàT and relatively basic, it 
was used in a "send/receive and forget" way, meaning that once a message was 
received or sent, we couldn't access it anymore. Also it was not handling all 
the elements found in core XMPP (the RFC), and badly handling  duplicate 
messages (which happen often with backlogs in MUC for instance) among other 
issues.

So here are the novelties:

- each message has now an unique id, internal to SàT. It can be used to modify 
a message after receiving it. That was a missing feature blocking us for the 
implementation of many exciting stuff like MAM (message history on server), 
delivery receipt, last message correction etc. You can expect these features 
for 0.7 release

- duplicate messages are checked and ignored in backend.

- message body and message subject now handle languages. They are received in 
a dict where the key is the language and the value is the body/subject in the 
given language. empty string is used when language is unknown. This a native 
XMPP feature that I was willing to implement for years, I think the ability to 
flag a message language is a killer feature which is largely underexploited in 
current XMPP clients. I'm looking forward to implement it in frontends
To send a message from frontend a dict is now needed too, with language as key 
or empty string if unknown.

- thread which were not handled at all are now added in extra dictionary if 
they are present, with thread and thread_parent keys. Threads are a way to 
follow a specific discussion in a conversation.

- delayed messages (e.g.: messages from backlog in MUC) are now saved in 
history, it was not the case before

- when using historyGet, message are always sorted in chronological order by 
backend, so frontend doesn't need to sort them again

- a timestamp is always sent with message. A received_timestamp can also be 
present in extra, that means that the message was delayed. In this case 
timestamp is the declared time of emission, and received_timestamp the time of 
reception.

Note that if a "received_timestamp" is present in a message, a frontend should 
check that the message is not already present in its buffer, specially just 
after a historyGet

- "info" messages can now be archived in backend, and will be mostly handled 
there (messages like join/leave in MUC).
This way we can keep track of what happened in history (a conversation can 
make more sense if we know than somebody joined or left). Also info messages 
will have subtype, so we can trigger a specific behaviour on some of them, 
or group/hide/separate repetitive info messages (like a user with bad 
connection which join/leave a lot of time in a short delay)

- I plan to use info messages in frontends to handle stuff like a last focus 
bar (i.e. displaying a line under the last displayed message, so it's easy to 
track new messages, it's a common feature in messaging softwares)

- in the backend we follow 2 new conventions: client is used to keep track of 
session, instead of profile, this avoid to do a host.getClient(profile) each 
time, and limit the need of C.PROF_KEY_NONE. Client is always the first 
parameter (while profile or profile_key was always the last before). You can get 
profile with client.profile. On the bridge and frontends this doesn't change 
anything
The second new convention is that we  use main category of a method (specially 
in bridge methods) first, then the verb. So getHistory and sendMessage are now 
named historyGet and messageSend, this way the methods are naturally grouped 
when sorted by name.
The code will be slowly refactored to follow these new conventions.

I think that's all. There may be other changes before the 0.7 release (and 
anyways as long as we don't have a 1.0, the API and bridge can change a lot).

I have done the main refactoring but lot of things are deactivated or broken, 
and the frontends are not taking profit of the novelties yet (actually Libervia 
is not updated, so it is broken at the moment).
There will be a refactoring in QuickFrontend too to handle uid, languages, 
etc.

This refactoring was a step necessary before implementing messages in Cagou.

The current code is available in the 0.7-dev branch (bookmark). Save your 
database before trying, as the schema change may result in data loss. Also 
once on refactoring_messages bookmark, you're database can't go back to 
previous schema, so it will not be usable anymore with 0.6.x version.

feedbacks welcome

Goffi