bug: missing messages while doing filter subscription #1211
Description
See discussion at: status-im/status-mobile#21120
status-im/status-go#5440 introduced the ticker.
Before the issue was probably already present (registering a filter was probably asynchronous, although I think on v1 it was sync), but most likely was never picked up as the window was smaller.
Potential solutions:
we mitigate but reducing the timing, but the window would still be present
we delay the sending until the filter is registered, that would add a concept of dependency to sending a message (i.e sending message X depends on having registered message Y)
we refetch from store node once a new filter is subscribed, for the period between scheduling the filter and once the filter has been actually registered, with some padding of course to accomoddate the timing. (there's still a window where the message would not be on the store node, nor the message has been pushed by the filter though).
Either ways, this looks like a fairly critical message reliability issue, lowering the batch ticker (it's currently I believe 5 seconds, which is quite high) is probably the quickest to lower the possibility of that happening.
As a way to not block release, I'd say that we can reduce the ticker length to a length which fast enough to make this issue potentially not happen (maybe something like 500ms or less?).
Then we can try work out a proper solution. @chaitanyaprem and I had talked before about this possible window on which messages would not be received due to the ticker and doing the storenode request could be a good solution indeed, taking into account that there might be a delay for the message to arrive to the storenode.
Nwaku storenodes will only accept messages with a timestamp +-20s its current time, so based on this, when we register a filter and the ticker is triggered, we asynchronously wait 20s, and then send a store request. What do you think,