Friday, October 7, 2011

Coherence write behind, finding not-yet-stored entries


Write behind strategy may be very useful in certain cases. Using Coherence you can use this pattern in very smart way - synchronously replicate your data in memory over several nodes, then asynchronously write to slow external storage. This way you will not lose any data in case of single server outage. One unpleasant thing is that Coherence does not retain sequence of updates in certain cases, but wait,  this is a distributed system after all :)

But there is a catch. Let's assume you are receiving message via JMS, put data to Coherence grid, start processing etc. But you do not want to acknowledge message until it is written to persistent storage.
While Coherence let your data survive single node failure, there could be network failure, logical bug in your system or outage of persistent storage your are writing to. You want to be consistent, you can process message, but you do not want to confirm its delivery (delete it from one persistent storage) until it is not written into another persistent storage.

So, is it possible to find which entries in cache are not persisted yet?
Yes, it is. Internally Coherence marks not-yet-stored entries with special flag. This flag is usually invisible for application, but we can access it using some low level Coherence API.

Below is StoreFlagExtractor returning FALSE for vulnerable entries.
public class StoreFlagExtractor extends AbstractExtractor implements PortableObject {

    private static final long serialVersionUID = 20010915L;

    public StoreFlagExtractor() {
    }
   
    @Override
    public int compare(Object object1, Object object2) {
        // make no sense
        throw new UnsupportedOperationException();
    }

    @Override
    public int compareEntries(com.tangosol.util.QueryMap.Entry entry1, com.tangosol.util.QueryMap.Entry entry2) {
        // make no sense
        throw new UnsupportedOperationException();
    }

    @Override
    public Object extract(Object object) {
        // decorator can be extracted only from binary entry
        throw new UnsupportedOperationException();
    }

    @Override
    @SuppressWarnings("rawtypes")
    public Object extractFromEntry(java.util.Map.Entry entry) {
        BinaryEntry binEntry = (BinaryEntry) entry;
        Binary binValue = binEntry.getBinaryValue();
        return extractInternal(binValue, binEntry);
    }

    @Override
    public Object extractOriginalFromEntry(com.tangosol.util.MapTrigger.Entry entry) {
        BinaryEntry binEntry = (BinaryEntry) entry;
        Binary binValue = binEntry.getOriginalBinaryValue();
        return extractInternal(binValue, binEntry);
    }
   
    private Object extractInternal(Binary binValue, BinaryEntry entry) {
        if (ExternalizableHelper.isDecorated(binValue)) {
            Binary store = ExternalizableHelper.getDecoration(binValue, ExternalizableHelper.DECO_STORE);
            if (store != null) {
                Object st = ExternalizableHelper.fromBinary(store, entry.getSerializer());
                return st;
            }
        }
        return Boolean.TRUE;
    }

    @Override
    public void readExternal(PofReader paramPofReader) throws IOException {
        // do nothing
    }

    @Override
    public void writeExternal(PofWriter paramPofWriter) throws IOException {
        // do nothing
    }
}

Using this extractor you can easily query unstored entries, set listeners and even build indexes and CQ for such entries.
UPDATE: Changes of STORE decoration are not triggering cache events (only backing map events) and index updates. Nor CQ nor indexes nor listener would not work.

2 comments:

  1. Hi Alex, great article!
    My situation is a different. I have a Balance Cache, and decreases to the balance are very often and minor important but increases are rare and very important.

    We want to provide very high throughput for decreases and avoid the risk of loss of increases at the same time.

    Could you give us some hints? I'm wondering is it possible to perform Write-Through for some particular "put()" actions and Write-Behind for the rest?

    ReplyDelete
    Replies
    1. It is not impossible. But I would not advise you to go this way. Main problem - if you would segregate credits and debits you may loose order of operation.
      My advise, would be tune write behind for minimal latency. You could set very low write-delay and write-batch-factor=1. You would have very short delay. Now I can verify store status for your credit transactions.
      In theory, it would be possible to subclass ReadWriteBackingMap to have hybrid write-through / write-behind strategy, but it wouldn't be easy.

      Delete