Client side data portal cache (example)

[ossendorf-at-hoelscher]

@rockfordlhotka As discussed here our current implementation.
First all necessary code and afterwards some brief explanations.

The caching interface + criteria marker interface

internal interface IOurDataPortalCache
{
    /// <summary>
    /// Returns the previously cached value or gets it through the <paramref name="getFunc"/>.
    /// </summary>
    /// <param name="objectType">The type of the object.</param>
    /// <param name="criteria">The criteria.</param>
    /// <param name="getFunc">The factory method to get an instance of <paramref name="objectType"/>.</param>
    /// <returns>The data portal result.</returns>
    Task<DataPortalResult> GetOrCreate(Type objectType, object criteria, Func<Task<DataPortalResult>> getFunc);
}

/// <summary>
/// Marker interface to show the (criteria) object should be respected for caching.
/// </summary>
public interface IHasCacheableKey
{
    /// <summary>
    /// Get the cache key to identify this objects values.
    /// </summary>
    /// <returns>The cache key.</returns>
    int GetCacheKey();
}

The main interface IOurDataPortalCache only provides the single method to move every logic related to caching (e.g. locking, checking etc.) into the actual implementation to not leak any details into the decorator.
The IHasCacheableKey is used in criterias to differentiate between multiple result lists of one type. For example LookupROList has a discriminator which identifies disjunct sets of the type. So with the interface you can cache those different results. A simple example would be

[Serializable]
public class LookupROListCriteria : MobileObject, IHasCacheableKey 
{
     public string Discriminator { get; set; }
    
    /// <inheritdoc />
    public int GetCacheKey()
    {
        return HashCode.Combine(Discriminator);
    }
}

With this you can cache values for all discriminators without polluting/corrupting other values of this type. Drawback: You need more memory. Total would be Count of Discriminators + 1 (whole list).

The options to define which types should be cached

/// <summary>
/// Options of the data portal cache.
/// </summary>
public class OurDataPortalCacheOptions
{
    /// <summary>
    /// Gets or sets the types to cache.
    /// </summary>
    public HashSet<Type> TypesToCache
    {
        get;
    } = [];
}

Defines the types which should be cached when used during a fetch.

Our in-memory implementation with IMemoryCache

/// <summary>
/// In-Memory implementation of <see cref="IOurDataPortalCache"/>.
/// </summary>
internal partial class OurDataPortalCache : IOurDataPortalCache
{
    // Only necessary and used for cache invalidation but not part of this code.
    private readonly ConcurrentDictionary<CacheKey, byte> _knownKeys = [];
    private readonly OurDataPortalCacheOptions _options;
    private readonly IMemoryCache _memoryCache;
    private readonly ILogger<OurDataPortalCache> _logger;

    /// <summary>
    /// Initializes a new instance of <see cref="OurDataPortalCache"/>.
    /// </summary>
    /// <param name="options">The caching options.</param>
    public OurDataPortalCache(IOptions<OurDataPortalCacheOptions> options, IMemoryCache memoryCache, ILogger<OurDataPortalCache> logger)
    {
        _options = options.Value;
        _memoryCache = memoryCache;
        _logger = logger;
    }

    /// <inheritdoc />
    public async Task<DataPortalResult> GetOrCreate(Type objectType, object criteria, Func<Task<DataPortalResult>> getFunc)
    {
        // Type is not registered for caching or criteria is null don't cache anything so just pass through to the actual operation
        if (!_options.TypesToCache.Contains(objectType) || criteria is null)
        {
            return await getFunc();
        }

        int? criteriaKey;
        // EmptyCriteria is special cased because it's an FetchAsync() without any parameters which have to be cached too but need
        // a cache key which can be distinguished from marked types
        if (criteria is EmptyCriteria)
        {
            criteriaKey = null;
        } 
        // Used of the marker interface to get the cache key part
        else if (criteria is IHasCacheableKey cacheableKey)
        {
            criteriaKey = cacheableKey.GetCacheKey();
        }
        // Type is marked for caching but neither a parameterless FetchAsync() or a known criteria is used hence pass through
        else
        {
            return await getFunc();
        }

        // Create the unique cache key
        var cacheKey = new CacheKey(objectType, criteriaKey);

        // Let's do memory cache do it's magic and return the result (either cached or retrieved from server)
        return await _memoryCache.GetOrCreateAsync(cacheKey, _ =>
        {
            _knownKeys.AddOrUpdate(cacheKey, default(byte), (_, old) => old);
            LogCacheMiss(objectType);
            return getFunc();
        });
    }

    [LoggerMessage(Level = LogLevel.Debug, Message = "Cache miss. Value for {Type} is retrieved from server.")]
    private partial void LogCacheMiss(Type type);

    // Record for easy comparison features
    private record CacheKey(Type Type, int? Key);
}

The decorator to forward calls to our cache

/// <summary>
/// Adds caching capabilities to the <see cref="IDataPortalProxy"/>.
/// </summary>
internal partial class CachingDataPortalProxyDecorator : IDataPortalProxy
{
    private readonly IDataPortalProxy _decorated;
    private readonly ILogger<CachingDataPortalProxyDecorator> _logger;
    private readonly IOurDataPortalCache _cache;

    /// <summary>
    /// Initializes a new instance of <see cref="CachingDataPortalProxyDecorator"/>.
    /// </summary>
    /// <param name="decorated">The decorated <see cref="IDataPortalProxy"/>.</param>
    /// <param name="logger">The logger.</param>
    /// <param name="cache">The data portal cache.</param>
    public CachingDataPortalProxyDecorator(IDataPortalProxy decorated, ILogger<CachingDataPortalProxyDecorator> logger, IOurDataPortalCache cache)
    {
        _decorated = decorated;
        _logger = logger;
        _cache = cache;
    }

    /// <inheritdoc />
    public bool IsServerRemote => _decorated.IsServerRemote;

    /// <inheritdoc />
    public Task<DataPortalResult> Create(Type objectType, object criteria, DataPortalContext context, bool isSync)
    {
        return _decorated.Create(objectType, criteria, context, isSync);
    }

    /// <inheritdoc />
    public Task<DataPortalResult> Delete(Type objectType, object criteria, DataPortalContext context, bool isSync)
    {
        return _decorated.Delete(objectType, criteria, context, isSync);
    }

    /// <inheritdoc />
    public async Task<DataPortalResult> Fetch(Type objectType, object criteria, DataPortalContext context, bool isSync)
    {
        if (isSync)
        {
            LogSyncFetchExecuted(objectType, criteria?.GetType().ToString() ?? "<unknown>");
        }

        return await _cache.GetOrCreate(objectType, criteria, () => _decorated.Fetch(objectType, criteria, context, isSync));
    }

    /// <inheritdoc />
    public Task<DataPortalResult> Update(object obj, DataPortalContext context, bool isSync)
    {
        return _decorated.Update(obj, context, isSync);
    }

    [LoggerMessage(Level = LogLevel.Warning, Message = "A synchronous fetch for {Type} is requested with a critery of type {CriteriaType}.")]
    private partial void LogSyncFetchExecuted(Type type, string criteriaType);
}

Not much to do here. Route all calls (except Fetch) to the decorated instance. Send the Fetch call through our cache.
The sync warning is to find Fetch() issues (sync deadlocks in our case) and resolve them.

DI registration extension

/// <summary>
/// Adds a cache to <see cref="IDataPortal{T}"/> and <see cref="IChildDataPortal{T}"/>.
/// </summary>
/// <param name="services">The services.</param>
/// <param name="configure">The cache configuration.</param>
/// <returns>The services.</returns>
public static IServiceCollection AddDataPortalCache(this IServiceCollection services, Action<OurDataPortalCacheOptions> configure)
{
    services.AddMemoryCache();
    services.AddOptions<OurDataPortalCacheOptions>().Configure(configure);
    // The Decorate method comes from: https://github.com/khellang/Scrutor
    services.Decorate<IDataPortalProxy, CachingDataPortalProxyDecorator>();
    services.AddSingleton<IOurDataPortalCache, OurDataPortalCache>();
    return services;
}

Register all necessary services so the cache decorator can do it's work.

Registration in app

services.AddDataPortalCache(options =>
{
    options.TypesToCache.Add(typeof(ConfigROChild));
    options.TypesToCache.Add(typeof(LookupROList));
    options.TypesToCache.Add(typeof(LookupList));
});

Configure the app and which types should be cached.

[rockfordlhotka]

Thank you for sharing this.

I'd like to make sure @mtavares628 sees this thread as well, because he was instrumental in what's been added to CSLA 8 this far.

I think that this implementation and what we've added to CSLA 8 should be compatible, or very close?

Specifically, you have a way to determine whether a type is cacheable, and also a cache. And you have a decorator for the client-side data portal to check the cache as appropriate.

In the CSLA 8 change, the client-side data portal always invokes a check for a cacheable type, and if it is cacheable then it invokes the cache to get the result. For example, in the fetch operation:

csla/Source/Csla/DataPortalT.cs

Line 245 in dd16fda

if (Cache.IsCacheable(objectType, criteria, DataPortalOperations.Fetch))

The differences I see are that the current CSLA implementation doesn't provide access to the isSync value to the cache. Is that important? If so, why?

Also, the current CSLA implementation asks you basically to cache (or at least return) a DataPortalResult, not just the object being retrieved. I can't say that we though through the pros and cons of this in any depth. My instinct says that caching the result could be valuable, in that a failed result can be cached, or a success result. I'm not sure that's actually valuable (to cache a failed request).

[mtavares628]

I think either option will work for my needs, but option 1 probably provides more flexibility overall for people.

However, I just want to point out that your current implementation would allow for the DB calls to potentially get called more than once. the IMemoryCache.GetOrCreateAsync doesn't provide any locking, so it's possible that multiple thread calls to the delegate function that hits the database could be made:

        public static async Task<TItem?> GetOrCreateAsync<TItem>(this IMemoryCache cache, object key, Func<ICacheEntry, Task<TItem>> factory, MemoryCacheEntryOptions? createOptions)
        {
            if (!cache.TryGetValue(key, out object? result))
            {
                using ICacheEntry entry = cache.CreateEntry(key);

                if (createOptions != null)
                {
                    entry.SetOptions(createOptions);
                }

                result = await factory(entry).ConfigureAwait(false);
                entry.Value = result;
            }

            return (TItem?)result;
        }

This is a problem for my implementation because I have this caching within my business rules, and they end up getting called multiple times in very quick succession.

[rockfordlhotka]

Although this puts a burden on the cache implementer, we could consider making the cache and data portal call into something that (to CSLA) is "atomic". So in the data portal we would do this:

  if (Cache.IsCacheable())
  {
    result = Cache.GetResult(Type objectType, 
                                             object criteria, 
                                             DataPortalOperations operation, 
                                             async () => await proxy.Fetch(objectType, criteria, dpContext, isSync))
  }

Then IDataPortalCache would have two methods: IsCacheable and GetResult.

The cache implementation can do whatever kind of locking (or not) is desired and can invoke the data portal function if the item isn't in the cache.

Or even get rid of IsCacheable and always invoke the cache to get the result - leaving it up to the cache implementation to determine whether to just invoke the data portal or go through the cache.

[mtavares628]

I would say you can get rid of IsCacheable at that point. It was receiving the same 3 parameters as the GetResult method would anyway, so performing the check in a single function makes sense.

[StefanOssendorf]

@mtavares628 You are correct. This is currently not an issue but I'll fix that with another dictionary :D
ConcurrentDictionary<CacheKey, object> which will ensure I have exactly 1 lock object per key.
And locking against that key after checking the cache should do the trick.

@rockfordlhotka
With that you could implement a NullCache-Implementation which does not cache at all.
And if you want to provide something "built in". You could create a new Package Csla.Client.InMemoryDataPortalCache which implements an abstract class which does maybe the stuff I provided above with an internal IsCacheable() and GetCacheKey() method for the user to implement. That could be a simple InMemory-Cache implementation. And if the user want to go fancy he has the interface to implement.

[rockfordlhotka]

Added issue #3655

[ossendorf-at-hoelscher]

@mtavares628 I changed our implementation based on your comment. Now it locks based on the generated CacheKey

private async Task<DataPortalResult> GetOrCreate(CacheKey cacheKey, Func<Task<DataPortalResult>> getFunc)
{
    if (_memoryCache.TryGetValue(cacheKey, out DataPortalResult? value))
    {
        return value!;
    }

    var semaphore = _synchronizationLocks.AddOrUpdate(cacheKey, _ => new SemaphoreSlim(1), (_, x) => x);

    await semaphore.WaitAsync();
    try
    {
        return (await _memoryCache.GetOrCreateAsync(cacheKey, _ =>
        {
            LogCacheMiss(cacheKey.Type);
            return getFunc();
        }))!;
    }
    finally
    {
        semaphore.Release();
    }
}