I decided to publish some interesting libraries I develop for internal experiments – after all, one can only watch their side projects gather dust for so long before letting them roam free on GitHub. This time, I want to show off a TTL-based in-memory cache for coroutines, complete with a neat annotation-based code generation trick and some commentary on why I went the coroutine route.
The Big Picture
Caching is one of those things that’s deceptively simple on the surface and full of complexities once you dive in. There’s always that balance between fresh data and not hitting your data store every five milliseconds.
So why did I decide to go with coroutines rather than the usual blocking approach? Kotlin coroutines make concurrency more natural, particularly in scenarios where you’re fanning out calls to multiple data sources. They let you efficiently handle loads of I/O-bound tasks without tying up threads, all while keeping the code easier to read than classic callback soup.
Real-world cases where a TTL cache can save your bacon:
- Microservice aggregator: When you pull data from multiple (sometimes sluggish) endpoints, caching results for even a few minutes can lighten the load and keep your service snappy.
- Frequent reads, infrequent writes: Think of a configuration that rarely changes, but is read hundreds of times a second. A short-lived cache can drastically reduce calls to your database or an external service.
- Rate-limited APIs: If you’re consuming a third-party API with strict limits, you don’t want multiple coroutines hammering it for the same data. A TTL-based approach can help you avoid blowing your quota while making sure you refresh as soon as your data’s “use by” date has passed.
- Expensive computations: Some workloads are just CPU-heavy. Coalescing calls in addition to caching can minimize redundant computations and free up CPU time for real business logic (or for your local build that always seems to need more resources).
TTLCache
The star of the show is TTLCache<K, V>, a locked-down (via Mutex) LinkedHashMap that expires items after a user-defined duration. It supports optional size-based eviction if you need more control over memory usage, and it can coalesce simultaneous load requests to avoid concurrency floods.
suspend fun getOrPut(key: K, loader: suspend () -> V): V
- It checks if your key is still valid in the cache.
- If it’s not cached or is expired, it invokes the provided
loader(). - If you turned on “coalescing,” multiple concurrent calls for the same key get merged into one loader invocation. Less concurrency meltdown, more calm logs.
EvictionPolicy
An enum controlling how we boot items out when the cache is full:
NONE: Items only leave when they expire.LRU: Least Recently Used. Because sometimes you don’t want that one rarely accessed item hogging space until the heat death of the universe.FIFO: The first item in is the first evicted when you reach capacity. Useful if you just want a simple queue approach.
CacheConfig
A data class that neatly bundles our TTL, max size, eviction policy, and coalescing preference – letting you pass it around as a single object. Defaults included if you feel lazy.
CacheManager
An object that acts as a registry of caches keyed by a name: String. It ensures consistent configuration across your app: if you ask for the same name, you get the same cache, ignoring subsequent attempts to reconfigure it.
Annotation-Based Wrapping
Ever get tired of writing the same “check the cache, if not present call the function, then store it” pattern? Enter a custom @Cacheable annotation. Using the Kotlin Symbol Processing (KSP) API, the CacheableProcessor looks for methods annotated with @Cacheable and generates a “Cached” version.
For example:
@Cacheable(ttlSeconds = 60, maxSize = 100, useLRU = true, coalesce = true)
suspend fun getData(id: String): SomeData {
// ...
}
You’ll get a generated getDataCached() that delegates to our TTLCache logic for you.
Wrapping Up
That’s it: a TTL-based coroutine cache that handles concurrency gracefully, offers multiple eviction policies, and even coalesces concurrent calls to the same key. If you feel like exploring or incorporating it into your own projects, the code is right here:
Give it a whirl – because rewriting caching logic is about as much fun as debugging concurrency issues at 3 AM. Enjoy!