Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add redis backed cache as alternative for infinispan #3534

Closed
wants to merge 29 commits into from
Closed

WIP: Add redis backed cache as alternative for infinispan #3534

Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
7007975
WIP: Refactor client-device-connection-infinispan module into 3 cache…
StFS Aug 24, 2023
a202675
Restructure modules as per PR comments
StFS Aug 26, 2023
f68e63d
WIP: experimenting with using the vertx-redis client library.
StFS Aug 29, 2023
2a20828
Add Lifecycle interface to RedisCache and simplify creation and closi…
StFS Aug 30, 2023
f7010c7
Merge remote-tracking branch 'origin/master' into implement_redis_bac…
StFS Sep 25, 2023
2f41346
Minor code formatting cleanup from PR comments.
StFS Sep 25, 2023
d51a705
Minor code formatting cleanup from PR comments.
StFS Sep 25, 2023
7e9242e
Clenup of dependencies and some requests for reviews on how that shou…
StFS Sep 27, 2023
e66006d
Integration tests with RedisCache actually working.
StFS Sep 27, 2023
b3a45c2
working on dependencies
StFS Sep 27, 2023
d94f28c
working on dependencies
StFS Sep 27, 2023
58d5fef
Merge remote-tracking branch 'origin/master' into implement_redis_bac…
StFS Sep 28, 2023
91549e6
working on dependencies
StFS Sep 28, 2023
0e6474c
working on dependencies
StFS Sep 28, 2023
00c20fb
First implementation of a Quarkus Redis Client cache.
StFS Oct 3, 2023
f4c31b6
Minor refactoring to unify the names of the Redis cache implementations.
StFS Oct 3, 2023
2669578
Merge remote-tracking branch 'origin/master' into implement_redis_bac…
StFS Oct 26, 2023
27a5715
Merge branch 'eclipse-hono:master' into implement_redis_backed_cache
StFS Nov 27, 2023
d474a01
Merge remote-tracking branch 'origin/master' into implement_redis_bac…
StFS Dec 7, 2023
9bf6855
Merge remote-tracking branch 'origin/master' into implement_redis_bac…
StFS Dec 19, 2023
372c9f4
Baby steps. Now using the quarkus vertx redis API that is injected by…
StFS Mar 2, 2024
c84ce95
Add config relocate interceptor that redirects requests for quarkus.r…
StFS Mar 2, 2024
0f40bf4
Add config relocate interceptor that redirects requests for quarkus.r…
StFS Mar 2, 2024
0043ce8
Merge remote-tracking branch 'origin/implement_redis_backed_cache' in…
StFS Mar 2, 2024
6583875
Merge remote-tracking branch 'origin' into implement_redis_backed_cache
StFS Mar 23, 2024
2b1fe30
Merge remote-tracking branch 'upstream/master' into implement_redis_b…
StFS Mar 23, 2024
b7320d7
Major refactoring, not using Quarkus Redis client anymore and instead…
StFS Mar 25, 2024
37c555b
Minor cleanup of PR
StFS Mar 25, 2024
0f60496
Minor cleanup of PR
StFS Mar 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions bom/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,17 @@ quarkus.vertx.max-event-loop-execute-time=${max.event-loop.execute-time:20000}
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-redis-client</artifactId>
<version>${quarkus.platform.version}</version>
<exclusions>
<exclusion>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-ide-launcher</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-vertx</artifactId>
Expand Down Expand Up @@ -431,11 +442,21 @@ quarkus.vertx.max-event-loop-execute-time=${max.event-loop.execute-time:20000}
<artifactId>hono-service-command-router</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>client-device-connection</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>client-device-connection-infinispan</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>client-device-connection-redis</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-client-application</artifactId>
Expand Down
24 changes: 22 additions & 2 deletions client-device-connection-infinispan/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,20 +20,27 @@
<version>2.5.0-SNAPSHOT</version>
<relativePath>../bom</relativePath>
</parent>
<!-- TODO: review in PR request, is there a reason this artifact ID does not start with "hono-"? -->
<artifactId>client-device-connection-infinispan</artifactId>

<name>Hotrod Device Connection client</name>
<description>A Hotrod based client for accessing device connection information in an Infinispan data grid.</description>

<dependencies>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>client-device-connection</artifactId>
</dependency>

<dependency> <!-- TODO: should I remove hono-legal and just bring it in transiently from client-device-connection? -->
StFS marked this conversation as resolved.
Show resolved Hide resolved
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-legal</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>

<dependency>
<groupId>org.infinispan</groupId>
<artifactId>infinispan-core-jakarta</artifactId>
Expand All @@ -59,18 +66,31 @@
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-client-common</artifactId>
<artifactId>hono-core</artifactId>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-client-common</artifactId>
</dependency>

<dependency> <!-- TODO: changed from vertx-web because there only seems to be dependency on -core. Maybe there are runtime dependencies though??? -->
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sophokles73 do you know? Is it safe to change this or do you think there might be some runtime dependencies that we need from -web?

<groupId>io.vertx</groupId>
<artifactId>vertx-web</artifactId>
<artifactId>vertx-core</artifactId>
</dependency>

<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>

<dependency> <!-- TODO: added because there is direct dependency on it but it was brought in transiently before (probably through Quarkus) -->
<groupId>io.smallrye.config</groupId>
<artifactId>smallrye-config-core</artifactId>
</dependency>

<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-core</artifactId>
Expand Down
18 changes: 17 additions & 1 deletion ...inispan/src/main/java/org/eclipse/hono/deviceconnection/infinispan/client/BasicCache.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
package org.eclipse.hono.deviceconnection.infinispan.client;

import java.net.HttpURLConnection;
import java.util.LinkedList;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
Expand All @@ -24,6 +25,7 @@
import java.util.function.Function;

import org.eclipse.hono.client.ServerErrorException;
import org.eclipse.hono.deviceconnection.common.Cache;
import org.eclipse.hono.util.Futures;
import org.eclipse.hono.util.Lifecycle;
import org.infinispan.commons.api.BasicCacheContainer;
Expand Down Expand Up @@ -155,6 +157,7 @@ protected <T> void postCacheAccess(final AsyncResult<T> cacheOperationResult) {

@Override
public Future<Void> put(final K key, final V value) {
LOG.info("BASIC: put {}={}", key, value);
Objects.requireNonNull(key);
Objects.requireNonNull(value);

Expand All @@ -163,6 +166,7 @@ public Future<Void> put(final K key, final V value) {

@Override
public Future<Void> put(final K key, final V value, final long lifespan, final TimeUnit lifespanUnit) {
LOG.info("BASIC: put {}={} ({} {})", key, value, lifespan, lifespanUnit);
Objects.requireNonNull(key);
Objects.requireNonNull(value);
Objects.requireNonNull(lifespanUnit);
Expand All @@ -172,13 +176,15 @@ public Future<Void> put(final K key, final V value, final long lifespan, final T

@Override
public Future<Void> putAll(final Map<? extends K, ? extends V> data) {
LOG.info("BASIC: putAll ({})", data.size());
Objects.requireNonNull(data);

return withCache(aCache -> aCache.putAllAsync(data));
}

@Override
public Future<Void> putAll(final Map<? extends K, ? extends V> data, final long lifespan, final TimeUnit lifespanUnit) {
LOG.info("BASIC: putAll ({}) ({} {})", data.size(), lifespan, lifespanUnit);
Objects.requireNonNull(data);
Objects.requireNonNull(lifespanUnit);

Expand All @@ -187,6 +193,7 @@ public Future<Void> putAll(final Map<? extends K, ? extends V> data, final long

@Override
public Future<Boolean> remove(final K key, final V value) {
LOG.info("BASIC: remove {}={}", key, value);
Objects.requireNonNull(key);
Objects.requireNonNull(value);

Expand All @@ -195,16 +202,25 @@ public Future<Boolean> remove(final K key, final V value) {

@Override
public Future<V> get(final K key) {
LOG.info("BASIC: get {}", key);
Objects.requireNonNull(key);

return withCache(aCache -> aCache.getAsync(key));
}

@Override
public Future<Map<K, V>> getAll(final Set<? extends K> keys) {
LOG.info("BASIC: getAll {}", keys.size());
Objects.requireNonNull(keys);

return withCache(aCache -> aCache.getAllAsync(keys));
final LinkedList<String> keyList = new LinkedList<>(keys.stream().map(String::valueOf).toList());
keyList.forEach(i -> LOG.info("BASIC: Key: {}", i));
final var result = withCache(aCache -> aCache.getAllAsync(keys));
result.onSuccess(r -> {
LOG.info("BASIC: Got {} items back...", r.size());
r.forEach((k, v) -> LOG.info("Iterating through result list: {}", v));
});
return result;
}

/**
Expand Down
1 change: 1 addition & 0 deletions ...nispan/src/main/java/org/eclipse/hono/deviceconnection/infinispan/client/HotrodCache.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
import java.util.Objects;
import java.util.concurrent.atomic.AtomicBoolean;

import org.eclipse.hono.deviceconnection.common.CommonCacheConfig;
import org.infinispan.client.hotrod.RemoteCache;
import org.infinispan.client.hotrod.RemoteCacheContainer;
import org.infinispan.client.hotrod.RemoteCacheManager;
Expand Down
2 changes: 2 additions & 0 deletions .../java/org/eclipse/hono/deviceconnection/infinispan/client/QuarkusPropertyBindingTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@

import javax.security.sasl.Sasl;

import org.eclipse.hono.deviceconnection.common.CommonCacheConfig;
import org.eclipse.hono.deviceconnection.common.CommonCacheOptions;
import org.eclipse.hono.test.ConfigMappingSupport;
import org.infinispan.client.hotrod.configuration.ClusterConfiguration;
import org.infinispan.client.hotrod.configuration.Configuration;
Expand Down
122 changes: 122 additions & 0 deletions client-device-connection-redis/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2023 Contributors to the Eclipse Foundation

See the NOTICE file(s) distributed with this work for additional
information regarding copyright ownership.

This program and the accompanying materials are made available under the
terms of the Eclipse Public License 2.0 which is available at
http://www.eclipse.org/legal/epl-2.0

SPDX-License-Identifier: EPL-2.0
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-bom</artifactId>
<version>2.5.0-SNAPSHOT</version>
<relativePath>../bom</relativePath>
</parent>
<!-- TODO: review in PR request, is there a reason this artifact ID does not start with "hono-"? -->
StFS marked this conversation as resolved.
Show resolved Hide resolved
<artifactId>client-device-connection-redis</artifactId>

<name>Redis Device Connection client</name>
<description>A Redis based client for accessing device connection information in a Redis cluster.</description>

<dependencies>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>client-device-connection</artifactId>
</dependency>

<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-redis-client</artifactId>
</dependency>

<dependency> <!-- TODO: review should this be included here or just in the parent module? -->
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-legal</artifactId>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-core</artifactId>
</dependency>

<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>

<dependency>
<groupId>io.smallrye.config</groupId>
<artifactId>smallrye-config</artifactId>
</dependency>

<dependency>
<groupId>io.vertx</groupId>
<artifactId>vertx-core</artifactId>
</dependency>

<!-- testing -->
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-params</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.truth</groupId>
<artifactId>truth</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.vertx</groupId>
<artifactId>vertx-junit5</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>core-test-utils</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.jboss.jandex</groupId>
<artifactId>jandex-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
Loading
Loading