Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add redis backed cache as alternative for infinispan #3534

Closed
wants to merge 29 commits into from
Closed

WIP: Add redis backed cache as alternative for infinispan #3534

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
7007975
WIP: Refactor client-device-connection-infinispan module into 3 cache…
StFS Aug 24, 2023
a202675
Restructure modules as per PR comments
StFS Aug 26, 2023
f68e63d
WIP: experimenting with using the vertx-redis client library.
StFS Aug 29, 2023
2a20828
Add Lifecycle interface to RedisCache and simplify creation and closi…
StFS Aug 30, 2023
f7010c7
Merge remote-tracking branch 'origin/master' into implement_redis_bac…
StFS Sep 25, 2023
2f41346
Minor code formatting cleanup from PR comments.
StFS Sep 25, 2023
d51a705
Minor code formatting cleanup from PR comments.
StFS Sep 25, 2023
7e9242e
Clenup of dependencies and some requests for reviews on how that shou…
StFS Sep 27, 2023
e66006d
Integration tests with RedisCache actually working.
StFS Sep 27, 2023
b3a45c2
working on dependencies
StFS Sep 27, 2023
d94f28c
working on dependencies
StFS Sep 27, 2023
58d5fef
Merge remote-tracking branch 'origin/master' into implement_redis_bac…
StFS Sep 28, 2023
91549e6
working on dependencies
StFS Sep 28, 2023
0e6474c
working on dependencies
StFS Sep 28, 2023
00c20fb
First implementation of a Quarkus Redis Client cache.
StFS Oct 3, 2023
f4c31b6
Minor refactoring to unify the names of the Redis cache implementations.
StFS Oct 3, 2023
2669578
Merge remote-tracking branch 'origin/master' into implement_redis_bac…
StFS Oct 26, 2023
27a5715
Merge branch 'eclipse-hono:master' into implement_redis_backed_cache
StFS Nov 27, 2023
d474a01
Merge remote-tracking branch 'origin/master' into implement_redis_bac…
StFS Dec 7, 2023
9bf6855
Merge remote-tracking branch 'origin/master' into implement_redis_bac…
StFS Dec 19, 2023
372c9f4
Baby steps. Now using the quarkus vertx redis API that is injected by…
StFS Mar 2, 2024
c84ce95
Add config relocate interceptor that redirects requests for quarkus.r…
StFS Mar 2, 2024
0f40bf4
Add config relocate interceptor that redirects requests for quarkus.r…
StFS Mar 2, 2024
0043ce8
Merge remote-tracking branch 'origin/implement_redis_backed_cache' in…
StFS Mar 2, 2024
6583875
Merge remote-tracking branch 'origin' into implement_redis_backed_cache
StFS Mar 23, 2024
2b1fe30
Merge remote-tracking branch 'upstream/master' into implement_redis_b…
StFS Mar 23, 2024
b7320d7
Major refactoring, not using Quarkus Redis client anymore and instead…
StFS Mar 25, 2024
37c555b
Minor cleanup of PR
StFS Mar 25, 2024
0f60496
Minor cleanup of PR
StFS Mar 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 20 additions & 5 deletions bom/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,21 @@ quarkus.vertx.max-event-loop-execute-time=${max.event-loop.execute-time:20000}
<artifactId>hono-client-core</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>client-device-connection</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>client-device-connection-infinispan</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>client-device-connection-redis</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-client-common</artifactId>
Expand Down Expand Up @@ -441,11 +456,6 @@ quarkus.vertx.max-event-loop-execute-time=${max.event-loop.execute-time:20000}
<artifactId>hono-service-command-router</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>client-device-connection-infinispan</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-client-application</artifactId>
Expand Down Expand Up @@ -583,6 +593,11 @@ quarkus.vertx.max-event-loop-execute-time=${max.event-loop.execute-time:20000}
<artifactId>vertx-mongo-client</artifactId>
<version>${vertx.version}</version>
</dependency>
<dependency>
<groupId>io.vertx</groupId>
<artifactId>vertx-redis-client</artifactId>
<version>${vertx.version}</version>
</dependency>
<dependency>
<groupId>io.vertx</groupId>
<artifactId>vertx-kafka-client</artifactId>
Expand Down
51 changes: 34 additions & 17 deletions client-device-connection-infinispan/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
<version>2.5.0-SNAPSHOT</version>
<relativePath>../bom</relativePath>
</parent>
<!-- TODO: review in PR request, is there a reason this artifact ID does not start with "hono-"? -->
<artifactId>client-device-connection-infinispan</artifactId>

<name>Hotrod Device Connection client</name>
Expand All @@ -28,12 +29,9 @@
<dependencies>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-legal</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<artifactId>client-device-connection</artifactId>
</dependency>

<dependency>
<groupId>org.infinispan</groupId>
<artifactId>infinispan-client-hotrod</artifactId>
Expand All @@ -52,18 +50,8 @@
<groupId>org.infinispan</groupId>
<artifactId>infinispan-core</artifactId>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-client-common</artifactId>
</dependency>
<dependency>
<groupId>io.vertx</groupId>
<artifactId>vertx-web</artifactId>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>

<!-- TODO: PR review, should this not come transiently from the client-device-connection module? -->
StFS marked this conversation as resolved.
Show resolved Hide resolved
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-core</artifactId>
Expand Down Expand Up @@ -100,6 +88,35 @@
</exclusions>
</dependency>

<!-- TODO: PR review, what of the following is needed since we depend on the common module which has these? -->
<!--
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-legal</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-client-common</artifactId>
</dependency>
<dependency>
<groupId>io.vertx</groupId>
<artifactId>vertx-web</artifactId>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>4.3.1</version>
</dependency>
-->

<!-- testing -->
<dependency>
<groupId>org.junit.jupiter</groupId>
Expand Down
10 changes: 9 additions & 1 deletion ...inispan/src/main/java/org/eclipse/hono/deviceconnection/infinispan/client/BasicCache.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
import java.util.function.Function;

import org.eclipse.hono.client.ServerErrorException;
import org.eclipse.hono.deviceconnection.common.Cache;
import org.eclipse.hono.util.Futures;
import org.eclipse.hono.util.Lifecycle;
import org.infinispan.commons.api.BasicCacheContainer;
Expand Down Expand Up @@ -204,7 +205,14 @@ public Future<V> get(final K key) {
public Future<Map<K, V>> getAll(final Set<? extends K> keys) {
Objects.requireNonNull(keys);

return withCache(aCache -> aCache.getAllAsync(keys));
final var result = withCache(aCache -> aCache.getAllAsync(keys));
result.onSuccess(r -> {
LOG.info("BasicCache: getAll() ({})", keys.size());
final var resultKeys = r.keySet();
r.forEach((k, v) -> LOG.info("BasicCache#getAll() result: {}:{}", k, v));
keys.forEach((k) -> LOG.info("BasicCache#getAll() keys exist in result: {} / {}", k, resultKeys.contains(k)));
});
return result;
}

/**
Expand Down
1 change: 1 addition & 0 deletions ...nispan/src/main/java/org/eclipse/hono/deviceconnection/infinispan/client/HotrodCache.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
import java.util.Objects;
import java.util.concurrent.atomic.AtomicBoolean;

import org.eclipse.hono.deviceconnection.common.CommonCacheConfig;
import org.infinispan.client.hotrod.RemoteCache;
import org.infinispan.client.hotrod.RemoteCacheContainer;
import org.infinispan.client.hotrod.RemoteCacheManager;
Expand Down
2 changes: 2 additions & 0 deletions .../java/org/eclipse/hono/deviceconnection/infinispan/client/QuarkusPropertyBindingTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@

import javax.security.sasl.Sasl;

import org.eclipse.hono.deviceconnection.common.CommonCacheConfig;
import org.eclipse.hono.deviceconnection.common.CommonCacheOptions;
import org.eclipse.hono.test.ConfigMappingSupport;
import org.infinispan.client.hotrod.configuration.ClusterConfiguration;
import org.infinispan.client.hotrod.configuration.Configuration;
Expand Down
210 changes: 210 additions & 0 deletions client-device-connection-redis/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2019, 2022 Contributors to the Eclipse Foundation
StFS marked this conversation as resolved.
Show resolved Hide resolved

See the NOTICE file(s) distributed with this work for additional
information regarding copyright ownership.

This program and the accompanying materials are made available under the
terms of the Eclipse Public License 2.0 which is available at
http://www.eclipse.org/legal/epl-2.0

SPDX-License-Identifier: EPL-2.0
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-bom</artifactId>
<version>2.5.0-SNAPSHOT</version>
<relativePath>../bom</relativePath>
</parent>
<!-- TODO: review in PR request, is there a reason this artifact ID does not start with "hono-"? -->
StFS marked this conversation as resolved.
Show resolved Hide resolved
<artifactId>client-device-connection-redis</artifactId>

<name>Redis Device Connection client</name>
<description>A Redis based client for accessing device connection information in a Redis cluster.</description>

<dependencies>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>client-device-connection</artifactId>
</dependency>

<dependency>
<groupId>io.vertx</groupId>
<artifactId>vertx-redis-client</artifactId>
</dependency>

<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-core</artifactId>
<optional>true</optional>
<exclusions>
<exclusion>
<groupId>org.jboss.logmanager</groupId>
<artifactId>jboss-logmanager-embedded</artifactId>
</exclusion>
<exclusion>
<groupId>org.jboss.logging</groupId>
<artifactId>jboss-logging-annotations</artifactId>
</exclusion>
<exclusion>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-development-mode-spi</artifactId>
</exclusion>
<exclusion>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-bootstrap-runner</artifactId>
</exclusion>
<exclusion>
<groupId>org.jboss.slf4j</groupId>
<artifactId>slf4j-jboss-logmanager</artifactId>
</exclusion>
<exclusion>
<groupId>org.graalvm.sdk</groupId>
<artifactId>graal-sdk</artifactId>
</exclusion>
<exclusion>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-fs-util</artifactId>
</exclusion>
</exclusions>
</dependency>

<!--
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-legal</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
<groupId>org.infinispan</groupId>
<artifactId>infinispan-client-hotrod</artifactId>
<exclusions>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty-transport-native-epoll</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.infinispan</groupId>
<artifactId>infinispan-query-dsl</artifactId>
</dependency>
<dependency>
<groupId>org.infinispan</groupId>
<artifactId>infinispan-core</artifactId>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>hono-client-common</artifactId>
</dependency>
<dependency>
<groupId>io.vertx</groupId>
<artifactId>vertx-web</artifactId>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-core</artifactId>
<optional>true</optional>
<exclusions>
<exclusion>
<groupId>org.jboss.logmanager</groupId>
<artifactId>jboss-logmanager-embedded</artifactId>
</exclusion>
<exclusion>
<groupId>org.jboss.logging</groupId>
<artifactId>jboss-logging-annotations</artifactId>
</exclusion>
<exclusion>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-development-mode-spi</artifactId>
</exclusion>
<exclusion>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-bootstrap-runner</artifactId>
</exclusion>
<exclusion>
<groupId>org.jboss.slf4j</groupId>
<artifactId>slf4j-jboss-logmanager</artifactId>
</exclusion>
<exclusion>
<groupId>org.graalvm.sdk</groupId>
<artifactId>graal-sdk</artifactId>
</exclusion>
<exclusion>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-fs-util</artifactId>
</exclusion>
</exclusions>
</dependency>
-->

<!-- testing -->
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-params</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.truth</groupId>
<artifactId>truth</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.vertx</groupId>
<artifactId>vertx-junit5</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.eclipse.hono</groupId>
<artifactId>core-test-utils</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.jboss.jandex</groupId>
<artifactId>jandex-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
StFS marked this conversation as resolved.
Show resolved Hide resolved
Loading