Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue in shardIteratorType #226

Open
katyayanaanand opened this issue Oct 25, 2024 · 1 comment
Open

Issue in shardIteratorType #226

katyayanaanand opened this issue Oct 25, 2024 · 1 comment

Comments

@katyayanaanand
Copy link

I saw events lying in my kinesis consumer which were never getting consumed since multiple days and they would be lost after retention time so I decided to change consumer property for shardIteratorType from default LATEST to TRIM_HORIZON. But my idle events present in queue are still not consumed, but new events are getting consumed properly.

kinesis.bindings.consumeMessage-in-0.consumer.checkpointMode=manual,
kinesis.bindings.consumeMessage-in-0.consumer.listenerMode=batch,
kinesis.bindings.consumeMessage-in-0.consumer.shardIteratorType=TRIM_HORIZON,
kinesis.binder.kplKclEnabled=true

I got following in the logs :

The [ShardConsumer{shardOffset=KinesisShardOffset{iteratorType=TRIM_HORIZON, sequenceNumber='null', timestamp=null, stream='message_queue', shard='shardId-000000000000', reset=false}, state=NEW}]
has been started.

Got an exception java.lang.NullPointerException during
[ShardConsumer{shardOffset=KinesisShardOffset{iteratorType=TRIM_HORIZON, sequenceNumber='null', timestamp=null, stream='message_queue', shard='shardId-000000000000', reset=false}, state=CONSUME}] task invocation.
Process will be retried on the next iteration.

Here are some more details :
Java 8, normal kinesis consumer and not fan out.

dependencies :
'org.springframework.boot:spring-boot-starter-web'
'org.springframework.boot:spring-boot-starter-actuator'
'io.micrometer:micrometer-registry-influx'
'org.springframework.cloud:spring-cloud-stream-binder-kinesis:2.2.0'
'com.amazonaws:aws-java-sdk-kinesis:1.12.376'
'com.amazonaws:amazon-kinesis-client:1.14.9'

@artembilan
Copy link
Contributor

Would be great if you can share more info about that NullPointerException.
I mean what is exactly stack trace.

Plus, keep in mind that 2.2.x version of this project is out of support.

On the other hand I wonder if this is related to old fixed issue: #187.
Therefore would you mind trying this in your project: #187 (comment) ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants