Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

“The worker got stuck at a certain block height and kept reporting this error: ‘Failed to fetch seqno 40100000: [Error : 651 : (0,9000000000000000,45432823): ltdb: block not found]’ #73

Open
shenao78 opened this issue Sep 2, 2024 · 11 comments

Comments

@shenao78
Copy link

shenao78 commented Sep 2, 2024

I can confirm that the node is synchronizing blocks, and I can see that the last_known_seqno is increasing. However, it consistently fails to retrieve the block (0,9000000000000000,45432823), continuously throwing this error until I restart the worker, after which it runs normally. Is this a bug? Could it be fixed quickly, or can you tell me where the problem might be? Many thanks!
image

@kdimentionaltree
Copy link
Collaborator

Hello, in this case please restart worker.

@shenao78
Copy link
Author

shenao78 commented Sep 3, 2024

Hello, in this case please restart worker.

Do you know the reason? Do you have a plan to fix this? Restarting is not a long-term solution.

@dungeon-master-666
Copy link
Collaborator

Hello, in this case please restart worker.

Do you know the reason? Do you have a plan to fix this? Restarting is not a long-term solution.

Yes, the problem is identified and the fix is in #74. It'll be merged to main very soon. Thanks for reporting.
If you still face the issue, let us know.

@duktig666
Copy link

I had the same problem

Whether can modify https://github.com/toncenter/ton-indexer gitsubmodule as soon as possible

@duktig666
Copy link

image

The problem I ran into was that these logs would loop and not insert data into the database.

@duktig666
Copy link

@dungeon-master-666

I encountered a similar problem, submitted in toncenter/ton-indexer#79

Can you help me with that

@dungeon-master-666
Copy link
Collaborator

@duktig666 based on your screenshot you're trying to index old blocks (seqno 340). Make sure that you have archival node. Otherwise use flag --from to index blocks that your node contains.

@duktig666
Copy link

duktig666 commented Sep 6, 2024

@duktig666 based on your screenshot you're trying to index old blocks (seqno 340). Make sure that you have archival node. Otherwise use flag --from to index blocks that your node contains.

@dungeon-master-666 So which block should I start indexing? Or how can I find suitable information?

I run FullNode+Liteserver, not an archive node.

@dungeon-master-666
Copy link
Collaborator

@dungeon-master-666 So which block should I start indexing? Or how can I find suitable information?

With synchronized node you can start safely from the current blockchain state: https://toncenter.com/api/v2/getMasterchainInfo.

@duktig666
Copy link

@dungeon-master-666 So which block should I start indexing? Or how can I find suitable information?

With synchronized node you can start safely from the current blockchain state: https://toncenter.com/api/v2/getMasterchainInfo.

@dungeon-master-666 If start indexing from the current most recent block state, have problems querying historical data.

I understand that indexing starts at 1 and needs to run the archive node instead of the full node. That should do it?

@aobalv123
Copy link

aobalv123 commented Dec 16, 2024

There is still this problem

[ 2][t 6][2024-12-16 04:35:41.293204115][IndexScheduler.cpp:142][!indexscheduler] Rescheduling seqno 42992316
[ 3][t 5][2024-12-16 04:35:42.290409250][IndexScheduler.cpp:235][!indexscheduler] 42992315 / 42992316 0.344095/s (2s) Q[0M, 0b, 0t, 0m, 0T]
[ 1][t 6][2024-12-16 04:35:42.292091640][IndexScheduler.cpp:131][!indexquery] Failed to fetch seqno 42992316: [Error : 651 : (0,e000000000000000,47888879): ltdb: block not found]
[ 2][t 6][2024-12-16 04:35:42.292121091][IndexScheduler.cpp:142][!indexscheduler] Rescheduling seqno 42992316
[ 1][t 5][2024-12-16 04:35:43.295142571][IndexScheduler.cpp:131][!indexquery] Failed to fetch seqno 42992316: [Error : 651 : (0,e000000000000000,47888879): ltdb: block not found]
[ 2][t 5][2024-12-16 04:35:43.295179388][IndexScheduler.cpp:142][!indexscheduler] Rescheduling seqno 42992316
[ 3][t 1][2024-12-16 04:35:45.293664990][InsertManagerBase.cpp:81][!insertmanager] Inserting batch[mb=2, b=10, txs=160, msgs=259, traces=123]
[ 3][t 1][2024-12-16 04:35:45.529976221][InsertManagerPostgres.cpp:1243][!insert_batch_postgres] Indexed mintless jetton: 0:09F2E59DEC406AB26A5259A45D7FF23EF11F3E5C7C21DE0B0D2A1CBE52B76B3D
[ 3][t 1][2024-12-16 04:35:47.295803417][InsertManagerBase.cpp:81][!insertmanager] Inserting batch[mb=1, b=8, txs=127, msgs=201, traces=98]
[ 3][t 7][2024-12-16 04:35:49.103657834][TraceAssembler.cpp:109][!trace_assembler] Pending traces: 70341 Pending edges: 1561 Broken traces: 0
[ 3][t 6][2024-12-16 04:35:49.297950140][InsertManagerBase.cpp:81][!insertmanager] Inserting batch[mb=1, b=4, txs=45, msgs=79, traces=37]
[ 3][t 4][2024-12-16 04:35:52.300874759][IndexScheduler.cpp:235][!indexscheduler] 42992319 / 42992320 0.349430/s (2s) Q[1M, 5b, 111t, 185m, 85T]
[ 3][t 1][2024-12-16 04:35:52.300891529][InsertManagerBase.cpp:81][!insertmanager] Inserting batch[mb=1, b=5, txs=111, msgs=185, traces=85]
[ 3][t 1][2024-12-16 04:35:52.464706123][InsertManagerPostgres.cpp:1243][!insert_batch_postgres] Indexed mintless jetton: 0:09F2E59DEC406AB26A5259A45D7FF23EF11F3E5C7C21DE0B0D2A1CBE52B76B3D
[ 3][t 6][2024-12-16 04:35:54.303060474][InsertManagerBase.cpp:81][!insertmanager] Inserting batch[mb=1, b=5, txs=86, msgs=133, traces=68]
[ 1][t 5][2024-12-16 04:35:55.309743881][IndexScheduler.cpp:131][!indexquery] Failed to fetch seqno 42992322: [Error : 651 : (0,2000000000000000,48035355): ltdb: block not found]
[ 2][t 5][2024-12-16 04:35:55.309787604][IndexScheduler.cpp:142][!indexscheduler] Rescheduling seqno 42992322
[ 1][t 5][2024-12-16 04:35:56.307246888][IndexScheduler.cpp:131][!indexquery] Failed to fetch seqno 42992322: [Error : 651 : (0,2000000000000000,48035355): ltdb: block not found]
[ 2][t 5][2024-12-16 04:35:56.307290589][IndexScheduler.cpp:142][!indexscheduler] Rescheduling seqno 42992322
[ 1][t 4][2024-12-16 04:35:57.336554923][IndexScheduler.cpp:131][!indexquery] Failed to fetch seqno 42992322: [Error : 651 : (0,2000000000000000,48035355): ltdb: block not found]
[ 2][t 4][2024-12-16 04:35:57.336592670][IndexScheduler.cpp:142][!indexscheduler] Rescheduling seqno 42992322
[ 1][t 7][2024-12-16 04:35:58.309557467][IndexScheduler.cpp:131][!indexquery] Failed to fetch seqno 42992322: [Error : 651 : (0,2000000000000000,48035355): ltdb: block not found]
[ 2][t 7][2024-12-16 04:35:58.309608263][IndexScheduler.cpp:142][!indexscheduler] Rescheduling seqno 42992322
[ 3][t 4][2024-12-16 04:35:59.105185807][TraceAssembler.cpp:109][!trace_assembler] Pending traces: 70325 Pending edges: 1548 Broken traces: 0
[ 3][t 2][2024-12-16 04:36:00.309588736][InsertManagerBase.cpp:81][!insertmanager] Inserting batch[mb=1, b=5, txs=70, msgs=118, traces=53]
[ 3][t 6][2024-12-16 04:36:02.310796849][IndexScheduler.cpp:235][!indexscheduler] 42992322 / 42992322 0.344374/s (0s) Q[0M, 0b, 0t, 0m, 0T]
[ 3][t 6][2024-12-16 04:36:04.312389679][InsertManagerBase.cpp:81][!insertmanager] Inserting batch[mb=1, b=1, txs=3, msgs=1, traces=3]
[ 3][t 1][2024-12-16 04:36:07.315628235][InsertManagerBase.cpp:81][!insertmanager] Inserting batch[mb=1, b=1, txs=3, msgs=1, traces=3]
[ 3][t 3][2024-12-16 04:36:09.106547318][TraceAssembler.cpp:109][!trace_assembler] Pending traces: 70357 Pending edges: 1585 Broken traces: 0
[ 3][t 1][2024-12-16 04:36:09.317872046][InsertManagerBase.cpp:81][!insertmanager] Inserting batch[mb=1, b=5, txs=291, msgs=501, traces=216]

There is still this problem

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants