Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mgatk tenx mode missing depth and sparse_matrices files #63

Open
triciapark opened this issue Jul 21, 2022 · 4 comments
Open

mgatk tenx mode missing depth and sparse_matrices files #63

triciapark opened this issue Jul 21, 2022 · 4 comments
Assignees
@caleblareau
Labels
bug

Comments

@triciapark
Copy link

triciapark commented Jul 21, 2022 •
edited
Loading

Hello, I've been following the wiki to try to run mgatk tenx (v0.6.1) on 10x scRNA data using the following command:

mgatk tenx -bt CB -g mm10 --snake-stdout -c 12 -ub UB -i results/22_6/outs/possorted_genome_bam.bam -b results/22_6/outs/filtered_feature_bc_matrix/barcodes.tsv -o mgatk_results/sample6 -n sample6

But it seems like there are some missing files. This is the log file:

Config file mgatk_results/sample6/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 12
Rules claiming more threads will be scaled down.
Job stats:
job                           count    min threads    max threads
--------------------------  -------  -------------  -------------
all                               1              1              1
call_variants                     1              1              1
make_depth_table                  1              1              1
make_final_sparse_matrices        1              1              1
process_one_slice                12              1              1
total                            16              1              1

Select jobs to execute...

[Thu Jul 21 11:36:25 2022]
rule process_one_slice:
    input: mgatk_results/sample6/.internal/samples/barcodes.12.bam.txt
    output: mgatk_results/sample6/qc/depth/barcodes.12.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.12.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.12.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.12.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.12.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.12.coverage.txt
    jobid: 3
    reason: Missing output files: mgatk_results/sample6/temp/sparse_matrices/barcodes.12.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.12.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.12.A.txt, mgatk_results/sample6/qc/depth/barcodes.12.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.12.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.12.coverage.txt
    wildcards: sample=barcodes.12
    resources: tmpdir=/tmp

[Thu Jul 21 11:36:25 2022]
rule process_one_slice:
    input: mgatk_results/sample6/.internal/samples/barcodes.7.bam.txt
    output: mgatk_results/sample6/qc/depth/barcodes.7.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.7.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.7.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.7.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.7.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.7.coverage.txt
    jobid: 5
    reason: Missing output files: mgatk_results/sample6/temp/sparse_matrices/barcodes.7.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.7.coverage.txt, mgatk_results/sample6/qc/depth/barcodes.7.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.7.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.7.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.7.G.txt
    wildcards: sample=barcodes.7
    resources: tmpdir=/tmp

[Thu Jul 21 11:36:25 2022]
rule process_one_slice:
    input: mgatk_results/sample6/.internal/samples/barcodes.11.bam.txt
    output: mgatk_results/sample6/qc/depth/barcodes.11.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.11.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.11.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.11.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.11.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.11.coverage.txt
    jobid: 7
    reason: Missing output files: mgatk_results/sample6/qc/depth/barcodes.11.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.11.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.11.coverage.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.11.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.11.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.11.G.txt
    wildcards: sample=barcodes.11
    resources: tmpdir=/tmp

[Thu Jul 21 11:36:25 2022]
rule process_one_slice:
    input: mgatk_results/sample6/.internal/samples/barcodes.10.bam.txt
    output: mgatk_results/sample6/qc/depth/barcodes.10.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.10.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.10.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.10.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.10.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.10.coverage.txt
    jobid: 9
    reason: Missing output files: mgatk_results/sample6/qc/depth/barcodes.10.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.10.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.10.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.10.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.10.coverage.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.10.G.txt
    wildcards: sample=barcodes.10
    resources: tmpdir=/tmp

[Thu Jul 21 11:36:26 2022]
rule process_one_slice:
    input: mgatk_results/sample6/.internal/samples/barcodes.5.bam.txt
    output: mgatk_results/sample6/qc/depth/barcodes.5.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.5.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.5.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.5.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.5.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.5.coverage.txt
    jobid: 11
    reason: Missing output files: mgatk_results/sample6/temp/sparse_matrices/barcodes.5.coverage.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.5.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.5.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.5.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.5.T.txt, mgatk_results/sample6/qc/depth/barcodes.5.depth.txt
    wildcards: sample=barcodes.5
    resources: tmpdir=/tmp

[Thu Jul 21 11:36:26 2022]
rule process_one_slice:
    input: mgatk_results/sample6/.internal/samples/barcodes.1.bam.txt
    output: mgatk_results/sample6/qc/depth/barcodes.1.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.1.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.1.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.1.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.1.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.1.coverage.txt
    jobid: 13
    reason: Missing output files: mgatk_results/sample6/temp/sparse_matrices/barcodes.1.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.1.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.1.A.txt, mgatk_results/sample6/qc/depth/barcodes.1.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.1.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.1.coverage.txt
    wildcards: sample=barcodes.1
    resources: tmpdir=/tmp

[Thu Jul 21 11:36:26 2022]
rule process_one_slice:
    input: mgatk_results/sample6/.internal/samples/barcodes.3.bam.txt
    output: mgatk_results/sample6/qc/depth/barcodes.3.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.3.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.3.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.3.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.3.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.3.coverage.txt
    jobid: 2
    reason: Missing output files: mgatk_results/sample6/temp/sparse_matrices/barcodes.3.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.3.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.3.G.txt, mgatk_results/sample6/qc/depth/barcodes.3.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.3.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.3.coverage.txt
    wildcards: sample=barcodes.3
    resources: tmpdir=/tmp

[Thu Jul 21 11:36:26 2022]
rule process_one_slice:
    input: mgatk_results/sample6/.internal/samples/barcodes.4.bam.txt
    output: mgatk_results/sample6/qc/depth/barcodes.4.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.4.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.4.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.4.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.4.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.4.coverage.txt
    jobid: 4
    reason: Missing output files: mgatk_results/sample6/temp/sparse_matrices/barcodes.4.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.4.coverage.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.4.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.4.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.4.T.txt, mgatk_results/sample6/qc/depth/barcodes.4.depth.txt
    wildcards: sample=barcodes.4
    resources: tmpdir=/tmp

[Thu Jul 21 11:36:26 2022]
rule process_one_slice:
    input: mgatk_results/sample6/.internal/samples/barcodes.6.bam.txt
    output: mgatk_results/sample6/qc/depth/barcodes.6.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.6.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.6.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.6.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.6.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.6.coverage.txt
    jobid: 6
    reason: Missing output files: mgatk_results/sample6/temp/sparse_matrices/barcodes.6.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.6.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.6.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.6.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.6.coverage.txt, mgatk_results/sample6/qc/depth/barcodes.6.depth.txt
    wildcards: sample=barcodes.6
    resources: tmpdir=/tmp

[Thu Jul 21 11:36:26 2022]
rule process_one_slice:
    input: mgatk_results/sample6/.internal/samples/barcodes.9.bam.txt
    output: mgatk_results/sample6/qc/depth/barcodes.9.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.9.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.9.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.9.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.9.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.9.coverage.txt
    jobid: 8
    reason: Missing output files: mgatk_results/sample6/temp/sparse_matrices/barcodes.9.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.9.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.9.coverage.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.9.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.9.G.txt, mgatk_results/sample6/qc/depth/barcodes.9.depth.txt
    wildcards: sample=barcodes.9
    resources: tmpdir=/tmp

[Thu Jul 21 11:36:26 2022]
rule process_one_slice:
    input: mgatk_results/sample6/.internal/samples/barcodes.8.bam.txt
    output: mgatk_results/sample6/qc/depth/barcodes.8.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.8.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.8.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.8.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.8.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.8.coverage.txt
    jobid: 10
    reason: Missing output files: mgatk_results/sample6/temp/sparse_matrices/barcodes.8.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.8.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.8.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.8.A.txt, mgatk_results/sample6/qc/depth/barcodes.8.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.8.coverage.txt
    wildcards: sample=barcodes.8
    resources: tmpdir=/tmp

[Thu Jul 21 11:36:26 2022]
rule process_one_slice:
    input: mgatk_results/sample6/.internal/samples/barcodes.2.bam.txt
    output: mgatk_results/sample6/qc/depth/barcodes.2.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.2.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.2.C.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.2.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.2.T.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.2.coverage.txt
    jobid: 12
    reason: Missing output files: mgatk_results/sample6/temp/sparse_matrices/barcodes.2.coverage.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.2.C.txt, mgatk_results/sample6/qc/depth/barcodes.2.depth.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.2.G.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.2.A.txt, mgatk_results/sample6/temp/sparse_matrices/barcodes.2.T.txt
    wildcards: sample=barcodes.2
    resources: tmpdir=/tmp

Terminating processes on user request, this might take some time.
Complete log: .snakemake/log/2022-07-21T113623.063765.snakemake.log

And ls -lRh of the output folder shows this:

.:
total 5.0K
drwxr-xr-x 2 parkt reznik 4.0K Jul 21 11:14 fasta
drwxr-xr-x 2 parkt reznik 4.0K Jul 21 11:14 final
drwxr-xr-x 4 parkt reznik 4.0K Jul 21 11:36 logs
drwxr-xr-x 3 parkt reznik 4.0K Jul 21 11:38 qc
drwxr-xr-x 7 parkt reznik 4.0K Jul 21 11:38 temp

./fasta:
total 33K
-rw-r--r-- 1 parkt reznik 17K Jul 21 11:14 chrM.fasta
-rw-r--r-- 1 parkt reznik  19 Jul 21 11:14 chrM.fasta.fai

./final:
total 128K
-rw-r--r-- 1 parkt reznik 117K Jul 21 11:14 chrM_refAllele.txt

./logs:
total 3.0K
-rw-r--r-- 1 parkt reznik  124 Jul 21 11:36 base.mgatk.log
drwxr-xr-x 2 parkt reznik 4.0K Jul 21 11:37 filterlogs
drwxr-xr-x 2 parkt reznik 4.0K Jul 21 11:36 rmdupslogs
-rw-r--r-- 1 parkt reznik  530 Jul 21 11:36 sample6.parameters.txt

./logs/filterlogs:
total 6.0K
-rw-r--r-- 1 parkt reznik 26 Jul 21 11:37 barcodes.10.filter.log
-rw-r--r-- 1 parkt reznik 26 Jul 21 11:37 barcodes.11.filter.log
-rw-r--r-- 1 parkt reznik 26 Jul 21 11:37 barcodes.12.filter.log
-rw-r--r-- 1 parkt reznik 26 Jul 21 11:37 barcodes.1.filter.log
-rw-r--r-- 1 parkt reznik 26 Jul 21 11:37 barcodes.2.filter.log
-rw-r--r-- 1 parkt reznik 26 Jul 21 11:37 barcodes.3.filter.log
-rw-r--r-- 1 parkt reznik 26 Jul 21 11:37 barcodes.4.filter.log
-rw-r--r-- 1 parkt reznik 26 Jul 21 11:37 barcodes.5.filter.log
-rw-r--r-- 1 parkt reznik 26 Jul 21 11:37 barcodes.6.filter.log
-rw-r--r-- 1 parkt reznik 26 Jul 21 11:37 barcodes.7.filter.log
-rw-r--r-- 1 parkt reznik 26 Jul 21 11:37 barcodes.8.filter.log
-rw-r--r-- 1 parkt reznik 26 Jul 21 11:37 barcodes.9.filter.log

./logs/rmdupslogs:
total 0

./qc:
total 1.0K
drwxr-xr-x 2 parkt reznik 4.0K Jul 21 11:36 quality

./qc/quality:
total 0

./temp:
total 5.0K
drwxr-xr-x 2 parkt reznik 4.0K Jul 21 11:36 barcoded_bams
drwxr-xr-x 2 parkt reznik 4.0K Jul 21 11:14 barcode_files
drwxr-xr-x 2 parkt reznik 4.0K Jul 21 11:36 quality
drwxr-xr-x 2 parkt reznik 4.0K Jul 21 11:36 ready_bam
drwxr-xr-x 2 parkt reznik 4.0K Jul 21 11:37 temp_bam

./temp/barcoded_bams:
total 163M
-rw-r--r-- 1 parkt reznik 14M Jul 21 11:36 barcodes.10.bam
-rw-r--r-- 1 parkt reznik 616 Jul 21 11:36 barcodes.10.bam.bai
-rw-r--r-- 1 parkt reznik 13M Jul 21 11:36 barcodes.11.bam
-rw-r--r-- 1 parkt reznik 616 Jul 21 11:36 barcodes.11.bam.bai
-rw-r--r-- 1 parkt reznik 14M Jul 21 11:36 barcodes.12.bam
-rw-r--r-- 1 parkt reznik 616 Jul 21 11:36 barcodes.12.bam.bai
-rw-r--r-- 1 parkt reznik 14M Jul 21 11:36 barcodes.1.bam
-rw-r--r-- 1 parkt reznik 616 Jul 21 11:36 barcodes.1.bam.bai
-rw-r--r-- 1 parkt reznik 14M Jul 21 11:36 barcodes.2.bam
-rw-r--r-- 1 parkt reznik 616 Jul 21 11:36 barcodes.2.bam.bai
-rw-r--r-- 1 parkt reznik 14M Jul 21 11:36 barcodes.3.bam
-rw-r--r-- 1 parkt reznik 616 Jul 21 11:36 barcodes.3.bam.bai
-rw-r--r-- 1 parkt reznik 14M Jul 21 11:36 barcodes.4.bam
-rw-r--r-- 1 parkt reznik 616 Jul 21 11:36 barcodes.4.bam.bai
-rw-r--r-- 1 parkt reznik 14M Jul 21 11:36 barcodes.5.bam
-rw-r--r-- 1 parkt reznik 616 Jul 21 11:36 barcodes.5.bam.bai
-rw-r--r-- 1 parkt reznik 14M Jul 21 11:36 barcodes.6.bam
-rw-r--r-- 1 parkt reznik 616 Jul 21 11:36 barcodes.6.bam.bai
-rw-r--r-- 1 parkt reznik 14M Jul 21 11:36 barcodes.7.bam
-rw-r--r-- 1 parkt reznik 616 Jul 21 11:36 barcodes.7.bam.bai
-rw-r--r-- 1 parkt reznik 14M Jul 21 11:36 barcodes.8.bam
-rw-r--r-- 1 parkt reznik 616 Jul 21 11:36 barcodes.8.bam.bai
-rw-r--r-- 1 parkt reznik 14M Jul 21 11:36 barcodes.9.bam
-rw-r--r-- 1 parkt reznik 616 Jul 21 11:36 barcodes.9.bam.bai

./temp/barcode_files:
total 384K
-rw-r--r-- 1 parkt reznik 26K Jul 21 11:14 barcodes.10.txt
-rw-r--r-- 1 parkt reznik 26K Jul 21 11:14 barcodes.11.txt
-rw-r--r-- 1 parkt reznik 26K Jul 21 11:14 barcodes.12.txt
-rw-r--r-- 1 parkt reznik 26K Jul 21 11:14 barcodes.1.txt
-rw-r--r-- 1 parkt reznik 26K Jul 21 11:14 barcodes.2.txt
-rw-r--r-- 1 parkt reznik 26K Jul 21 11:14 barcodes.3.txt
-rw-r--r-- 1 parkt reznik 26K Jul 21 11:14 barcodes.4.txt
-rw-r--r-- 1 parkt reznik 26K Jul 21 11:14 barcodes.5.txt
-rw-r--r-- 1 parkt reznik 26K Jul 21 11:14 barcodes.6.txt
-rw-r--r-- 1 parkt reznik 26K Jul 21 11:14 barcodes.7.txt
-rw-r--r-- 1 parkt reznik 26K Jul 21 11:14 barcodes.8.txt
-rw-r--r-- 1 parkt reznik 26K Jul 21 11:14 barcodes.9.txt

./temp/quality:
total 0

./temp/ready_bam:
total 0

./temp/temp_bam:
total 111M
-rw-r--r-- 1 parkt reznik 4.7M Jul 21 11:37 barcodes.10.temp0.bam
-rw-r--r-- 1 parkt reznik 4.7M Jul 21 11:37 barcodes.10.temp1.bam
-rw-r--r-- 1 parkt reznik  616 Jul 21 11:37 barcodes.10.temp1.bam.bai
-rw-r--r-- 1 parkt reznik 4.5M Jul 21 11:37 barcodes.11.temp0.bam
-rw-r--r-- 1 parkt reznik 4.5M Jul 21 11:37 barcodes.11.temp1.bam
-rw-r--r-- 1 parkt reznik  616 Jul 21 11:37 barcodes.11.temp1.bam.bai
-rw-r--r-- 1 parkt reznik 4.7M Jul 21 11:37 barcodes.12.temp0.bam
-rw-r--r-- 1 parkt reznik 4.7M Jul 21 11:37 barcodes.12.temp1.bam
-rw-r--r-- 1 parkt reznik  616 Jul 21 11:37 barcodes.12.temp1.bam.bai
-rw-r--r-- 1 parkt reznik 4.6M Jul 21 11:37 barcodes.1.temp0.bam
-rw-r--r-- 1 parkt reznik 4.6M Jul 21 11:37 barcodes.1.temp1.bam
-rw-r--r-- 1 parkt reznik  616 Jul 21 11:37 barcodes.1.temp1.bam.bai
-rw-r--r-- 1 parkt reznik 4.8M Jul 21 11:37 barcodes.2.temp0.bam
-rw-r--r-- 1 parkt reznik 4.8M Jul 21 11:37 barcodes.2.temp1.bam
-rw-r--r-- 1 parkt reznik  616 Jul 21 11:37 barcodes.2.temp1.bam.bai
-rw-r--r-- 1 parkt reznik 4.8M Jul 21 11:37 barcodes.3.temp0.bam
-rw-r--r-- 1 parkt reznik 4.8M Jul 21 11:37 barcodes.3.temp1.bam
-rw-r--r-- 1 parkt reznik  616 Jul 21 11:37 barcodes.3.temp1.bam.bai
-rw-r--r-- 1 parkt reznik 4.6M Jul 21 11:37 barcodes.4.temp0.bam
-rw-r--r-- 1 parkt reznik 4.6M Jul 21 11:37 barcodes.4.temp1.bam
-rw-r--r-- 1 parkt reznik  616 Jul 21 11:37 barcodes.4.temp1.bam.bai
-rw-r--r-- 1 parkt reznik 4.6M Jul 21 11:37 barcodes.5.temp0.bam
-rw-r--r-- 1 parkt reznik 4.6M Jul 21 11:37 barcodes.5.temp1.bam
-rw-r--r-- 1 parkt reznik  616 Jul 21 11:37 barcodes.5.temp1.bam.bai
-rw-r--r-- 1 parkt reznik 4.7M Jul 21 11:37 barcodes.6.temp0.bam
-rw-r--r-- 1 parkt reznik 4.7M Jul 21 11:37 barcodes.6.temp1.bam
-rw-r--r-- 1 parkt reznik  616 Jul 21 11:37 barcodes.6.temp1.bam.bai
-rw-r--r-- 1 parkt reznik 4.7M Jul 21 11:37 barcodes.7.temp0.bam
-rw-r--r-- 1 parkt reznik 4.7M Jul 21 11:37 barcodes.7.temp1.bam
-rw-r--r-- 1 parkt reznik  616 Jul 21 11:37 barcodes.7.temp1.bam.bai
-rw-r--r-- 1 parkt reznik 4.5M Jul 21 11:37 barcodes.8.temp0.bam
-rw-r--r-- 1 parkt reznik 4.5M Jul 21 11:37 barcodes.8.temp1.bam
-rw-r--r-- 1 parkt reznik  616 Jul 21 11:37 barcodes.8.temp1.bam.bai
-rw-r--r-- 1 parkt reznik 4.6M Jul 21 11:37 barcodes.9.temp0.bam
-rw-r--r-- 1 parkt reznik 4.6M Jul 21 11:37 barcodes.9.temp1.bam
-rw-r--r-- 1 parkt reznik  616 Jul 21 11:37 barcodes.9.temp1.bam.bai

Additional Information:
I was able to run mgatk on the example code with the call mgatk tenx -i test_barcode.bam -b test_barcodes.txt -o mgatk_results/test -n test -bt CB -c 2
I am using the following versions:
Python 3.9.7
R 4.0.0
Snakemake 7.8.5
@triciapark triciapark added the bug label Jul 21, 2022
@teresa1997
Copy link

Hi, we have a similar problem.
I'm running mgatk in tenx mode

mgatk tenx -n 10 -i /Output_CellRanger/AUGU_possorted_genome_bam.bam -n AUGU -o /Output_CellRanger/output_mgatk -b /Output_CellRanger/AUGU_barcodes.txt -bt CB

But as @triciapark we have some missing files.

Here the log

Config file /Output_CellRanger/output_mgatk/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Job stats:
job                           count    min threads    max threads
--------------------------  -------  -------------  -------------
all                               1              1              1
call_variants                     1              1              1
make_depth_table                  1              1              1
make_final_sparse_matrices        1              1              1
process_one_slice                10              1              1
total                            14              1              1

Select jobs to execute...

[Tue Jul  4 10:57:08 2023]
rule process_one_slice:
    input: /Output_CellRanger/output_mgatk/.internal/samples/barcodes.5.bam.txt
    output: /Output_CellRanger/output_mgatk/qc/depth/barcodes.5.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.5.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.5.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.5.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.5.T.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.5.coverage.txt
    jobid: 5
    reason: Missing output files: /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.5.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.5.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.5.T.txt, /Output_CellRanger/output_mgatk/qc/depth/barcodes.5.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.5.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.5.coverage.txt
    wildcards: sample=barcodes.5
    resources: tmpdir=/scratch/tmp

[Tue Jul  4 10:57:08 2023]
rule process_one_slice:
    input: /Output_CellRanger/output_mgatk/.internal/samples/barcodes.6.bam.txt
    output: /Output_CellRanger/output_mgatk/qc/depth/barcodes.6.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.6.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.6.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.6.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.6.T.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.6.coverage.txt
    jobid: 10
    reason: Missing output files: /Output_CellRanger/output_mgatk/qc/depth/barcodes.6.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.6.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.6.coverage.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.6.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.6.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.6.T.txt
    wildcards: sample=barcodes.6
    resources: tmpdir=/scratch/tmp

[Tue Jul  4 10:57:08 2023]
rule process_one_slice:
    input: /Output_CellRanger/output_mgatk/.internal/samples/barcodes.3.bam.txt
    output: /Output_CellRanger/output_mgatk/qc/depth/barcodes.3.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.3.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.3.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.3.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.3.T.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.3.coverage.txt
    jobid: 7
    reason: Missing output files: /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.3.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.3.coverage.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.3.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.3.T.txt, /Output_CellRanger/output_mgatk/qc/depth/barcodes.3.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.3.G.txt
    wildcards: sample=barcodes.3
    resources: tmpdir=/scratch/tmp

[Tue Jul  4 10:57:08 2023]
rule process_one_slice:
    input: /Output_CellRanger/output_mgatk/.internal/samples/barcodes.1.bam.txt
    output: /Output_CellRanger/output_mgatk/qc/depth/barcodes.1.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.1.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.1.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.1.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.1.T.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.1.coverage.txt
    jobid: 4
    reason: Missing output files: /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.1.T.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.1.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.1.A.txt, /Output_CellRanger/output_mgatk/qc/depth/barcodes.1.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.1.coverage.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.1.G.txt
    wildcards: sample=barcodes.1
    resources: tmpdir=/scratch/tmp

[Tue Jul  4 10:57:08 2023]
rule process_one_slice:
    input: /Output_CellRanger/output_mgatk/.internal/samples/barcodes.4.bam.txt
    output: /Output_CellRanger/output_mgatk/qc/depth/barcodes.4.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.4.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.4.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.4.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.4.T.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.4.coverage.txt
    jobid: 3
    reason: Missing output files: /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.4.T.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.4.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.4.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.4.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.4.coverage.txt, /Output_CellRanger/output_mgatk/qc/depth/barcodes.4.depth.txt
    wildcards: sample=barcodes.4
    resources: tmpdir=/scratch/tmp

[Tue Jul  4 10:57:08 2023]
rule process_one_slice:
    input: /Output_CellRanger/output_mgatk/.internal/samples/barcodes.2.bam.txt
    output: /Output_CellRanger/output_mgatk/qc/depth/barcodes.2.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.2.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.2.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.2.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.2.T.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.2.coverage.txt
    jobid: 9
    reason: Missing output files: /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.2.T.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.2.C.txt, /Output_CellRanger/output_mgatk/qc/depth/barcodes.2.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.2.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.2.coverage.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.2.G.txt
    wildcards: sample=barcodes.2
    resources: tmpdir=/scratch/tmp

[Tue Jul  4 10:57:08 2023]
rule process_one_slice:
    input: /Output_CellRanger/output_mgatk/.internal/samples/barcodes.9.bam.txt
    output: /Output_CellRanger/output_mgatk/qc/depth/barcodes.9.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.9.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.9.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.9.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.9.T.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.9.coverage.txt
    jobid: 6
    reason: Missing output files: /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.9.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.9.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.9.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.9.T.txt, /Output_CellRanger/output_mgatk/qc/depth/barcodes.9.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.9.coverage.txt
    wildcards: sample=barcodes.9
    resources: tmpdir=/scratch/tmp

[Tue Jul  4 10:57:08 2023]
rule process_one_slice:
    input: /Output_CellRanger/output_mgatk/.internal/samples/barcodes.8.bam.txt
    output: /Output_CellRanger/output_mgatk/qc/depth/barcodes.8.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.8.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.8.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.8.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.8.T.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.8.coverage.txt
    jobid: 2
    reason: Missing output files: /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.8.coverage.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.8.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.8.T.txt, /Output_CellRanger/output_mgatk/qc/depth/barcodes.8.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.8.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.8.C.txt
    wildcards: sample=barcodes.8
    resources: tmpdir=/scratch/tmp

[Tue Jul  4 10:57:08 2023]
rule process_one_slice:
    input: /Output_CellRanger/output_mgatk/.internal/samples/barcodes.10.bam.txt
    output: /Output_CellRanger/output_mgatk/qc/depth/barcodes.10.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.10.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.10.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.10.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.10.T.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.10.coverage.txt
    jobid: 11
    reason: Missing output files: /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.10.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.10.T.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.10.coverage.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.10.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.10.G.txt, /Output_CellRanger/output_mgatk/qc/depth/barcodes.10.depth.txt
    wildcards: sample=barcodes.10
    resources: tmpdir=/scratch/tmp

[Tue Jul  4 10:57:08 2023]
rule process_one_slice:
    input: /Output_CellRanger/output_mgatk/.internal/samples/barcodes.7.bam.txt
    output: /Output_CellRanger/output_mgatk/qc/depth/barcodes.7.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.7.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.7.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.7.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.7.T.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.7.coverage.txt
    jobid: 8
    reason: Missing output files: 
/Output_CellRanger/output_mgatk/qc/depth/barcodes.7.depth.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.7.G.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.7.T.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.7.C.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.7.A.txt, /Output_CellRanger/output_mgatk/temp/sparse_matrices/barcodes.7.coverage.txt
    wildcards: sample=barcodes.7
    resources: tmpdir=/scratch/tmp

Config file /Output_CellRanger/output_mgatk/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Config file /Output_CellRanger/output_mgatk/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Config file /Output_CellRanger/output_mgatk/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Config file /Output_CellRanger/output_mgatk/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Building DAG of jobs...
Config file /Output_CellRanger/output_mgatk/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Config file /Output_CellRanger/output_mgatk/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Config file /Output_CellRanger/output_mgatk/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Config file /Output_CellRanger/output_mgatk/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Config file /Output_CellRanger/output_mgatk/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Config file /Output_CellRanger/output_mgatk/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Building DAG of jobs...
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Error in checkGrep(grep(".A.txt", files)) : 
  Improper folder specification; file missing / extra file present. See documentation
Calls: importMito -> checkGrep
Execution halted

This is the listo of the output files:

.:
total 30K
drwxr-sr-x 2 cvt   3 Jul  3 09:45 final
drwxr-sr-x 4 cvt   7 Jul  3 09:46 logs
drwxr-sr-x 4 cvt   4 Jul  3 09:46 qc

./final:
total 114K
-rw-r--r-- 1 cvt 119K Jul  4 10:55 chrM_refAllele.txt

./logs:
total 9.0K
-rw-r--r-- 1 cvt  778 Jul  4 10:57 AUGU.parameters.txt
-rw-r--r-- 1 cvt    0 Jul  4 10:57 AUGU.snakemake_tenx.log
-rw-r--r-- 1 cvt  4.9K Jul  4 10:57 base.mgatk.log
drwxr-sr-x 2 cvt    2 Jul  3 09:46 filterlogs
drwxr-sr-x 2 cvt    2 Jul  3 09:46 rmdupslogs

./logs/filterlogs:
total 0

./logs/rmdupslogs:
total 0

./qc:
total 1.0K
drwxr-sr-x 2 cvt 2 Jul  3 09:46 depth
drwxr-sr-x 2 cvt 2 Jul  3 09:46 quality

./qc/depth:
total 0

./qc/quality:
total 0

I also tried with the test file

mgatk bcall -i test_barcode.bam -n bc1 -o bc1dmem -bt CB -b test_barcodes.txt -z`

but I have the same error 

`[W::hts_idx_load3] The index file is older than the data file: bc1dmem/temp/barcoded_bams/GCCTAGGCAGTTCGGC-1.bam.bai
[W::hts_idx_load3] The index file is older than the data file: bc1dmem/temp/barcoded_bams/CTAACTTAGAGCCACA-1.bam.bai
[W::hts_idx_load3] The index file is older than the data file: bc1dmem/temp/barcoded_bams/CACCACTAGGAGGCGA-1.bam.bai
Config file bc1dmem/.internal/parseltongue/snake.gather.yaml is extended by additional config specified via the command line.
Config file bc1dmem/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Building DAG of jobs...
Using shell: /usr/bin/bash
Using shell: /usr/bin/bash
Provided cores: 10
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Rules claiming more threads will be scaled down.
Job stats:
job                   count    min threads    max threads
------------------  -------  -------------  -------------
all                       1              1              1
make_sample_list          1              1              1
process_one_sample        6              1              1
total                     8              1              1

Job stats:
job                           count    min threads    max threads
--------------------------  -------  -------------  -------------
all                               1              1              1
make_depth_table                  1              1              1
make_final_sparse_matrices        1              1              1
total                             3              1              1

Select jobs to execute...
Select jobs to execute...

[Tue Jul  4 13:20:36 2023]
rule make_depth_table:
    output: bc1dmem/final/bc1.depthTable.txt
    jobid: 1
    reason: Missing output files: bc1dmem/final/bc1.depthTable.txt
    resources: tmpdir=/scratch/tmp


[Tue Jul  4 13:20:36 2023]
rule process_one_sample:
    input: bc1dmem/.internal/samples/CTAACTTAGAGCCACA-1.bam.txt
    output: bc1dmem/temp/ready_bam/CTAACTTAGAGCCACA-1.qc.bam, bc1dmem/temp/ready_bam/CTAACTTAGAGCCACA-1.qc.bam.bai, bc1dmem/qc/depth/CTAACTTAGAGCCACA-1.depth.txt, bc1dmem/temp/sparse_matrices/CTAACTTAGAGCCACA-1.A.txt, bc1dmem/temp/sparse_matrices/CTAACTTAGAGCCACA-1.C.txt, bc1dmem/temp/sparse_matrices/CTAACTTAGAGCCACA-1.G.txt, bc1dmem/temp/sparse_matrices/CTAACTTAGAGCCACA-1.T.txt, bc1dmem/temp/sparse_matrices/CTAACTTAGAGCCACA-1.coverage.txt
    jobid: 6
    reason: Missing output files: bc1dmem/qc/depth/CTAACTTAGAGCCACA-1.depth.txt
    wildcards: sample=CTAACTTAGAGCCACA-1
    resources: tmpdir=/scratch/tmp

[Tue Jul  4 13:20:36 2023]
rule process_one_sample:
    input: bc1dmem/.internal/samples/barcodes.3.bam.txt
    output: bc1dmem/temp/ready_bam/barcodes.3.qc.bam, bc1dmem/temp/ready_bam/barcodes.3.qc.bam.bai, bc1dmem/qc/depth/barcodes.3.depth.txt, bc1dmem/temp/sparse_matrices/barcodes.3.A.txt, bc1dmem/temp/sparse_matrices/barcodes.3.C.txt, bc1dmem/temp/sparse_matrices/barcodes.3.G.txt, bc1dmem/temp/sparse_matrices/barcodes.3.T.txt, bc1dmem/temp/sparse_matrices/barcodes.3.coverage.txt
    jobid: 4
    reason: Missing output files: bc1dmem/qc/depth/barcodes.3.depth.txt
    wildcards: sample=barcodes.3
    resources: tmpdir=/scratch/tmp

[Tue Jul  4 13:20:36 2023]
rule process_one_sample:
    input: bc1dmem/.internal/samples/CACCACTAGGAGGCGA-1.bam.txt
    output: bc1dmem/temp/ready_bam/CACCACTAGGAGGCGA-1.qc.bam, bc1dmem/temp/ready_bam/CACCACTAGGAGGCGA-1.qc.bam.bai, bc1dmem/qc/depth/CACCACTAGGAGGCGA-1.depth.txt, bc1dmem/temp/sparse_matrices/CACCACTAGGAGGCGA-1.A.txt, bc1dmem/temp/sparse_matrices/CACCACTAGGAGGCGA-1.C.txt, bc1dmem/temp/sparse_matrices/CACCACTAGGAGGCGA-1.G.txt, bc1dmem/temp/sparse_matrices/CACCACTAGGAGGCGA-1.T.txt, bc1dmem/temp/sparse_matrices/CACCACTAGGAGGCGA-1.coverage.txt
    jobid: 5
    reason: Missing output files: bc1dmem/qc/depth/CACCACTAGGAGGCGA-1.depth.txt
    wildcards: sample=CACCACTAGGAGGCGA-1
    resources: tmpdir=/scratch/tmp

[Tue Jul  4 13:20:36 2023]
rule process_one_sample:
    input: bc1dmem/.internal/samples/barcodes.2.bam.txt
    output: bc1dmem/temp/ready_bam/barcodes.2.qc.bam, bc1dmem/temp/ready_bam/barcodes.2.qc.bam.bai, bc1dmem/qc/depth/barcodes.2.depth.txt, bc1dmem/temp/sparse_matrices/barcodes.2.A.txt, bc1dmem/temp/sparse_matrices/barcodes.2.C.txt, bc1dmem/temp/sparse_matrices/barcodes.2.G.txt, bc1dmem/temp/sparse_matrices/barcodes.2.T.txt, bc1dmem/temp/sparse_matrices/barcodes.2.coverage.txt
    jobid: 7
    reason: Missing output files: bc1dmem/qc/depth/barcodes.2.depth.txt
    wildcards: sample=barcodes.2
    resources: tmpdir=/scratch/tmp

[Tue Jul  4 13:20:36 2023]
rule process_one_sample:
    input: bc1dmem/.internal/samples/barcodes.1.bam.txt
    output: bc1dmem/temp/ready_bam/barcodes.1.qc.bam, bc1dmem/temp/ready_bam/barcodes.1.qc.bam.bai, bc1dmem/qc/depth/barcodes.1.depth.txt, bc1dmem/temp/sparse_matrices/barcodes.1.A.txt, bc1dmem/temp/sparse_matrices/barcodes.1.C.txt, bc1dmem/temp/sparse_matrices/barcodes.1.G.txt, bc1dmem/temp/sparse_matrices/barcodes.1.T.txt, bc1dmem/temp/sparse_matrices/barcodes.1.coverage.txt
    jobid: 2
    reason: Missing output files: bc1dmem/qc/depth/barcodes.1.depth.txt
    wildcards: sample=barcodes.1
    resources: tmpdir=/scratch/tmp

[Tue Jul  4 13:20:36 2023]
rule process_one_sample:
    input: bc1dmem/.internal/samples/GCCTAGGCAGTTCGGC-1.bam.txt
    output: bc1dmem/temp/ready_bam/GCCTAGGCAGTTCGGC-1.qc.bam, bc1dmem/temp/ready_bam/GCCTAGGCAGTTCGGC-1.qc.bam.bai, bc1dmem/qc/depth/GCCTAGGCAGTTCGGC-1.depth.txt, bc1dmem/temp/sparse_matrices/GCCTAGGCAGTTCGGC-1.A.txt, bc1dmem/temp/sparse_matrices/GCCTAGGCAGTTCGGC-1.C.txt, bc1dmem/temp/sparse_matrices/GCCTAGGCAGTTCGGC-1.G.txt, bc1dmem/temp/sparse_matrices/GCCTAGGCAGTTCGGC-1.T.txt, bc1dmem/temp/sparse_matrices/GCCTAGGCAGTTCGGC-1.coverage.txt
    jobid: 3
    reason: Missing output files: bc1dmem/qc/depth/GCCTAGGCAGTTCGGC-1.depth.txt
    wildcards: sample=GCCTAGGCAGTTCGGC-1
    resources: tmpdir=/scratch/tmp

Config file bc1dmem/.internal/parseltongue/snake.gather.yaml is extended by additional config specified via the command line.
Config file bc1dmem/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Config file bc1dmem/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Config file bc1dmem/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Building DAG of jobs...
Building DAG of jobs...
Building DAG of jobs...
Config file bc1dmem/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Config file bc1dmem/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Config file bc1dmem/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Select jobs to execute...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Using shell: /usr/bin/bash
Select jobs to execute...
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
[Tue Jul  4 13:20:39 2023]
Finished job 1.
1 of 3 steps (33%) done
Select jobs to execute...

[Tue Jul  4 13:20:39 2023]
rule make_final_sparse_matrices:
    output: bc1dmem/final/bc1.A.txt.gz, bc1dmem/final/bc1.C.txt.gz, bc1dmem/final/bc1.G.txt.gz, bc1dmem/final/bc1.T.txt.gz, bc1dmem/final/bc1.coverage.txt.gz
    jobid: 2
    reason: Missing output files: bc1dmem/final/bc1.G.txt.gz, bc1dmem/final/bc1.A.txt.gz, bc1dmem/final/bc1.C.txt.gz, bc1dmem/final/bc1.T.txt.gz, bc1dmem/final/bc1.coverage.txt.gz
    resources: tmpdir=/scratch/tmp

[W::hts_idx_load3] The index file is older than the data file: bc1dmem/temp/barcoded_bams/GCCTAGGCAGTTCGGC-1.bam.bai
[W::hts_idx_load3] The index file is older than the data file: bc1dmem/temp/barcoded_bams/CACCACTAGGAGGCGA-1.bam.bai
[W::hts_idx_load3] The index file is older than the data file: bc1dmem/temp/barcoded_bams/CTAACTTAGAGCCACA-1.bam.bai
Config file bc1dmem/.internal/parseltongue/snake.gather.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Select jobs to execute...
gzip: bc1dmem/final/bc1.A.txt: No such file or directory
gzip: bc1dmem/final/bc1.C.txt: No such file or directory
gzip: bc1dmem/final/bc1.G.txt: No such file or directory
gzip: bc1dmem/final/bc1.T.txt: No such file or directory
gzip: bc1dmem/final/bc1.coverage.txt: No such file or directory
Waiting at most 5 seconds for missing files.
Error in checkGrep(grep(".A.txt", files)) : 
  Improper folder specification; file missing / extra file present. See documentation
Calls: importMito -> checkGrep
Execution halted

I am using the following versions:
Python 3.11.4
R 4.2.0
Snakemake 7.30.1

@caleblareau
Copy link
Owner

Hi all, apologies for the delayed response-- these seem like two separate issues.

@triciapark it looks like you are erroring out in the remove duplicates step-- do you have java in your environment?

@teresa1997 your issue looks to be where the final R script is not running that collates the data. My guess is that you are missing R packages (GenomicRanges, data.table, SummarizedExperiment, Matrix)

@teresa1997
Copy link

Hi @caleblareau thanks for the answer. The packages are installed. Any other guess?

@FionaMoon
Copy link

Hi @caleblareau I also met this problem.
All required R package are installed.
My java version is

openjdk version "1.8.0_332"
OpenJDK Runtime Environment (Zulu 8.62.0.19-CA-linux64) (build 1.8.0_332-b09)
OpenJDK 64-Bit Server VM (Zulu 8.62.0.19-CA-linux64) (build 25.332-b09, mixed mode)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@caleblareau caleblareau

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@caleblareau @FionaMoon @teresa1997 @triciapark