Crash with Illegal Instruction under Rosetta2 emulation in remote container

[Thomas1664]

Type: Bug

I'm running a x64 container on my M2 MacBook.
The entire container seems to randomly crash with Illegal Instruction:

2024-12-22 15:38:44 Container started
2024-12-22 16:27:41 Container started
2024-12-22 16:46:43 Container started
2024-12-22 16:27:27 Illegal instruction
2024-12-22 16:46:32 Illegal instruction
2024-12-22 17:09:36 Illegal instruction
2024-12-22 17:47:34 Illegal instruction
2024-12-22 17:49:43 Illegal instruction
2024-12-22 18:05:51 Illegal instruction
2024-12-22 18:37:52 Illegal instruction
2024-12-22 19:11:42 Illegal instruction
2024-12-22 17:15:07 Container started
2024-12-22 17:45:01 Container started
2024-12-22 17:47:53 Container started
2024-12-22 17:50:00 Container started
2024-12-22 17:53:28 Container started
2024-12-22 18:06:40 Container started
2024-12-22 18:23:54 Container started
2024-12-22 18:37:08 Container started
2024-12-22 18:38:00 Container started
2024-12-22 18:46:27 Container started
2024-12-22 19:40:52 Container started

This usually means something is trying to use AVX2 instructions which can't be emulated by Rosetta.

The crashes seem to be somehow related to Jupyter Notebooks: The crashes seem to occur more frequently when I search for a Jupyter Kernel.
Sometimes the container also crashes when it is idle for 1h.

I don't know if the issue is related to an extension or not because the remote extension host also seems to randomly crash, but it doesn't take the entire container down.

I didn't experience crashes when running the container without vscode.

I also found #214630 but I'm not sure if this is related because the extension host starts most of the time without any crashes.

VS Code version: Code 1.96.2 (Universal) (fabdb6a, 2024-12-19T10:22:47.216Z)
OS version: Darwin arm64 24.2.0
Modes:
Remote OS version: Linux x64 6.10.14-linuxkit

System Info

Item	Value
CPUs	Apple M2 (8 x 2400)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled webgl: enabled webgl2: enabled webgpu: enabled webnn: disabled_off
Load (avg)	5, 4, 3
Memory (System)	8.00GB (0.08GB free)
Process Argv
Screen Reader	no
VM	0%

Item	Value
Remote	Entwicklungscontainer: Anaconda (Python 3) @ desktop-linux
OS	Linux x64 6.10.14-linuxkit
CPUs	unknown (8 x 0)
Memory (System)	3.83GB (2.33GB free)
VM	0%

Extensions (11)

Extension	Author (truncated)	Version
jupyter-keymap	ms-	1.1.2
remote-containers	ms-	0.394.0
vscode-language-pack-de	MS-	1.96.2024121109
debugpy	ms-	2024.14.0
python	ms-	2024.22.0
vscode-pylance	ms-	2024.12.1
jupyter	ms-	2024.11.0
jupyter-keymap	ms-	1.1.2
jupyter-renderers	ms-	1.0.21
vscode-jupyter-cell-tags	ms-	0.1.9
vscode-jupyter-slideshow	ms-	0.1.6

[Thomas1664]

For some reason I can't get any core dumps following these steps.

 $ coredumpctl list
No journal files were found.
No coredumps found.

[deepak1556]

For some reason I can't get any core dumps following

Can you confirm if the ulimit values are configured correctly in the container ?

If you have a sample container and some steps to repro, it would be helpful.

[Thomas1664]

Can you confirm if the ulimit values are configured correctly in the container ?

The issue was that I couldn't configure ulimit correctly but I finally got it to work by setting "runArgs": ["--init", "--ulimit", "core=-1"] in devcontainer.json.

I will provide the core dump once I get another crash.

If you have a sample container and some steps to repro, it would be helpful.

I think the meaningful parts of the Dockerfile are

FROM mcr.microsoft.com/devcontainers/anaconda:1-3

SHELL ["/bin/bash", "-c"]

RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
    && apt-get -y install --no-install-recommends software-properties-common screen cmake autoconf libxslt-dev \
    libfontconfig1-dev pkg-config liblapack-dev libblas-dev gfortran libxml2-dev liblzma-dev libbz2-dev libjpeg-dev \
    libexpat1-dev libxml-parser-perl ghostscript

# Install R from CRAN
RUN gpg --keyserver hkp://keyserver.ubuntu.com:80 \
    --recv-key '95C0FAF38DB3CCAD0C080A7BDC78B2DDEABC47B7' \
    && gpg --armor --export '95C0FAF38DB3CCAD0C080A7BDC78B2DDEABC47B7' | \
    tee /etc/apt/trusted.gpg.d/cran_debian_key.asc \
    && add-apt-repository 'deb http://cloud.r-project.org/bin/linux/debian bookworm-cran40/' \
    && apt-get update && apt-get -y install --no-install-recommends r-base

USER vscode
RUN conda init bash && source ~/.bashrc && source activate base

I found that using the Jupyter Notebooks and the Python extensions increases the chances for a crash (although I didn't try it for long without extensions).

I think to trigger a crash it should be enough to build the container as x86 on a device with an Apple chip, open a folder with Jupyter Notebooks and Python code (not sure if strictly necessary), select a Kernel for the Notebook and run the Notebook, and let vscode run in the background for a couple of hours.

[Thomas1664]

Can you confirm if the ulimit values are configured correctly in the container ?

@deepak1556 Now ulimit is configured correctly:

$ ulimit -c
unlimited
$ 

(from the Docker terminal)

But coredumpctl still can't find any core dumps in the container. The output of cat /proc/sys/kernel/core_pattern is core. I don't know where I can find this location.