CI: remove some inter-job dependencies, run cross-compile task with make -j, use /tmp for Go build cache

CI: remove some inter-job dependencies, run cross-compile task with make -j, use /tmp for Go build cache #5244