Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[upstream]-Xcomp -XX:TieredStopAtLevel=1选项x64平台运行java/foreign/TestDowncall.java用例随机报Out of space in CodeCache for adapters #114

Open
sendaoYan opened this issue Aug 4, 2023 · 5 comments
Open

[upstream]-Xcomp -XX:TieredStopAtLevel=1选项x64平台运行java/foreign/TestDowncall.java用例随机报Out of space in CodeCache for adapters #114

sendaoYan opened this issue Aug 4, 2023 · 5 comments
Assignees
@kuaiwei

Comments

@sendaoYan
Copy link
Collaborator

t-one job：https://tone.aliyun-inc.com/ws/xesljfzh/test_result/210413?tab=1

test comand:

wget -nv https://dragonwell.oss-cn-shanghai.aliyuncs.com/test-50/OpenJDK17U-jdk_x64_linux_dragonwell_2023-07-21-07-22.tar.gz -O binary.tar.gz
wget -nv https://dragonwell.oss-cn-shanghai.aliyuncs.com/test-50/OpenJDK17U-testimage_x64_linux_dragonwell_2023-07-21-07-22.tar.gz -O test-image.tar.gz
wget -nv https://compiler-ci-bucket.oss-cn-hangzhou.aliyuncs.com/jdk/tools/jtreg-6.1.1.zip -O jtreg.zip
unzip jtreg.zip
cd jtreg 
export JT_HOME=$PWD ; export PATH=$PWD/bin:$PATH
cd -
git clone -b wip_dragonwell_standard_merge_branch "[email protected]:dragonwell-project/dragonwell17.git" jdk-repo
tar zxvf test-image.tar.gz
tar zxvf binary.tar.gz
cd dragonwell-17.0.7.0.7+7-GA
export JAVA_HOME=$PWD ; export PATH=$JAVA_HOME/bin:$PATH ; export TEST_JDK_HOME=$JAVA_HOME
cd -
cd jdk-repo
export test="-Xcomp -XX:TieredStopAtLevel=1 -nativepath:/home/yansendao.ysd/software/jdk/2023-08/dragonwell-17.0.7.0.7+7-GA-test-image/jdk/jtreg/native test/jdk/java/foreign/TestDowncall.java"
function runJtreg() { jtreg -ea -esa -timeoutFactor:4 -v:fail,error,time,nopass -nr -w $dir/index-$1 $test &> $dir/$1.log ; if [[ 0 -ne $? ]] ; then echo -n "$1 " ; else rm -rf $dir/index-$1 $dir/$1.log ; fi ; } ; export -f runJtreg ; export dir="tmp-jtreg-"`basename ${test##* } .java` ; rm -rf $dir ; mkdir -p $dir ; time seq 50 | xargs -i -n 1 -P `nproc` bash -c "runJtreg {}" ; echo total fail number: `ls $dir/*.log 2> /dev/null | wc | awk '{print $1}'`

dragonwell17复现概率：20/50

image

10.log

temurin复现概率：0/50

image

image

错误信息：

WARNING: Using incubator modules: jdk.incubator.foreign
OpenJDK 64-Bit Server VM warning: CodeCache is full. Compiler has been disabled.
OpenJDK 64-Bit Server VM warning: Try increasing the code cache size using -XX:ReservedCodeCacheSize=
java.lang.VirtualMachineError: Out of space in CodeCache for adapters
	at java.base/java.time.zone.ZoneOffsetTransition.<init>(ZoneOffsetTransition.java:175)
	at java.base/java.time.zone.ZoneRules.<init>(ZoneRules.java:300)
	at java.base/java.time.zone.ZoneRules.readExternal(ZoneRules.java:478)
	at java.base/java.time.zone.Ser.readInternal(Ser.java:196)
	at java.base/java.time.zone.Ser.read(Ser.java:189)
	at java.base/java.time.zone.TzdbZoneRulesProvider.provideRules(TzdbZoneRulesProvider.java:135)
	at java.base/java.time.zone.ZoneRulesProvider.getRules(ZoneRulesProvider.java:235)
	at java.base/java.time.ZoneRegion.ofId(ZoneRegion.java:121)
	at java.base/java.time.ZoneId.of(ZoneId.java:410)
	at java.base/java.time.ZoneId.of(ZoneId.java:358)
	at java.base/java.time.ZoneId.of(ZoneId.java:314)
	at java.base/java.util.TimeZone.toZoneId0(TimeZone.java:574)
	at java.base/java.util.TimeZone.toZoneId(TimeZone.java:551)
	at java.base/java.util.TimeZone.toZoneId0(TimeZone.java:563)
	at java.base/java.util.TimeZone.toZoneId(TimeZone.java:551)
	at java.base/sun.util.cldr.CLDRTimeZoneNameProviderImpl.getDisplayNameArray(CLDRTimeZoneNameProviderImpl.java:94)
	at java.base/sun.util.locale.provider.TimeZoneNameProviderImpl.getDisplayName(TimeZoneNameProviderImpl.java:99)
	at java.base/sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter.getName(TimeZoneNameUtility.java:271)
	at java.base/sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter.getObject(TimeZoneNameUtility.java:229)
	at java.base/sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter.getObject(TimeZoneNameUtility.java:215)
	at java.base/sun.util.locale.provider.LocaleServiceProviderPool.getLocalizedObjectImpl(LocaleServiceProviderPool.java:275)
	at java.base/sun.util.locale.provider.LocaleServiceProviderPool.getLocalizedObject(LocaleServiceProviderPool.java:236)
	at java.base/sun.util.locale.provider.TimeZoneNameUtility.retrieveDisplayNamesImpl(TimeZoneNameUtility.java:197)
	at java.base/sun.util.locale.provider.TimeZoneNameUtility.retrieveDisplayName(TimeZoneNameUtility.java:150)
	at java.base/java.util.TimeZone.getDisplayName(TimeZone.java:402)
	at java.base/java.text.SimpleDateFormat.subFormat(SimpleDateFormat.java:1306)
	at java.base/java.text.SimpleDateFormat.format(SimpleDateFormat.java:1001)
	at java.base/java.text.SimpleDateFormat.format(SimpleDateFormat.java:971)
	at java.base/java.text.DateFormat.format(DateFormat.java:339)
	at java.base/java.text.Format.format(Format.java:159)
	at org.testng.util.TimeUtils.formatTimeInLocalOrSpecifiedTimeZone(TimeUtils.java:23)
	at org.testng.reporters.XMLReporter.addDurationAttributes(XMLReporter.java:212)
	at org.testng.reporters.XMLReporter.getSuiteAttributes(XMLReporter.java:196)
	at org.testng.reporters.XMLReporter.writeSuiteToBuffer(XMLReporter.java:140)
	at org.testng.reporters.XMLReporter.writeSuite(XMLReporter.java:108)
	at org.testng.reporters.XMLReporter.generateReport(XMLReporter.java:76)
	at org.testng.TestNG.generateReports(TestNG.java:1099)
	at org.testng.TestNG.run(TestNG.java:1042)
	at com.sun.javatest.regtest.agent.TestNGRunner.main(TestNGRunner.java:94)
	at com.sun.javatest.regtest.agent.TestNGRunner.main(TestNGRunner.java:54)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
	at java.base/java.lang.Thread.run(Thread.java:833)

环境信息：

image
@sendaoYan sendaoYan mentioned this issue Aug 4, 2023
Closed
@sendaoYan sendaoYan changed the title [regress]-Xcomp -XX:TieredStopAtLevel=1选项x64平台运行test/jdk/java/foreign/TestDowncall.java用例随机报Out of space in CodeCache for adapters [regress]-Xcomp -XX:TieredStopAtLevel=1选项x64平台运行java/foreign/TestDowncall.java用例随机报Out of space in CodeCache for adapters Aug 4, 2023
@sendaoYan
Copy link
Collaborator Author

https://code.alibaba-inc.com/os-quality/tone-matrix/codereview/13542757

@sendaoYan
Copy link
Collaborator Author

vmTestbase/vm/mlvm/meth/stress/compiler/deoptimize/Test.java 存在类似的问题

@sendaoYan sendaoYan mentioned this issue Aug 4, 2023
Closed
@sendaoYan
Copy link
Collaborator Author

在同一台机器上，都是跑50次，dragonwell17跑50次出现14次fail，temurin17全部pass

image

测试命令:

export test="-jdk:/home/yansendao.ysd/software/jdk/temurin/jdk-17.0.8+7 -nativepath:/home/yansendao.ysd/software/jdk/2023-08/dragonwell-17.0.7.0.7+7-GA-test-image/jdk/jtreg/native -Xcomp -XX:TieredStopAtLevel=1 test/jdk/java/foreign/TestDowncall.java"
function runJtreg() { jtreg -ea -esa -timeoutFactor:4 -v:fail,error,time,nopass -nr -w $dir/index-$1 $test &> $dir/$1.log ; if [[ 0 -ne $? ]] ; then echo -n "$1 " ; else rm -rf $dir/index-$1 $dir/$1.log ; fi ; } ; export -f runJtreg ; export dir="tmp-jtreg-"`basename ${test##* } .java` ; rm -rf $dir ; mkdir -p $dir ; time seq 50 | xargs -i -n 1 -P `nproc` bash -c "runJtreg {}" ; echo total fail number: `ls $dir/*.log 2> /dev/null | wc | awk '{print $1}'`

@lusou-zhangquan
Copy link

用例失败场景分析

JVM打开code cache sweep日志选项-Xlog:codecache+sweep*=trace:unload_pid%p.txt，对比发现fail和pass的日志却别主要在于code cache即将满的时候是否能够顺利完成回收：
fail:

[11.868s][debug][codecache,sweep,start] CodeCache flushing
[12.176s][debug][codecache,sweep      ] CodeCache: size=49152Kb used=47667Kb max_used=47667Kb free=1485Kb
[12.177s][debug][codecache,sweep,start] CodeCache flushing
[12.537s][debug][codecache,sweep      ] CodeCache: size=49152Kb used=49020Kb max_used=49020Kb free=131Kb
[12.537s][debug][codecache,sweep,start] CodeCache flushing

pass：

[12.566s][debug][codecache,sweep,start] CodeCache flushing
[13.227s][debug][codecache,sweep      ] CodeCache: size=49152Kb used=49134Kb max_used=49136Kb free=17Kb
[13.227s][debug][codecache,sweep      ] restart compiler
[13.227s][debug][codecache,sweep,start] CodeCache flushing
> [13.405s][debug][codecache,sweep      ] CodeCache: size=49152Kb used=27058Kb max_used=49151Kb free=22093Kb
> [13.405s][debug][codecache,sweep      ] restart compiler
[13.515s][debug][codecache,sweep,start] CodeCache flushing
[13.528s][debug][codecache,sweep      ] CodeCache: size=49152Kb used=9135Kb max_used=49151Kb free=40016Kb

代码差异分析

  1. 对比dragonwell与上游openjdk的hotspot相关代码，无差异；
  2. 由于temurin无源码，比对二进制发现temurin多了一些symbol，其中包括_ZN26GrowableArrayWithAllocatorIP14CompiledMethod13GrowableArrayIS1_EE4growEi，demangle之后为GrowableArrayWithAllocator<CompiledMethod*, GrowableArray<CompiledMethod*> >::grow(int)，检索了目前jdk master相关的合入，目前没有找到相关修改；

@lusou-zhangquan
Copy link

lusou-zhangquan commented Aug 10, 2023 •
edited
Loading

本地编译上游对应tag的版本，测试发现与dragonwell行为一致，失败概率约为21/50
dongonwell17当前tag对应的git log

1e51eed4510 (origin/standard, origin/HEAD, standard) [Misc] Update version
208e346a771 Merge pull request #116 from dragonwell-project/wip_dragonwell_standard_squash_branch
8ea58af8e0f (origin/wip_dragonwell_standard_squash_branch, origin/wip_dragonwell_standard_merge_branch) Merge branch 'standard' into wip_dragonwell_standard_merge_branch
3197a9b0299 (HEAD -> openjdk-17.0.8, tag: jdk-17.0.8-ga, tag: jdk-17.0.8+7) 8311467: [17u] Remove designator DEFAULT_PROMOTED_VERSION_PRE=ea for release 17.0.8
25178cd7e27 8308682: Enhance AES performance
bc9d9c0e705 8305312: Enhanced path handling

opendjk17对应的上游tag

3197a9b0299 (HEAD -> openjdk-17.0.8, tag: jdk-17.0.8-ga, tag: jdk-17.0.8+7) 8311467: [17u] Remove designator DEFAULT_PROMOTED_VERSION_PRE=ea for release 17.0.8
25178cd7e27 8308682: Enhance AES performance
bc9d9c0e705 8305312: Enhanced path handling

openjdk17运行情况

openjdk version "17.0.8-internal" 2023-07-18
OpenJDK Runtime Environment (build 17.0.8-internal+0-adhoc..jdk17)
OpenJDK 64-Bit Server VM (build 17.0.8-internal+0-adhoc..jdk17, mixed mode, sharing)

24 26 29 32 18 44 48 2 36 45 30 17 47 19 12 50 6 5 41 37 4
real    0m43.160s
user    40m38.818s
sys     2m46.160s
total fail number: 21

@sendaoYan sendaoYan changed the title [regress]-Xcomp -XX:TieredStopAtLevel=1选项x64平台运行java/foreign/TestDowncall.java用例随机报Out of space in CodeCache for adapters [upstream]-Xcomp -XX:TieredStopAtLevel=1选项x64平台运行java/foreign/TestDowncall.java用例随机报Out of space in CodeCache for adapters Aug 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kuaiwei kuaiwei

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@kuaiwei @sendaoYan @lusou-zhangquan