Add ml xgboost workload #638
Conversation
|
@bobjiang82 #628 is merged. could you rebase the code to resolve the conflict? |
|
@xwu99 Done. |
docs/run-sparkbench.md
Outdated
|
|
||
|
|
||
| ### 8. Run xgboost workload ### | ||
|
|
There was a problem hiding this comment.
Could you change xgboost to XGBoost and following the same?
docs/run-sparkbench.md
Outdated
| ``` | ||
|
|
||
| #### 8.a latest xgboost release (default) #### | ||
|
|
There was a problem hiding this comment.
don't need to use 8.a, 8.b., need to use correct captial cases for titles.
There was a problem hiding this comment.
I think you don't need to write this since it's already written in the above section 4. Run a workload
I suggest you seperate the doc out and only merge code and make sure it's runnable with default HiBench process.
docs/run-sparkbench.md
Outdated
| If you only have the xgboost jar files, just copy them to $SPARK_HOME/jars/ and update the relevant versions for xgboost4j and xgboost4j-spark in sparkbench/ml/pom.xml to get aligned.<br> | ||
| For example, if xgboost is built from source on a Linux platform, the jars will be generated and installed to ```~/.m2/repository/ml/dmlc/xgboost4j_<scala version>/<xgboost version>-SNAPSHOT/``` and ```~/.m2/repository/ml/dmlc/xgboost4j-spark_<scala version>/<xgboost version>-SNAPSHOT/``` respectively. To use them, copy the 2 jars to $SPARK_HOME/jars/ and update the relevant versions for xgboost4j and xgboost4j-spark in the pom.xml files.<br> | ||
| After that, build hibench, prepare data and run xgboost benchmark. | ||
|
|
There was a problem hiding this comment.
Generally, the doc style is not consistent as the original doc. and too complicated to follow.
I suggest rewrite or remove. We can merge code first. It should be runnable with default setting.
docs/run-sparkbench.md
Outdated
| ``` | ||
|
|
||
| #### 8.a latest xgboost release (default) #### | ||
|
|
There was a problem hiding this comment.
I think you don't need to write this since it's already written in the above section 4. Run a workload
I suggest you seperate the doc out and only merge code and make sure it's runnable with default HiBench process.
commit code first and continue to refine doc.
|
Updated to merge the code first and continue to refine the doc. |
Thanks! could you add this to CI
Thanks, could you add this to |
|
Added xgboost to conf/benchmarks.lst and travis/benchmarks_ml.lst |
|
@bobjiang82 could you modify bin/run_all.sh to mask out hadoop since this is for spark only. |
sync the forked repo with HiBench base
Note: based on Xiaochang's PR #628.