Skip to content

Commit

Permalink
Remove unnecessary outputCommitter setting (#465)
Browse files Browse the repository at this point in the history
Signed-off-by: Andras Salamon <[email protected]>
  • Loading branch information
asalamon74 authored Sep 5, 2024
1 parent 76cddac commit fbd9d31
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 14 deletions.
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- Removed obsolete backported classes for Jackson 1.5 support ([#471](https://github.com/opensearch-project/opensearch-hadoop/pull/471))

### Fixed
- Remove unnecessary outputCommitter setting ([#465](https://github.com/opensearch-project/opensearch-hadoop/pull/465))

### Security

Expand Down Expand Up @@ -117,4 +118,4 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
[1.2.0]: https://github.com/opensearch-project/opensearch-hadoop/compare/v1.1.0...v1.2.0
[1.1.0]: https://github.com/opensearch-project/opensearch-hadoop/compare/v1.0.1...v1.1.0
[1.0.1]: https://github.com/opensearch-project/opensearch-hadoop/compare/v1.0.0...v1.0.1
[1.0.0]: https://github.com/opensearch-project/opensearch-hadoop/compare/v7.13.4...v1.0.0
[1.0.0]: https://github.com/opensearch-project/opensearch-hadoop/compare/v7.13.4...v1.0.0
Original file line number Diff line number Diff line change
Expand Up @@ -118,19 +118,6 @@ private void setUserProviderIfNotSet(Map<String, String> jobProperties) {

// NB: save the table properties in a special place but nothing else; otherwise the settings might trip on each other
private void init(TableDesc tableDesc, boolean read) {
Configuration cfg = getConf();
// NB: we can't just merge the table properties in, we need to save them per input/output otherwise clashes occur which confuse Hive

Settings settings = HadoopSettingsManager.loadFrom(cfg);
//settings.setProperty((read ? HiveConstants.INPUT_TBL_PROPERTIES : HiveConstants.OUTPUT_TBL_PROPERTIES), IOUtils.propsToString(tableDesc.getProperties()));
if (read) {
// no generic setting
}
else {
// replace the default committer when using the old API
HadoopCfgUtils.setOutputCommitterClass(cfg, OpenSearchOutputFormat.OpenSearchOutputCommitter.class.getName());
}

Assert.hasText(tableDesc.getProperties().getProperty(TABLE_LOCATION), String.format(
"no table location [%s] declared by Hive resulting in abnormal execution;", TABLE_LOCATION));
}
Expand Down

0 comments on commit fbd9d31

Please sign in to comment.