v2.0.5

What's Changed

• Fix #29 by @DoodleBears in #30

Full Changelog: v2.0.4...v2.0.5

v2.0.4

What's Changed

• Fix #27 by @DoodleBears in #28

Full Changelog: v2.0.3...v2.0.4

v2.0.3

What's Changed

• Fix #25 by @DoodleBears in #26

Full Changelog: v2.0.2...v2.0.3

v2.0.2

Summary

• fix(splitter): substring inside section merging error

Full Changelog: v2.0.1...v2.0.2

v2.0.1

Summary

• fix(splitter): when merge across punctuation on section stage the lang of substring inside is wrong
• fix(splitter): when merge across digit on section stage the substring inside section is not merged

Full Changelog: v2.0.0...v2.0.1

v2.0.0

Summary

• Improve stability when dealing with long and short text (by separate the splitting logic based on language SET {ZH_JA, KO, OTHERS}
• add parameter for whether to merge over, (default is True, when set to False, this part will been separated in result substring)
  • newline
  • digit
  • punctuation (allow to set no need to merge punctuation List[str] )

What's Changed

• Close #23 by @DoodleBears in #24

Full Changelog: v1.4.1...v2.0.0

v1.4.1

Full Changelog: v1.4.0...v1.4.1

v1.3.9

What's Changed

• Fix #18 by @DoodleBears in #19

Full Changelog: v1.3.8...v1.3.9

v1.3.8

What's Changed

• Fix #16 by @DoodleBears in #17

Full Changelog: v1.3.7...v1.3.8

v1.3.7

Summary

• fix(detector): avoid divide by zero since word frequency can be zero (if text does not exist in that language)

Full Changelog: v1.3.6...v1.3.7