feat: P4ADEV-1658 wf treasury opi ingestion file reading validating save data

[domenicogi]

Description

List of Changes

Motivation and Context

Reading files and validate them. save data

How Has This Been Tested?

• Pre-Deploy Test
  • Unit
  • Integration (Narrow)
• Post-Deploy Test
  • Isolated Microservice
  • Broader Integration
  • Acceptance
  • Performance & Load

Types of changes

• PATCH - Bug fix (backwards compatible bug fixes)
• MINOR - New feature (add functionality in a backwards compatible manner)
• MAJOR - Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.

[antonioT90]

The PII segregation is not demanded to the activity, but to the DAO!
Activity should not be aware of such persistence requirement

[domenicogi]

deleted FlussoTesoreriaPIIDao and all references

[antonioT90]

The PII segregation is not demanded to the activity, but to the DAO!
Activity should not be aware of such persistence requirement.

ps: please align with @marcomatteuccieng to check if we will store all these info, treasury should not more have PII

[domenicogi]

deleted FlussoTesoreriaPIIDTO

[antonioT90]

The PII segregation is not demanded to the activity, but to the DAO!
Activity should not be aware of such persistence requirement

[domenicogi]

deleted hash properties

[antonioT90]

let's not use assert to throw an errore if a condition fails!
assertions could be disabled, let's just throw an exception...

Anyway, version cannot be null! you could simply remove this line

[domenicogi]

removed assert

[antonioT90]

instead of having a single method inside which you will write many if/else based on the version, let's just split and move the logics on a separate service, calling it directly on the try/catch above to build the common output of both processes.

You could have 2 Services:

1. TreasuryOpiv14HandlerService
2. TreasuryOpiv161HandlerService

These services will work the related DTOs, in order to avoid classes where you are actually using Object and cast, or again a if based on the version field...

Such services could extend a common base abstract class using generics to reference the version related type.
This service will implement the Strategy design pattern (BaseTreasuryOpiHandlerService) which will contain the common algorithm to process the file:

Map<TreasuryOpEnum, List<Pair<TreasuryDTO, FlussoTesoreriaPIIDTO>>> handle(T treasuryOpiData, IngestionFlowFileDTO ingestionFlowFileDTO, int totalNumberOfTreasuryOpiFiles) {
   validateTotalNumberOfPages(treasuryOpiData, totalNumberOfTreasuryOpiFiles);
}

private void validateTotalNumberOfPages(T treasuryOpiData, int totalNumberOfTreasuryOpiFiles) {
   if(getTotalNumberOfTreasuryOpiFiles(treasuryOpiData) != totalNumberOfTreasuryOpiFiles){
      throw new TreasuryOpiInvalidFileException(...); // if you want to log also the file name, add it as input to your methods
   }
...
}

protected abstract int getTotalNumberOfTreasuryOpiFiles(T treasuryOpiData);

[domenicogi]

done refactor according to Strategy pattern

[antonioT90]

Please, split this class in order to have mappers specialized on the particular input instead of using the reflection to get data!

Reflection is a very slow method, very error-prone, to avoid if possible!

[domenicogi]

done refactor according to Strategy pattern

[antonioT90]

Please, split this into separate class specialized on the particular type instead of using reflection

[domenicogi]

done refactor according to Strategy pattern

[sonarqubecloud]

Quality Gate failed

Failed conditions
22.2% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

[antonioT90]

Use new column names

[antonioT90]

Why get(0)?

[antonioT90]

Don't do this tò retrieve dynamically classes! Spring allow to inject beans using Generic thus this class could be removed (moreover you were using @SuppressWarning which should be avoided! Using It could be and hint of bad design)

[antonioT90]

Isn't U Always the same type?

[antonioT90]

Instead of handling a List to transform into a map, let's handle a single item at time.
The mapped type should be the same for all mappers, the reduce operatori ti build the final map could be implementer once on the strategy class

[antonioT90]

The grouping Logic could be implemented once in the strategy class.
Moreover, this could be written cleaner and more optimized using collection's stream api and groupingBy collector

[antonioT90]

Is this class created for each row?
If not, you cannot use an instance variabile
If yes, probably you could avoid to create a new class for each record just to collect error data

[antonioT90]

PS: you could also create n classes for each check, implementing the same interface
Next you could inject It using List<...> type in order to let Spring collect all the validators and execute the checks in a more readable and maintanable way (easier to test because you will have 1 class per check to test)

[antonioT90]

You could use dependency injection to ask tò Spring for a Bean of type
private TreasuryValidatorService<...>
Instead of having a class to hold instances