Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

main to test-marshal #5199

Merged
merged 58 commits into from
May 10, 2024
Merged
Show file tree
Hide file tree
Changes from 47 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
bed6411
Changes to allow redactions.
Apr 9, 2024
6c65901
Rearrangement of buttons.
Apr 9, 2024
e20ab81
Draft commit for dev.
Apr 26, 2024
5cddef7
Draft commit for dev.
Apr 26, 2024
ae32b08
Changes to capture LAN pages.
Apr 29, 2024
6867634
Merge pull request #5170 from bcgov/dev-DV-FOIMOD-3112
sumathi-thirumani Apr 29, 2024
3492c80
new endpoint for axisrequests pagecount
Apr 29, 2024
cb58b99
Changes to system variables naming..
Apr 29, 2024
be19f3e
updated POST API
Apr 30, 2024
55bd97b
refactored the code, removed unwanted codes
Apr 30, 2024
bf88f03
Merge branch 'dev-ST-FOIMOD-3112' into dev-DV-FOIMOD-3112
Apr 30, 2024
050aed6
Changes to integrate with cronjob.
Apr 30, 2024
c719bf9
FE tooltip update
Apr 30, 2024
db6f070
Changes to trigger cronjob.
Apr 30, 2024
efcd3fc
Merge pull request #5174 from bcgov/dev-DV-FOIMOD-3112
sumathi-thirumani Apr 30, 2024
3695e36
Merge pull request #5173 from bcgov/dev-ST-FOIMOD-3112
sumathi-thirumani Apr 30, 2024
3d610d7
Changes to include contentype in cronjob.
Apr 30, 2024
018f43d
Merge branch 'dev' into dev-ST-FOIMOD-3112
Apr 30, 2024
7e47392
Merge pull request #5175 from bcgov/dev-ST-FOIMOD-749
sumathi-thirumani Apr 30, 2024
7c911d2
Merge pull request #5165 from bcgov/dev
sumathi-thirumani Apr 30, 2024
34603aa
validation added for inputIds
May 1, 2024
2e7b890
Merge branch 'dev' into dev-DV-FOIMOD-3112
May 1, 2024
bf8789a
validation added for inputIds
May 1, 2024
f4d9df8
fixed security issue
May 1, 2024
794779a
Merge pull request #5177 from bcgov/dev-DV-FOIMOD-3112
divyav-aot May 1, 2024
e2e5802
Changes to ignore closed requests.
May 1, 2024
75fa99a
Merge pull request #5178 from bcgov/dev
divyav-aot May 1, 2024
b4bcffb
changed to refer mastertable.
May 1, 2024
29592bb
Changes to run at 10UTC.
May 1, 2024
da6530c
Merge pull request #5179 from bcgov/dev-ST-FOIMOD-3112
sumathi-thirumani May 1, 2024
7c9c345
Merge pull request #5180 from bcgov/dev
sumathi-thirumani May 1, 2024
0b2d1d3
cronjob.yaml
May 2, 2024
2c883df
Merge pull request #5182 from bcgov/dev-DV-FOIMOD-3112
divyav-aot May 2, 2024
0292cde
Merge pull request #5183 from bcgov/dev
divyav-aot May 2, 2024
127a879
orderid updated in full receipt
May 2, 2024
5bca34d
Merge pull request #5184 from bcgov/dev-DV-FOIMOD-3139
divyav-aot May 2, 2024
a632315
added commentsversion in notification manager
May 3, 2024
18126e4
Merge pull request #5186 from bcgov/dev-DV-FOIMOD-3139
divyav-aot May 3, 2024
2156687
Merge pull request #5187 from bcgov/dev
divyav-aot May 3, 2024
ef81736
bug fixed related to ministry request advancedsearch
May 6, 2024
c7da91c
bug fixed related to ministry request advancedsearch
May 6, 2024
b69403f
Merge pull request #5195 from bcgov/dev-DV-FOIMOD-3136
divyav-aot May 6, 2024
7765961
Merge pull request #5198 from bcgov/dev
divyav-aot May 6, 2024
b9b99c9
queue pagecount issue updated
May 7, 2024
538577b
pagecount changes
May 7, 2024
6ba1336
Merge pull request #5204 from bcgov/dev-DV-FOIMOD-3136
sumathi-thirumani May 7, 2024
b32a727
Merge pull request #5205 from bcgov/dev
sumathi-thirumani May 7, 2024
0d0de22
updated the lan tooltip
May 7, 2024
d5aa741
Lan Pages updated
May 7, 2024
a60942c
Merge pull request #5207 from bcgov/dev-DV-FOIMOD-3136
divyav-aot May 7, 2024
2a7e393
MCF added for LAN pages
May 8, 2024
a4ccb57
Merge pull request #5210 from bcgov/dev-DV-FOIMOD-3136
divyav-aot May 8, 2024
92a6b58
Changes to pass iaocode instead of bcgovcode.
May 8, 2024
1633075
Merge pull request #5212 from bcgov/dev-ST-FOIMOD-3136
sumathi-thirumani May 8, 2024
ee8ed49
Merge pull request #5211 from bcgov/dev
divyav-aot May 8, 2024
b8c1e43
Changes to resolve the cron error [Parse values]
May 9, 2024
863e8ef
Merge pull request #5213 from bcgov/dev-ST-FOIMOD-3136
divyav-aot May 9, 2024
beec57e
Merge pull request #5214 from bcgov/dev
sumathi-thirumani May 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ namespace MCS.FOI.AXISIntegration.DAL.UnitTests
public class RequestDAUnitTest
{

private Mock<ILogger> _mockLogger = new Mock<ILogger>();
private readonly Mock<ILogger<RequestsDA>> _mockLogger = new();

[TestMethod]
public void GetRequestTest()
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,4 @@
using MCS.FOI.AXISIntegration.DataModels;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace MCS.FOI.AXISIntegration.DAL.Interfaces
{
Expand All @@ -13,5 +8,8 @@ public interface IRequestDA

public AXISRequest GetAXISRequest(string request);

public string GetAXISRequestsPageCountString();

public string PostAXISRequestsPageCountString(string[] arrayOfRequestId);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@
{
private SqlConnection sqlConnection;

private readonly ILogger Ilogger;
private readonly ILogger<RequestsDA> Ilogger;

public static string ConnectionString;

public RequestsDA(ILogger _Ilogger)
public RequestsDA(ILogger<RequestsDA> _Ilogger)
{
Ilogger = _Ilogger;
SettingsManager.DBConnectionInitializer();
Expand Down Expand Up @@ -83,6 +83,7 @@
axisRequest.RequestDescriptionToDate = RequestsHelper.ConvertDateToString(row, "reqDescriptionToDate", "yyyy-MM-dd");
axisRequest.Ispiiredacted = true;
axisRequest.RequestPageCount = Convert.ToInt32(row["requestPageCount"]);
axisRequest.LANPageCount = Convert.ToInt32(row["lanPageCount"]);
axisRequest.SubjectCode = Convert.ToString(row["subjectCode"]);
axisRequest.IdentityVerified = Convert.ToString(row["identityVerified"]);
List<Ministry> ministryList = new()
Expand Down Expand Up @@ -122,6 +123,51 @@
return extensionList;
}

public Dictionary<string, PageCount> GetAXISRequestsPageCount(string[] arrayOfRequestId)
{
DataTable axisDataTable = GetAxisRequestsPageCount(arrayOfRequestId);

var axisRequestPageCountDict = axisDataTable.AsEnumerable()
.Where(rw => Convert.ToInt32(rw["requestPageCount"]) > 0)
.ToDictionary(
rw => Convert.ToString(rw["AXISRequestID"]),
rw => new PageCount
{
RequestPageCount = Convert.ToInt32(rw["requestPageCount"]),
LANPageCount = Convert.ToInt32(rw["lanPageCount"])
});
return axisRequestPageCountDict;

}

public Dictionary<string, PageCount> GetAXISRequestsPageCount()
{
DataTable axisDataTable = GetAxisRequestsPageCount();

var axisRequestPageCountDict = axisDataTable.AsEnumerable()
.Where(rw => Convert.ToInt32(rw["requestPageCount"]) > 0)
.ToDictionary(
rw => Convert.ToString(rw["AXISRequestID"]),
rw => new PageCount
{
RequestPageCount = Convert.ToInt32(rw["requestPageCount"]),
LANPageCount = Convert.ToInt32(rw["lanPageCount"])
});
return axisRequestPageCountDict;
}

public string PostAXISRequestsPageCountString(string[] requestIds)
{
Dictionary<string, PageCount> axisRequestPageCountList = GetAXISRequestsPageCount(requestIds);
return RequestsHelper.ConvertRequestToJSON(axisRequestPageCountList);
}

public string GetAXISRequestsPageCountString()
{
Dictionary<string, PageCount> axisRequestPageCountList = GetAXISRequestsPageCount();
return RequestsHelper.ConvertRequestToJSON(axisRequestPageCountList);
}

private DataTable GetAxisRequestData(string request)
{
ConnectionString = SettingsManager.ConnectionString;
Expand Down Expand Up @@ -152,6 +198,7 @@
sum(distinct case when requests.IREQUESTID = reviewlog.IREQUESTID and reviewlog.IDOCID = documents.IDOCID then documents.SIPAGECOUNT
when requests.IREQUESTID = redaction.IREQUESTID and redaction.IDOCID = ldocuments.IDOCID then ldocuments.SIPAGECOUNT
else 0 end) as requestPageCount,
(case when requestfields.CustomField91 > 0 then requestfields.CustomField91 else 0 end ) as lanPageCount,
REPLACE(requestfields.CUSTOMFIELD33, CHAR(160), ' ') as subjectCode,
requestfields.CUSTOMFIELD75 as identityVerified,
(SELECT TOP 1 cfr.sdtDueDate FROM tblRequestForDocuments cfr WITH (NOLOCK)
Expand Down Expand Up @@ -185,7 +232,8 @@
requesters.vcAddress1, requesters.vcAddress2, requesters.vcCity, requesters.vcZipCode,
requesters.vcHome, requesters.vcMobile, requesters.vcWork1, requesters.vcWork2, requesters.vcFirstName, requesters.vcLastName, requesters.vcMiddleName,
requests.iRequestID, requesters.vcCompany, requesters.vcEmailID, onbehalf.vcFirstName, onbehalf.vcLastName, onbehalf.vcMiddleName,
requestTypes.iLabelID, requests.vcVisibleRequestID, requests.tiOfficeID, office.OFFICE_ID,requestorfields.CUSTOMFIELD35, REPLACE(requestfields.CUSTOMFIELD33, CHAR(160), ' '),requestfields.CUSTOMFIELD75";
requestTypes.iLabelID, requests.vcVisibleRequestID, requests.tiOfficeID, office.OFFICE_ID,requestorfields.CUSTOMFIELD35,
REPLACE(requestfields.CUSTOMFIELD33, CHAR(160), ' '),requestfields.CUSTOMFIELD75, requestfields.CustomField91";
DataTable dataTable = new();
using (sqlConnection = new SqlConnection(ConnectionString))
{
Expand Down Expand Up @@ -299,5 +347,87 @@

}

private DataTable GetAxisRequestsPageCount(string[] arrayOfRequestId)
{
ConnectionString = SettingsManager.ConnectionString;
var inClauseValues = RequestsHelper.GetInClause(arrayOfRequestId);

string query = $@"Select vcVisibleRequestID as axisRequestId, sum(distinct case when requests.IREQUESTID = reviewlog.IREQUESTID and reviewlog.IDOCID = documents.IDOCID then documents.SIPAGECOUNT
when requests.IREQUESTID = redaction.IREQUESTID and redaction.IDOCID = ldocuments.IDOCID then ldocuments.SIPAGECOUNT
else 0 end) as requestPageCount,
(case when requestfields.CustomField91 > 0 then requestfields.CustomField91 else 0 end ) as lanPageCount
FROM
tblRequests requests WITH (NOLOCK)
LEFT OUTER JOIN dbo.TBLdocumentreviewlog reviewlog WITH (NOLOCK) ON requests.IREQUESTID = reviewlog.IREQUESTID
LEFT OUTER JOIN dbo.TBLDOCUMENTS documents WITH (NOLOCK) ON reviewlog.IDOCID = documents.IDOCID
LEFT OUTER JOIN dbo.TBLRedactionlayers redaction WITH (NOLOCK) ON requests.IREQUESTID = redaction.IREQUESTID
LEFT OUTER JOIN dbo.TBLDOCUMENTS ldocuments WITH (NOLOCK) ON redaction.IDOCID = ldocuments.IDOCID
LEFT OUTER JOIN dbo.TBLREQUESTCUSTOMFIELDS requestfields WITH (NOLOCK) ON requests.iRequestID = requestfields.iRequestID
WHERE vcVisibleRequestID IN ({inClauseValues})
GROUP BY vcVisibleRequestID, requestfields.CustomField91";

DataTable dataTable = new();
using (sqlConnection = new SqlConnection(ConnectionString))
{
using SqlDataAdapter sqlSelectCommand = new(query, sqlConnection);
divyav-aot marked this conversation as resolved.
Show resolved Hide resolved
try
{
sqlConnection.Open();
sqlSelectCommand.Fill(dataTable);
}
catch (SqlException ex)
{
Ilogger.Log(LogLevel.Error, ex.Message);
throw;
}
catch (Exception e)
{
Ilogger.Log(LogLevel.Error, e.Message);
throw;
}
}
return dataTable;
}

private DataTable GetAxisRequestsPageCount()
{
ConnectionString = SettingsManager.ConnectionString;

string query = @"Select vcVisibleRequestID as axisRequestId, sum(distinct case when requests.IREQUESTID = reviewlog.IREQUESTID and reviewlog.IDOCID = documents.IDOCID then documents.SIPAGECOUNT
when requests.IREQUESTID = redaction.IREQUESTID and redaction.IDOCID = ldocuments.IDOCID then ldocuments.SIPAGECOUNT
else 0 end) as requestPageCount,
(case when requestfields.CustomField91 > 0 then requestfields.CustomField91 else 0 end ) as lanPageCount
FROM
tblRequests requests WITH (NOLOCK)
LEFT OUTER JOIN dbo.TBLdocumentreviewlog reviewlog WITH (NOLOCK) ON requests.IREQUESTID = reviewlog.IREQUESTID
LEFT OUTER JOIN dbo.TBLDOCUMENTS documents WITH (NOLOCK) ON reviewlog.IDOCID = documents.IDOCID
LEFT OUTER JOIN dbo.TBLRedactionlayers redaction WITH (NOLOCK) ON requests.IREQUESTID = redaction.IREQUESTID
LEFT OUTER JOIN dbo.TBLDOCUMENTS ldocuments WITH (NOLOCK) ON redaction.IDOCID = ldocuments.IDOCID
LEFT OUTER JOIN dbo.TBLREQUESTCUSTOMFIELDS requestfields WITH (NOLOCK) ON requests.iRequestID = requestfields.iRequestID
GROUP BY vcVisibleRequestID, requestfields.CustomField91";

DataTable dataTable = new();
using (sqlConnection = new SqlConnection(ConnectionString))
{
using SqlDataAdapter sqlSelectCommand = new(query, sqlConnection);
try
{
sqlConnection.Open();
sqlSelectCommand.Fill(dataTable);
}
catch (SqlException ex)
{
Ilogger.Log(LogLevel.Error, ex.Message);
throw;
}
catch (Exception e)
{
Ilogger.Log(LogLevel.Error, e.Message);
throw;
}
}
return dataTable;
}

}
}
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
using System;
using System.Data;
using System.Globalization;
using System.Linq;

namespace MCS.FOI.AXISIntegration.DAL
{
Expand All @@ -13,11 +14,12 @@ public static string ConvertDateToString(DataRow row, string property, string fo
return row[property] == DBNull.Value ? null : Convert.ToDateTime(row[property]).ToString(format, CultureInfo.InvariantCulture);
}

public static string ConvertRequestToJSON(AXISRequest request)
public static string ConvertRequestToJSON(Object _object)
{
return JsonConvert.SerializeObject(request);
return JsonConvert.SerializeObject(_object);
}


public static string GetRequestType(string requestType)
{
if (requestType.ToLower().Contains(RequestTypes.General.ToString().ToLower()))
Expand Down Expand Up @@ -138,5 +140,11 @@ public static string GetDeliveryMode(string deliveryMode)
_ => "Secure File Transfer",
};
}

public static string GetInClause(string[] values)
{
// Join the array values with commas and enclose each value in single quotes
return string.Join(",", values.Select(val => $"'{val}'"));
}
}
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
using System;
using System.Collections.Generic;
using System.Collections.Generic;
using System.Runtime.Serialization;

namespace MCS.FOI.AXISIntegration.DataModels
Expand Down Expand Up @@ -120,6 +119,9 @@ public class AXISRequest
[DataMember(Name = "requestPageCount")]
public int RequestPageCount { get; set; }

[DataMember(Name = "lanPageCount")]
public int LANPageCount { get; set; }

[DataMember(Name = "subjectCode")]
public string SubjectCode { get; set; }

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
using System.Runtime.Serialization;

namespace MCS.FOI.AXISIntegration.DataModels
{

[DataContract]
public class AXISRequestPageCount
{
[DataMember(Name = "id")]
public string AXISRequestID { get; set; }

[DataMember]
public PageCount PageCountInfo { get; set; }
}

[DataContract]
public class PageCount
{
[DataMember(Name = "requestpagepount")]
public int RequestPageCount { get; set; }

[DataMember(Name = "lanpagepount")]
public int LANPageCount { get; set; }
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,6 @@
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Linq;

namespace MCS.FOI.AXISIntegrationWebAPI.Controllers
{
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
using MCS.FOI.AXISIntegration.DAL.Interfaces;
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Cors;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging;
using System;
using System.Collections.Generic;
using System.Text.RegularExpressions;

namespace MCS.FOI.AXISIntegrationWebAPI.Controllers
{
[Route("api/[controller]")]
[ApiController]
[EnableCors(PolicyName = "FOIOrigins")]
[Authorize]
public class RequestsPageCountController : ControllerBase
{

private readonly ILogger<RequestsPageCountController> _logger;
private readonly IRequestDA _requestDA;
TimeSpan timeout = TimeSpan.FromSeconds(3);

public RequestsPageCountController(ILogger<RequestsPageCountController> logger, IRequestDA requestDA)
{
_logger = logger;
_requestDA = requestDA;
}

[HttpPost]

public ActionResult<string> Post([FromBody] List<string> axisRequestIds)
{
try
{
if (axisRequestIds == null || axisRequestIds.Count == 0)
{
return BadRequest("The list of axisRequestIds is required.");
}
foreach (var requestId in axisRequestIds)
{
if (!IsValidRequestId(requestId))
{
return BadRequest($"Invalid axisRequestId: {requestId}");
}

if (ContainsSqlInjectionPattern(requestId))
{
return BadRequest($"Potential SQL injection detected in axisRequestId: {requestId}");
}
}
return _requestDA.PostAXISRequestsPageCountString(axisRequestIds.ToArray());

}
catch (Exception ex)
{
_logger.Log(LogLevel.Error, string.Format($"Exception happened on RequestspageCount POST operations, Error Message : {ex.Message} , Stack Trace :{ex.StackTrace}"));
return string.Format($"Exception happened on RequestspageCount POST operations, Error Message : {ex.Message}");
}

}
private bool IsValidRequestId(string requestId)
{
// RequestID should have letters-numbers-numbers format
string pattern = @"^[A-Za-z]+(?:[-]){0,2}\d+\-\d+$";
return Regex.IsMatch(requestId, pattern, RegexOptions.None, timeout);
}

private bool ContainsSqlInjectionPattern(string input)
{
string pattern = @"(?:\b(?:SELECT|INSERT|UPDATE|DELETE|DROP|ALTER)\b|\b(?:UNION\s+ALL|SELECT\s+.*?\s+FROM\s+.*?\s+WHERE\s+.*?))";
return Regex.IsMatch(input, pattern, RegexOptions.IgnoreCase, timeout);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ public void ConfigureServices(IServiceCollection services)
string[] _origins = SettingsManager.CORSORIGINS.Split(",");
services.AddCors(options =>
options.AddPolicy("FOIOrigins", p => p.WithOrigins(_origins)
.WithMethods("GET")
.WithMethods("GET", "POST")
.AllowAnyHeader()
.AllowAnyMethod()
)) ;
Expand All @@ -45,6 +45,8 @@ public void ConfigureServices(IServiceCollection services)
var serviceProvider = services.BuildServiceProvider();
var requestlogger = serviceProvider.GetService<ILogger<RequestSearchController>>();
services.AddSingleton(typeof(ILogger), requestlogger);
var requestpagecountlogger = serviceProvider.GetService<ILogger<RequestsPageCountController>>();
services.AddSingleton(typeof(ILogger), requestpagecountlogger);
services.AddScoped<IRequestDA, RequestsDA>();
services.AddScoped<IFOIFlowRequestUserDA,FOIFlowRequestUsersDA>();

Expand Down
2 changes: 2 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,8 @@ services:
- UNOPENED_REPORT_WAIT_DAYS=${UNOPENED_REPORT_WAIT_DAYS}
- UNOPENED_REPORT_JARO_CUTOFF=${UNOPENED_REPORT_JARO_CUTOFF}
- UNOPENED_REPORT_EMAIL_RECIPIENT=${UNOPENED_REPORT_EMAIL_RECIPIENT}
- AXIS_API_URL=${AXIS_API_URL}
- AXIS_SYNC_BATCHSIZE=${AXIS_SYNC_BATCHSIZE}
#- LOG_ROOT=${LOG_ROOT}
#- LOG_BASIC=${LOG_BASIC}
#- LOG_TRACING=${LOG_TRACING}
Expand Down
Loading
Loading