Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feat/#228] 아티클 목록 조회(무한스크롤) API 추가 #259

Merged
merged 46 commits into from
Jul 31, 2024

Conversation

hun-ca
Copy link
Member

@hun-ca hun-ca commented Jul 27, 2024

🎫 연관 이슈

resolved #228

💁‍♂️ PR 내용

  • 컨텐츠는 조합된 테이블(article_main_card)에 두지 않습니다 (article_ifo 에서 각각 조회)
    • 컨텐츠 용량이 크다는 점 고려
    • 추후 캐싱 적용 예정
  • 아티클 뷰가 같을 경우 최신 아티클이 더 우선순위가 높도록 반영

🙏 작업

관련 사전작업 PR
#251 #223 #216

🙈 PR 참고 사항

ARTICLE_VIEW_COUNT 테이블에 기존엔 아티클이 첫번째 조회가 되어야 row가 생성됐지만 아티클이 생성되기만 하면 뷰를 0으로 하여 인서트 하도록 수정.
해당 변경 사항에 대한 마이그레이션 SQL

INSERT INTO ARTICLE_VIEW_COUNT (article_id, view_count, category_cd)
SELECT id AS article_id, 0 AS view_count, category_cd
FROM ARTICLE_MST
WHERE id NOT IN (SELECT article_id FROM ARTICLE_VIEW_COUNT);

📸 스크린샷

아티클 목록 조회 (카테고리 전체)

스크린샷 2024-07-30 오전 12 13 50(2)

아티클 목록 조회 (카테고리 코드 10 - IT)

스크린샷 2024-07-30 오전 1 29 45(2)

prevArticleId 테스트

조회수 상황이 아래와 같을 때

image

/api/v1/articles?prevArticleId=1를 호출할 경우

응답에서 articleId 순서가 다음과 같음 (캡쳐로 올리기엔 너무 길어서 텍스트로 씁니다)

  • 107 - 104 - 102 - 8 - 7 - 6 - 5 - 4 - 3 - 2
    조회수가 같은 경우 articleId가 큰(최신글) 것을 더 위로 올리도록 조회됨

categoryCd & prevArticleId 테스트

조회수 상황은 위 'prevArticleId 테스트'와 동일한 상태일 경우

/api/v1/articles?prevArticleId=1&categoryCd=10를 호출하면

응답에서 articleId 순서는 다음과 같음

  • 107 - 104 - 106 - 105 - 103 - 102
    카테고리가 10(IT)인 것들만 조회수 기반 정렬 + 같은 경우 최신 아티클이 위에 가도록 정상 조회 확인

카테고리 조회

image

🤖 테스트 체크리스트

  • 체크 미완료
  • 체크 완료

@hun-ca hun-ca added the feature 새로운 기능을 만들 때 사용됩니다 label Jul 27, 2024
@hun-ca hun-ca self-assigned this Jul 27, 2024
@hun-ca hun-ca requested a review from belljun3395 as a code owner July 27, 2024 14:34
Comment on lines +99 to +131
private fun updateAndSortArticleViews(
articleRecords: Set<ArticleMainCardRecord>,
articleViewsRecords: Set<SelectArticleViewsRecord>,
): Set<ArticleMainCardRecord> {
val sortedSet = TreeSet(
Comparator<ArticleMainCardRecord> { a1, a2 ->
// views 값이 null일 경우 0으로 간주
val views1 = a1.views ?: 0
val views2 = a2.views ?: 0

// views 내림차순 정렬
val viewComparison = views2.compareTo(views1)

if (viewComparison != 0) {
viewComparison
} else {
// views가 같을 경우 articleId 내림차순 정렬(최신글)
val articleId1 = a1.articleId
val articleId2 = a2.articleId
articleId2.compareTo(articleId1)
}
}
)

val viewsMap = articleViewsRecords.associateBy({ it.articleId }, { it.views })

articleRecords.forEach { article ->
val updatedViews = viewsMap[article.articleId] ?: 0
article.views = updatedViews
sortedSet.add(article)
}

return sortedSet
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

참고로 아티클 조회수 별로 정렬하는 코드입니다. 조회수가 같을 경우엔 아티클 ID가 높은걸 우선순위 높게 했어요(최신 아티클이 먼저 보이도록)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

10개를 가지고 정렬하는거라 어떤 식으로 정렬하든 성능은 비슷할걸로 생각됩니다

Comment on lines 66 to 99
fun insertArticleMainCardsBulk(commands: Set<ArticleMainCardRecord>) {
dslContext.batch(
commands.map { command -> insertArticleMainCardsBulkQuery(command) }
).execute()
}

fun insertArticleMainCardsBulkQuery(command: ArticleMainCardRecord) =
dslContext.insertInto(
ARTICLE_MAIN_CARD,
ARTICLE_MAIN_CARD.ID,
ARTICLE_MAIN_CARD.TITLE,
ARTICLE_MAIN_CARD.MAIN_IMAGE_URL,
ARTICLE_MAIN_CARD.CATEGORY_CD,
ARTICLE_MAIN_CARD.CREATED_AT,
ARTICLE_MAIN_CARD.WRITER_ID,
ARTICLE_MAIN_CARD.WRITER_EMAIL,
ARTICLE_MAIN_CARD.WRITER_DESCRIPTION
).values(
command.articleId,
command.articleTitle,
command.mainImageUrl.toString(),
command.categoryCd,
command.createdAt,
command.writerId,
command.writerEmail,
JSON.valueOf(
commonJsonMapper.toJsonStr(
mapOf(
"name" to command.writerName,
"url" to command.writerImgUrl
)
)
)
)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

이렇게 배치 인서트로 하면 실행계획 뽑을때 단일 row 인서트만 실행계획으로 뽑히는 약간의 단점?이 있을거 같네요 values에 한번에 모든 row 넣을 수 있는 방법 생각해봐야겠습니다..

Copy link
Collaborator

@belljun3395 belljun3395 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UC 관련 리뷰 우선 추가합니다!

// 결과를 MainCard 테이블에 저장
articleMainCardDao.insertArticleMainCardsBulk(joinedArticleMainCardRecords) // TODO: 트랜잭션 분리 점검

existInArticleMainCardRecords = (existInArticleMainCardRecords + joinedArticleMainCardRecords).toSet()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

existInArticleMainCardRecords에 재할당 하는 것 보다
51번줄에서 articleMainCardRecords 변수를 선언하고 existInArticleMainCardRecords를 넣어둔 다음에 joinedArticleMainCardRecords도 거기다 추가하는 것은 어떤가요?

existInArticleMainCardRecords가 아래에서도 사용되는데 개인적으로 저는 조금 어색함이 있는 것 같아요

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

이슈 등록 #264

// 아티클 컨텐츠 조회
val selectArticleContentsRecords: List<SelectArticleContentsRecord> =
articleDao.selectArticleContents(existInArticleMainCardRecords.map { it.articleId }.toSet())
setContentsToRecords(selectArticleContentsRecords, existInArticleMainCardRecords)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

저는 setContentsToRecords와 같이 매핑하는 작업을 한 이후에는 새로운 객체를 반환하는 것을 선호하는데 혹시 contentMappedMainCardRecords와 같이 새로운 객체를 반환 받아서 이후 과정을 진행하는 것은 어떨까요?

제가 새로운 객체를 반환 받는 것을 선호하는 것은 그렇게 새로운 객체를 반환 받았을 때가 디버깅 과정이 조금 더 쉬웠고 조금 더 객체지향적인 코드를 작성할 수 있었던 것 같아요

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

넵 결국 같은 레퍼런스 참조하는 변수 리턴하는게 낫다는 말씀이시죠? Val 타입으로 받을 수도 있고 좋겠네요

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

이슈 등록 #264

articleDao.selectArticleContents(existInArticleMainCardRecords.map { it.articleId }.toSet())
setContentsToRecords(selectArticleContentsRecords, existInArticleMainCardRecords)

val sortedArticles = updateAndSortArticleViews(existInArticleMainCardRecords, articleViewsRecords)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

요기서 객체를 반환하는거면 setContentsToRecords도 통일하는 것도..?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

여기선 아에 레퍼런스가 바뀔 수 밖에 없어서(내부적으로 TreeSet 생성) 리턴한거긴 한데 통일해서 수정 해야갰네요

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

이슈 등록 #264

Comment on lines +74 to +76
setContentsToRecords(selectArticleContentsRecords, existInArticleMainCardRecords)

val sortedArticles = updateAndSortArticleViews(existInArticleMainCardRecords, articleViewsRecords)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

혹은 existInArticleMainCardRecords, selectArticleContentsRecords, articleViewsRecords를 멤버로 가지는 하나의 도메인 모델을 만들면 좋을 것 같아요!
ArticleMainCardsModel 같은?

왜냐하면 간단한 매핑이나 정렬이면 모르겠는데 지금처럼 해당 과정이 복잡하고 길다면 그 과정을 UC에서 구지 알필요는 없을 것 같아서요.

조금 더 자세히 설명해 보면 저는 ArticleMainCardsModel를 existInArticleMainCardRecords, selectArticleContentsRecords, articleViewsRecords와 함께 생성하며 우선 해당 정보들을 매핑하는 작업까지 할 것 같고
이후 sort(정렬전략) 메서드를 만들어서 해당 정보들을 정렬할 것 같아요.
이렇게 리펙을 하면 지금은 메인 화면에 정렬전략이 하나지만 추후 늘어났을 때도 대응이 편할 것 같아요.

model = if( 요청 정렬 전략 == 전략 1 ) {
        model.sort(전략1) 
    } else if (요청 정렬 전략 == 전략 2 ) {
        model.sort(전략2)
}

요런 식으로요

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

굳 이거 저도 생각했었는데 일단 빠르게 처내느라 바로 올려버림.. 추가로 이런 uc야 말로 유닛테스트가 의미있을거 같아요

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

이거 일단 이슈 등록하고 타 PR로 반영하겠습니다

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

이슈 등록 #264

Comment on lines -11 to 27
@Transactional(readOnly = true)
@Transactional // TODO: read only로 할 수 있도록 점검
fun execute(useCaseIn: ReadArticlesUseCaseIn): ReadArticlesUseCaseOut {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

어? 제가 알기로는 트랜잭션도 AOP를 사용해서 별도의 클래스로 빼서 사용해야한다고 알고 있는데 아닌가요?
메서드로 빼면 프록시 내부 호출이라 적용이 안될 것 같은데..?

Copy link
Member Author

@hun-ca hun-ca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

리뷰 후 변경사항만 코맨트로 남겨드림

@@ -1,7 +1,7 @@
spring:
datasource:
hikari:
jdbcUrl: ${DB_HOSTNAME}/api?allowPublicKeyRetrieval=true&rewriteBatchedStatements=true
jdbcUrl: ${DB_HOSTNAME}/api?allowPublicKeyRetrieval=true&rewriteBatchedStatements=true&allowMultiQueries=true
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

allowMultiQueries=true 옵션은 총5개 파일에 추가됐습니다.

Comment on lines +14 to +36
@Component
class ArticleMainCardMapper(
private val objectMapper: ObjectMapper,
) : RecordMapper<Record, ArticleMainCardRecord> {
override fun map(record: Record): ArticleMainCardRecord {
val workbooksJsonArrayStr: String = record.get(ArticleMainCardRecord::workbooks.name, JSON::class.java).data()

return ArticleMainCardRecord(
articleId = record.get(ArticleMainCardRecord::articleId.name, Long::class.java),
articleTitle = record.get(ArticleMainCardRecord::articleTitle.name, String::class.java),
mainImageUrl = record.get(ArticleMainCardRecord::mainImageUrl.name, URL::class.java),
categoryCd = record.get(ArticleMainCardRecord::categoryCd.name, Byte::class.java),
createdAt = record.get(ArticleMainCardRecord::createdAt.name, LocalDateTime::class.java),
writerId = record.get(ArticleMainCardRecord::writerId.name, Long::class.java),
writerEmail = record.get(ArticleMainCardRecord::writerEmail.name, String::class.java),
writerName = record.get(ArticleMainCardRecord::writerName.name, String::class.java),
writerImgUrl = record.get(ArticleMainCardRecord::writerImgUrl.name, URL::class.java),
workbooks = objectMapper.readValue<List<WorkbookRecord>>(workbooksJsonArrayStr)
)
}

fun toJsonStr(workbooks: List<WorkbookRecord>) = objectMapper.writeValueAsString(workbooks)
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

쿼리 결과 <-> ArticleMainCardRecord 클래스 변환 담당 mapper

  • json 다루는 필드들이 많아서 fetchInto에 클래스로 지정하지 않고 매퍼 타도록 구현

Comment on lines +56 to +68
fun selectRankByViewsQuery(query: SelectRankByViewsQuery) = dslContext
.select(field("row_rank_tb.offset", Long::class.java))
.from(
dslContext
.select(
ARTICLE_VIEW_COUNT.ARTICLE_ID,
rowNumber().over(orderBy(ARTICLE_VIEW_COUNT.VIEW_COUNT.desc())).`as`("offset")
)
.from(ARTICLE_VIEW_COUNT)
.asTable("row_rank_tb")
)
.where(field("row_rank_tb.${ARTICLE_VIEW_COUNT.ARTICLE_ID.name}")!!.eq(query.articleId))
.query
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

이전 스크롤에서 읽은 마지막 아티클 ID가 몇번째 로우인지 판단함. 해당 로우 순번은 추후 Offset 으로 잘라내기 위해 사용

Comment on lines +52 to +77
private fun selectByArticleMstAndMemberAndMappingWorkbookArticleAndWorkbookQuery(articleIds: Set<Long>) =
dslContext.select(
ARTICLE_MST.ID.`as`(ArticleMainCardRecord::articleId.name),
ARTICLE_MST.TITLE.`as`(ArticleMainCardRecord::articleTitle.name),
ARTICLE_MST.MAIN_IMAGE_URL.`as`(ArticleMainCardRecord::mainImageUrl.name),
ARTICLE_MST.CATEGORY_CD.`as`(ArticleMainCardRecord::categoryCd.name),
ARTICLE_MST.CREATED_AT.`as`(ArticleMainCardRecord::createdAt.name),
MEMBER.ID.`as`(ArticleMainCardRecord::writerId.name),
MEMBER.EMAIL.`as`(ArticleMainCardRecord::writerEmail.name),
jsonGetAttributeAsText(MEMBER.DESCRIPTION, "name").`as`(ArticleMainCardRecord::writerName.name),
jsonGetAttribute(MEMBER.DESCRIPTION, "url").`as`(ArticleMainCardRecord::writerImgUrl.name),
jsonArrayAgg(
jsonObject(
key("id").value(WORKBOOK.ID),
key("title").value(WORKBOOK.TITLE)
)
).`as`(ArticleMainCardRecord::workbooks.name)
)
.from(ARTICLE_MST)
.join(MEMBER).on(ARTICLE_MST.MEMBER_ID.eq(MEMBER.ID)).and(ARTICLE_MST.DELETED_AT.isNull)
.and(MEMBER.DELETED_AT.isNull)
.leftJoin(MAPPING_WORKBOOK_ARTICLE).on(ARTICLE_MST.ID.eq(MAPPING_WORKBOOK_ARTICLE.ARTICLE_ID)).and(MAPPING_WORKBOOK_ARTICLE.DELETED_AT.isNull)
.leftJoin(WORKBOOK).on(MAPPING_WORKBOOK_ARTICLE.WORKBOOK_ID.eq(WORKBOOK.ID)).and(WORKBOOK.DELETED_AT.isNull)
.where(ARTICLE_MST.ID.`in`(articleIds))
.groupBy(ARTICLE_MST.ID)
.query
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JDBC URL allowMultiQueries=true 옵션 추가후 jsonArrayAgg 정상 동작 확인했스비다~

Copy link
Member Author

@hun-ca hun-ca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

하나 질문좀...

Comment on lines +76 to +97
fun selectArticlesOrderByViewsQuery(query: SelectArticlesOrderByViewsQuery): SelectLimitPercentStep<Record2<Any, Any>> {
val articleViewCountOffsetTb = select()
.from(ARTICLE_VIEW_COUNT)
.where(ARTICLE_VIEW_COUNT.DELETED_AT.isNull)
.orderBy(ARTICLE_VIEW_COUNT.VIEW_COUNT.desc())
.limit(query.offset, Long.MAX_VALUE)
.asTable("article_view_count_offset_tb")

val baseQuery = dslContext.select(
field("article_view_count_offset_tb.article_id").`as`(SelectArticleViewsRecord::articleId.name),
field("article_view_count_offset_tb.view_count").`as`(SelectArticleViewsRecord::views.name)
)
.from(articleViewCountOffsetTb)

return if (query.category != null) {
baseQuery.where(field("article_view_count_offset_tb.category_cd").eq(query.category.code))
.limit(10)
} else {
baseQuery
.limit(10)
} // TODO: .query로 리턴하면 리턴 타입이 달라져 못받는 문제
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

query.category가 널이 아닐 경우만 where절을 타야하는 동적 쿼리인데, jOOQ에서 요렇게 짜는거 맞나요? 조언 부탁드립니다.
추가로 마지막에 .query를 하면 각각 경우에 리턴 타입이 달라져서 받기 어렵더라구요.. Any로도 못받음..

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

요런 식으로 하시면 될 것 같아요

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Copy link
Collaborator

@belljun3395 belljun3395 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

어프루브 늦었네요.! 🙇🏻

Copy link
Member Author

@hun-ca hun-ca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

갑자기 생각난 의문사항...

Comment on lines +121 to +126
fun selectArticleContentsQuery(articleIds: Set<Long>) = dslContext.select(
ArticleIfo.ARTICLE_IFO.ARTICLE_MST_ID.`as`(SelectArticleContentsRecord::articleId.name),
ArticleIfo.ARTICLE_IFO.CONTENT.`as`(SelectArticleContentsRecord::content.name)
).from(ArticleIfo.ARTICLE_IFO)
.where(ArticleIfo.ARTICLE_IFO.ARTICLE_MST_ID.`in`(articleIds))
.and(ArticleIfo.ARTICLE_IFO.DELETED_AT.isNull)
Copy link
Member Author

@hun-ca hun-ca Jul 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

아티클 컨텐츠만 뽑아오는건데 생각해보니까 이건 매번 스크롤마다 10개 아티클에 대해서 모두 수행되기 때문에 disk I/O 부하가 다소 있을거 같습니다

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

그런데 어딘가에서는 저장되고 io로 불러와야하니까 선택의 문제가 아닐까요?

우선 db에서 불러오고 쫌 더 좋은 방법 고민해봐요!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

글고 생각해보니까 요거는 정책으로 풀 수 있는 문제가 아닌가 합니다.

썸넬에서는 최대 5줄 보여준다 하면 우리가 미리 잘라서 넣어둘 수 있으니까요

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

맞아요 일단 메모리에 저장하면DB에서 꺼내는것보단 빠르긴 함. 말씀대로 5줄로 쭐이면 좋긴할듯

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

잴 많이 호출되는 api에서 잴 많이 disk I/O 부하를 주는 셈. 그렇다고 이걸 로컬 캐시에 넣는게 맞을까요?

@hun-ca hun-ca added the ⭐major⭐ 주요 이슈(추후 회고 및 복귀 시 참고용) label Aug 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature 새로운 기능을 만들 때 사용됩니다 ⭐major⭐ 주요 이슈(추후 회고 및 복귀 시 참고용)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

articles 목록 조회 API 신규 개발
2 participants