Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs(Meeting Notes) update documentation of week-04 meetings. #239

Merged
merged 1 commit into from
Jul 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/2024/scheduler/asset/arch_ver.01.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
54 changes: 54 additions & 0 deletions docs/2024/scheduler/updates/2024-05-30.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
title: Weekly Updates (Community Bonding)
author: Aaditya Singh
---
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
SPDX-FileCopyrightText: 2024 Aaditya Singh <[email protected]>
-->

# Meeting 5

*(May 30, 2024)*

## Attendees

- [Shaheem Azmal M MD](https://github.com/shaheemazmalmmd)
- [Gaurav Mishra](https://github.com/GMishx)
- [Kaushlendra Pratap](https://github.com/Kaushl2208)
- [Ayush Bhardwaj](https://github.com/hastagAB)
- [Samuel Dushimimana](https://github.com/dushimsam)
- [Soham Banerjee](https://github.com/soham4abc)
- [Shreya Singh](https://github.com/SinghShreya05)
- [Abdelrahman Jamal](https://github.com/Hero2323)
- [Aaditya Singh](https://github.com/Aaditya-Singh78)
- [Abhishek Kumar](https://github.com/abhi-kumar17871)
- [Akash Sah](https://github.com/Akashsah2003)
- [Divij Sharma](https://github.com/dvjsharma)
- [Rajul Jha](https://github.com/rajuljha)
- [Valens Niyonsenga](https://github.com/valens200)

## Discussion

### Mentors

- [Shaheem Azmal M MD](https://github.com/shaheemazmalmmd): Provided general updates regarding the project. Reminded contributors to update the GSoC Documentation tool before the Thursday call.
- [Gaurav Mishra](https://github.com/GMishx): Explained the rules and use of the GSoC Documentation tool, including how to contribute code and documentation.

### Contributors

- [Abdelrahman Jamal](https://github.com/Hero2323): Presented basic code to parse file text from FOSSology and prompt it to different LLMs. The LLMs provide license names and handle multiple licenses or return empty if no license is found. Currently assessing which LLM performs best.

- [Shreya Gautam](https://github.com/ShreyaGautamm): Absent.

- [Aaditya Singh](https://github.com/Aaditya-Singh78): Completed FOSSology installation and created a pull request to enhance cross-platform compatibility and line encoding issues. Discussed with the mentor how to resolve an issue with the mimetype agent.

- [Abhishek Kumar](https://github.com/abhi-kumar17871): Exploring SPDX 3.0 JSON format and plans to open a pull request soon.

- [Akash Sah](https://github.com/Akashsah2003): Investigated storing license expressions and proposed using Abstract Syntax. Has shared a document with mentors for decision-making.

- [Divij Sharma](https://github.com/dvjsharma): Reported no major updates; currently exploring OAuth implementation via API.

- [Rajul Jha](https://github.com/rajuljha): Discussed general milestones for the CI/CD project during a call with mentors. Working on line number extraction and differential scans.

- [Valens Niyonsenga](https://github.com/valens200): Limited updates, reviewed the concept regarding Monk-based text scanning and needs to discuss further how to start.
98 changes: 98 additions & 0 deletions docs/2024/scheduler/updates/2024-05-31.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
---
title: OVERHAULING SCHEDULER DESIGN (Discussion)
author: Aaditya Singh
---
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
SPDX-FileCopyrightText: 2024 Aaditya Singh <[email protected]>
-->

# Meeting 6

*(May 31, 2024)*

## Attendees

- [Anupam Ghosh](https://github.com/ag4ums)
- [Shaheem Azmal M MD](https://github.com/shaheemazmalmmd)
- [Gaurav Mishra](https://github.com/GMishx)
- [Kaushlendra Pratap](https://github.com/Kaushl2208)
- [Aaditya Singh](https://github.com/Aaditya-Singh78)

## Discussion

### Contributor

- [Aaditya Singh](https://github.com/Aaditya-Singh78): Focused on framing the current scheduler by studying the Job Scheduler `wiki` [Documentation](https://github.com/fossology/fossology/wiki/Job-Scheduler).

#### Current Scheduler Design

The schematic from the documentation provides an architectural overview of the current Job Scheduler:
<!-- change it! -->
![CurrentDesign](../asset/c_arch.jpg)

**Architectural Overview:**

The design is based on a client-server model, featuring:

1. **Scheduler Port**: Serves as the communication hub between clients and the scheduler.
2. **Main Thread**: Responsible for job queuing, allocation, and event management.
3. **Worker Threads**: Implement specific tasks asynchronously, enhancing throughput and response times.

**Key Components:**

1. **Scheduler**: Manages all job scheduling operations and acts as the primary interface for job requests.
2. **Asynchronous Workers**: Improve processing time by handling tasks in parallel.
3. **Logging and Monitoring**: Provides integrated systems for real-time monitoring and issue troubleshooting.

### Alignment with Documentation

**Mentor:**
- [Gaurav Mishra](https://github.com/GMishx): Confirmed alignment with the current architecture of the job scheduler.

#### New Scheduler Design

The revised design focuses on a more efficient multithreaded approach:
<!-- change it! -->
![NewDesign](../asset/arch_ver.01.jpg)

**Architectural Overview:**

1. **Main Thread**: Coordinates with various components and manages task execution.
2. **Worker Thread**: Manages interactions between different services to prevent conflicts.
3. **Event Queue**: Ensures proper handling of system-level events and error logging.

**Key Components:**

1. **Scheduler**: Uses a round-robin technique to manage tasks, ensuring efficient CPU time distribution.
2. **Queue Storage**: Manages tasks dynamically to adapt to workload changes.

#### Trade-offs

| Feature | Old Job Scheduler | New Job Scheduler |
|-----------------------|----------------------|----------------------|
| Maintainability | Easier | More challenging |
| Monitoring | Centralized | Decentralized |
| Resource Management | Static | Dynamic |

### Effectiveness of New Design

**Mentor:**
- [Gaurav Mishra](https://github.com/GMishx): Suggests a finite state machine architecture might be more suitable given the complexity, as it facilitates easier tracking and management.

### Addressing FOSSology Issue

**[#2742](https://github.com/fossology/fossology/issues/2742)**: Problem with incorrect MIME type recognition for text files.

**Mentor:**
- [Shaheem Azmal M MD](https://github.com/shaheemazmalmmd): Error found due to the "Ignore SCM files" option affecting MIME type recognition.

### Wfx Project Approach

**Mentor:**
- [Kaushlendra Pratap](https://github.com/Kaushl2208): Expressed interest in holding an internal discussion about the [wfx](https://github.com/siemens/wfx) project approach.

### Goals for the Week

**Contributor:**
- [Aaditya Singh](https://github.com/Aaditya-Singh78): Goals include rewriting the existing C code to Go and implementing a queue in Golang.