Skip to content

Commit

Permalink
First draft of the Open Research Challenge (#165)
Browse files Browse the repository at this point in the history
* added the first draft of the Open Research Challenge

* addressed toms comments

* added benjamins comments

* added if no live or video is possible the teams should request an exception

* remove the presentation judging criteria

* allow multiple contributions

* first vague note to multiple certificates

* added sentence for tied contributions

* add usability judging criteria
  • Loading branch information
arnemoos authored Dec 17, 2024
1 parent 587eb96 commit 58e9fbf
Show file tree
Hide file tree
Showing 3 changed files with 71 additions and 88 deletions.
4 changes: 2 additions & 2 deletions SPL-Challenges.tex
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ \section{Introduction}
\subsection{Code Publication}
Every team participating in a challenge must publish the corresponding code used in that competition according to Appendix A.7 of the SPL rule book, unless a specific challenge states otherwise.


\include{challenges/shared_autonomy_challenge}
\include{challenges/KICKin-Rollin-Challenge}
\include{challenges/open_research_challenge}

\end{document}
69 changes: 69 additions & 0 deletions challenges/open_research_challenge.tex
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
% !TeX root = ../SPL-Challenges.tex
% !TeX spellcheck = en_US
\section{Open Research Challenge}
The idea of this challenge is to create a platform within the Standard Platform League for teams to showcase innovations that contribute to the growth, development, and improvement of the league, beyond the core competition objectives. These innovations aim to enrich the league's ecosystem by fostering collaboration, technological advancement, and accessibility.

This challenge incentivizes teams to focus on league-wide growth areas such as software, hardware, and community alignment. It provides an opportunity for teams to make impactful contributions in areas like fostering innovation through creative solutions, strengthening collaboration within the SPL community, enhancing accessibility for new and existing teams, boosting engagement with spectators and sponsors, and supporting the long-term growth and sustainability of the league.

\subsection{Submission Requirements}
Teams must submit a brief description of their planned contribution by \textbf{July 1, 2025} to the Technical Committee (\url{[email protected]}).
Multiple contributions per team are allowed.
The description should include:
\begin{itemize}
\item \textbf{Objective}: The purpose and scope of the innovation.
\item \textbf{Expected Impact}: How it benefits SPL or improves existing systems.
\item \textbf{Implementation Feasibility}: Resources and time needed to accomplish/execute the contribution.
\end{itemize}

\subsection{Presentation Format}
Contributions will be presented through a \textbf{10-minute presentation} during the SPL competition.
Presentations must include:
\begin{itemize}
\item Clear explanation of the contribution and its impact.
\item A \textbf{live demonstration}, if feasible, or a \textbf{video demonstration} otherwise. If neither a live nor video demonstration is possible, the team must request an exception by notifying the Technical Committee and explaining the issue.
\end{itemize}
Presentations in the PDF format must be submitted to the Technical Committee on the day of the presentation and will also be uploaded to the \textbf{SPL website} for public review and reference.

\subsection{Full Code Release Requirement}
A \textbf{full code release} is mandatory for participating in this challenge. The code must be publicly available no later than the \textbf{main code release deadline} from the SPL rule book and must be announced over the \url{[email protected]} mailing list.

Ideally, the code should be released before the competition to allow interested teams and individuals to review and provide feedback!



\subsection{Evaluation Process}
Contributions will be evaluated during the competition using the \textbf{Borda count method\footnote{\url{https://en.wikipedia.org/wiki/Borda_count}}}, to ensure a fair and democratic assessment. Each team leader of the participating teams ranks all the presented contributions except their own. The rankings are aggregated to determine the overall score for each team. Equal overall scores are allowed. In such cases, tied contributions will share the same position in the overall ranking.\\
If a team submits multiple distinct contributions, each will be evaluated and ranked separately, potentially resulting in multiple individual rankings for that team.

\subsubsection*{Judging Criteria:}
\begin{itemize}
\item \textbf{Relevance}: Alignment with SPL’s goals of growth and improvement.
\item \textbf{Feasibility}: Practicality of implementation within league resources.
\item \textbf{Impact}: Potential benefit to the league or teams.
\item \textbf{Innovation}: Originality and creativity of the idea.
\item \textbf{Usability}: Ease of use, integration, and adaptability of the contribution.
\item \textbf{Live Demonstration}: Preference should be given to contributions that include a live demonstration.
\item \textbf{Code Release}: Consideration of whether the code was made available beforehand for review and that it is well documented.
%\item \textbf{Presentation Quality}: The clarity, engagement, and effectiveness with which the contribution and its impact are communicated.
\end{itemize}

Depending on the number of contributions, it may be necessary to conduct multiple Borda count evaluations. Each evaluation can focus on different sets of judging criteria, allowing every contribution to be assessed under the most appropriate context and ensuring a more nuanced and fair comparison.

\subsection{Scope of Contributions}
Projects should focus on innovations beneficial to the league but not necessarily related to winning matches. Examples include:
\begin{itemize}
\item \textbf{Common Simulator}: A platform to test robot software binaries in a shared virtual environment.
\item \textbf{Universal Log File Format}: Standardizing data recording formats for easier analysis and collaboration.
\item \textbf{Interoperable Code Standards}: Aligning software frameworks across teams and leagues for better portability and sharing.
\item \textbf{Shared Software Architecture}: Creating modular interfaces to simplify the exchange of robot components between teams.
\item \textbf{Semi-Automatic Refereeing}: Utilizing field-side cameras and AI for referee assistance.
\item \textbf{Real-Time Game Statistics}: Broadcasting match data through automated systems and providing teams with these statistics.
\end{itemize}

\subsection{Contributions Archive}
All contributions will be showcased on the \textbf{SPL website}, including:
\begin{itemize}
\item The presentations from the competition.
\item Full code releases for each contribution.
\end{itemize}
This ensures that every innovation is documented, celebrated, and accessible for future reference and use by the SPL community.
86 changes: 0 additions & 86 deletions challenges/shared_autonomy_challenge.tex

This file was deleted.

0 comments on commit 58e9fbf

Please sign in to comment.