From e0352aaac57fa3be064f8ee0b3e4aebf3e6286a9 Mon Sep 17 00:00:00 2001 From: Young Date: Wed, 18 Sep 2024 02:13:08 +0000 Subject: [PATCH] fix: actively raised errors aer also considered as negative feedback. --- rdagent/components/coder/factor_coder/prompts.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rdagent/components/coder/factor_coder/prompts.yaml b/rdagent/components/coder/factor_coder/prompts.yaml index 66d2c5bf..a3423ace 100644 --- a/rdagent/components/coder/factor_coder/prompts.yaml +++ b/rdagent/components/coder/factor_coder/prompts.yaml @@ -212,7 +212,7 @@ evaluator_final_decision_v1_system: |- The implementation final decision is considered in the following logic: 1. If the value and the ground truth value are exactly the same under a small tolerance, the implementation is considered correct. 2. If the value and the ground truth value have a high correlation on ic or rank ic, the implementation is considered correct. - 3. If no ground truth value is not provided, the implementation is considered correct if the code execution is successful and the code feedback is align with the scenario and factor description. + 3. If no ground truth value is provided, the implementation is considered correct if the code executes successfully (assuming the data provided is correct). Any exceptions, including those actively raised, are considered faults of the code. Additionally, the code feedback must align with the scenario and factor description. Please response the critic in the json format. Here is an example structure for the JSON output, please strictly follow the format: {