-
-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Building a training set of tags for 8th #109
Comments
Exercise: hello-worldCode
Tags:
|
Exercise: armstrong-numbersCode
Tags:No tags generated |
Exercise: isogramCode
Tags:
|
Exercise: acronymCode
Tags:No tags generated |
Exercise: gigasecondCode
Tags:
|
Exercise: trinaryCode
Tags:
|
Exercise: dartsCode
Tags:
|
Exercise: dartsCode
Tags:
|
Exercise: atbash-cipherCode
Tags:
|
Exercise: resistor-colorCode
Tags:
|
Exercise: etlCode
Tags:
|
Exercise: pop-countCode
Tags:
|
Everything after "\ " is a comment. So the guesses are a bit wide of the mark sometimes |
Yes, I'd expect lower-frequency languages like 8th to be less accurate, but once the tags are updated and we retrain it, it should learn :) |
This is an automated comment Hello 👋 Next week we're going to start using the tagging work people are doing on these. If you've already completed the work, thank you! If you've not, but intend to this week, that's great! If you're not going to get round to doing it, and you've not yet posted a comment letting us know, could you please do so, so that we can find other people to do it. Thanks! |
@axtens I will not be able to help out with this. |
Glenn, you've already done more than most -- well beyond the call of duty.
…On Wed, 15 Nov 2023 at 02:42, Glenn Jackman ***@***.***> wrote:
@axtens <https://github.com/axtens> I will not be able to help out with
this.
—
Reply to this email directly, view it on GitHub
<#109 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABTFVOY33FYB5APRN5AHY3YEO3RTAVCNFSM6AAAAAA6X2OL52VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJQHEZDONJYHE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
I will get to it. Not this week though.
…On Tue, 14 Nov 2023 at 20:45, Erik Schierboom ***@***.***> wrote:
*This is an automated comment*
Hello 👋 Next week we're going to start using the tagging work people are
doing on these. If you've already completed the work, thank you! If you've
not, but intend to this week, that's great! If you're not going to get
round to doing it, and you've not yet posted a comment letting us know,
could you please do so, so that we can find other people to do it. Thanks!
—
Reply to this email directly, view it on GitHub
<#109 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABTFVMD2VKBXN7PQXKRG5LYENRVXAVCNFSM6AAAAAA6X2OL52VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJQGEZTQMBZGU>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
@axtens Do you know when you'll have time? |
Will start tomorrow (Wednesday 2023-11-22) night.
…On Tue, 21 Nov 2023 at 21:33, Erik Schierboom ***@***.***> wrote:
@axtens <https://github.com/axtens> Do you know when you'll have time?
—
Reply to this email directly, view it on GitHub
<#109 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABTFVPKMM6DF7AOAFXHYG3YFSUSRAVCNFSM6AAAAAA6X2OL52VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRQHEZTKNRZGY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
@iHiD @ErikSchierboom you seem to have mis-scanned some. There is a full-blown version in the .meta folder but you seem to have taken some of the examples from the root of the task rather than its .meta. |
Yes, sorry about that! |
@axtens Did you get a chance to work on this? If not, that's fine too, and we can do it later. |
I've got as far as building a Google Sheets sheet with the various headings
and checkboxes. I just need time, something I don't have a lot of at the
moment.
https://docs.google.com/spreadsheets/d/10UfL8rsU1fzwyOsPjR4yDwvKGRG0FsB5F3-PwLwq0gs/edit?usp=sharing
…On Thu, 23 Nov 2023 at 20:25, Erik Schierboom ***@***.***> wrote:
@axtens <https://github.com/axtens> Did you get a chance to work on this?
If not, that's fine too, and we can do it later.
—
Reply to this email directly, view it on GitHub
<#109 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABTFVPRACRZ5W36O5YRSXTYF46FLAVCNFSM6AAAAAA6X2OL52VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRUGM2DKNBTGU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Hello lovely maintainers 👋
We've recently added "tags" to student's solutions. These express the constructs, paradigms and techniques that a solution uses. We are going to be using these tags for lots of things including filtering, pointing a student to alternative approaches, and much more.
In order to do this, we've built out a full AST-based tagger in C#, which has allowed us to do things like detect recursion or bit shifting. We've set things up so other tracks can do the same for their languages, but its a lot of work, and we've determined that actually it may be unnecessary. Instead we think that we can use machine learning to achieve tagging with good enough results. We've fine-tuned a model that can determine the correct tags for C# from the examples with a high success rate. It's also doing reasonably well in an untrained state for other languages. We think that with only a few examples per language, we can potentially get some quite good results, and that we can then refine things further as we go.
I released a new video on the Insiders page that talks through this in more detail.
We're going to be adding a fully-fledged UI in the coming weeks that allow maintainers and mentors to tag solutions and create training sets for the neural networks, but to start with, we're hoping you would be willing to manually tag 20 solutions for this track. In this post we'll add 20 comments, each with a student's solution, and the tags our model has generated. Your mission (should you choose to accept it) is to edit the tags on each issue, removing any incorrect ones, and add any that are missing. In order to build one model that performs well across languages, it's best if you stick as closely as possible to the C# tags as you can. Those are listed here. If you want to add extra tags, that's totally fine, but please don't arbitrarily reword existing tags, even if you don't like what Erik's chosen, as it'll just make it less likely that your language gets the correct tags assigned by the neural network.
To summarise - there are two paths forward for this issue:
If you tell us you're not able/wanting to help or there's no comment added, we'll automatically crowd-source this in a week or so.
Finally, if you have questions or want to discuss things, it would be best done on the forum, so the knowledge can be shared across all maintainers in all tracks.
Thanks for your help! 💙
Note: Meta discussion on the forum
The text was updated successfully, but these errors were encountered: