I can't say no. People ask me to be a science fair judge. If they have good food, I am there. Of course there are problems with the science fair.
- Students tend to focus on non-important things like the materials they used.
- There is often a lack of error analysis on any level. Students will take 1 data point and draw conclusions based on this.
- Don't get me started on "and so in the end my hypothesis was correct".
But we all want science fairs to work. We want them to be awesome. Change is difficult. But that is not what I want to talk about. I want to instead share a science fair project idea with you. I was actually going to do this project myself - but you know ...time...it keeps on ticking. There is only one of me (for now).
How Reliable are Science Fair Judges?
That is the title of my proposed science fair project. If you really insist on have a hypothesis, I guess it would be the sam as the title. Sort of a meta-science fair project, no?
The Problem
If you have been a judge, you know the problem. Suppose you have 3 different judges. Judge A thinks that the worst project shouldn't get anything lower than an 80 out of 100 score. Judge B wants to make sure that only the really good projects get a score of over 80. Judge C just came for the food.
The Plan
My first thought was to get more judges than I needed and have a set all judge the exact same projects. Then find some way to do an inter-rater reliability (something that I have looked at before - but I forgot how to do it). Really, you could just make up some measure of how well they correlate their scores. Maybe some judges focus on some aspects (like font size) and other focus on other things like "did they have a materials list".
The problem with the above is that first you would have to do this during the science fair for your science fair project. That means you would either have to do the analysis super quick (and make a poster quick) or wait a year. What student is going to wait a year? The second problem is confidentiality - you know, release forms and stuff. What a pain.
My second plan was to fake it. Make up maybe 10 fake science fair posters. I could even make fake presentations to go along with these. Either I could play the role of the student-presenter or I could get my kids to do it (why have kids if you can't use them for stuff like this?).
Even better. I could put these projects online and have a ton of people rate and evaluate several of them. Data would be pouring out of my ears. It would get all over the floor and make a mess.
Questions
- How many judges should evaluate each project to get a fair score?
- Are there any key features you can put in a poster or presentation to get a higher score? (like color or font)
- Do judges focus more on content, or presentation?
- How do judges respond (score) very creative projects that have an error?
- What would judges do if there was a project showing how cool the Power Balance bracelets were?
There you go. A project just for you. Let me know how well everything goes.
Oh! I forgot to list my materials:
- A very old Macbook Pro Computer
- A video camera
- A web sever
- Poster board
- My mom's color printer
- Thumb tacks
- 1 Kit Kat bar
