Chapter 1. MAVE Survey

Introduction

MAVE is Worth Publishers' new "Mechanical Answer Validity Evaluator". Based on contemporary research in computational linguistics, machine learning, and data mining, MAVE's purpose is to determine if a student made an honest attempt at answering a free-response (e.g. essay) question. It also checks for plagiarism—to make sure that two students don't enter the same (or very similar) answers to a question. We'd like to know what you think about MAVE.

The first important thing to understand is what MAVE does not try to do. According to the experts we've talked to, no current technology exists that will allow a machine learning algorithm to accurately evaluate the correctness of a free-response answer in a scalable way. The best we can do is determine the validity of an answer: whether or not the answer reflects an honest effort on the part of the student to attempt to answer the question.

Still, we think that an effective validity checker would be a huge pedagogical boon for formative assessment activities. Formative activities such as Worth's LearningCurve are great tools for learning, but the types of questions that LearningCurve and similar systems use (e.g. multiple-choice and fill-in-the-blank) can really only challenge students on the lower Bloom's levels (remembering, understanding, and, to some extent, applying). Using MAVE, we can ask students essay-type questions that get at higher Bloom's levels (analyze, evaluate, and create). Again, MAVE can't tell us whether students have mastered concepts at this level, but at least it can guarantee that students have thought about concepts at this level, and we think that getting students to think is a worthy goal in and of itself.

The tabs in this activity feature a series of questions that are designed to introduce you to MAVE. Please complete them in order and then fill out the survey at the end. Thanks!

MAVE (Mechanical Answer Validity Evaluator) Survey
false
true

1.1 The Sky (traditional)

Traditionally, when essay questions are given online, students receive credit as long as they enter something. Sometimes they might be required to enter a minimum number of words, but still, students are given credit for virtually any answer—even if it's clearly not an honest attempt at answering the question.

The question below is an example of such a question. Try it, and see what you can get away with entering while still receiving credit. On the next page, you'll see the difference that MAVE can make.

Question

XcScy3O7x9Qu+ykI4SfJlyM7E6chBPFD50E+jB1UbKoGw3e3
Correct.

1.2 The Sky (MAVE)

MAVE can determine whether an honest attempt was made at an answer. It works by analyzing the "features" (i.e. words and phrases) in the student's answer and comparing them to features of other answers that it knows are valid or invalid attempts at an answer.

Before you make an honest attempt at an answer, try a few things you think your students might enter if they didn't want to really answer the question (e.g. "I don't feel like answer this right now", or something like that). These attempts should be rejected by MAVE, but pay attention to the options the activity gives you when it thinks you've entered an invalid attempt.

Question

T7zCgj0P2ivOtw2u5ZDflSUQFcrCwDKvNes/CnrdpuldVWoUhPWV3RUItMHMb/ALxOlztUmSkC8HeTg0
When light from the sun hits gas molecules and dust in the air, some wavelengths are scattered while others are absorbed or pass right through. Blue wavelengths get scattered more often then other wavelengths. When we look up we see that scatted blue sunlight coming from everywhere, so the sky appears blue.

Once you have submitted a valid answer above, try submitting the same answer again to this copy of the question. MAVE should detect that you are copying from an answer that has already been submitted, even if you change a few words in the answer.

Question

T7zCgj0P2ivOtw2u5ZDflSUQFcrCwDKvNes/CnrdpuldVWoUhPWV3RUItMHMb/ALxOlztUmSkC8HeTg0
When light from the sun hits gas molecules and dust in the air, some wavelengths are scattered while others are absorbed or pass right through. Blue wavelengths get scattered more often then other wavelengths. When we look up we see that scatted blue sunlight coming from everywhere, so the sky appears blue.

1.3 Healthy Eating

Source: The Big Hurt by CWK Network/Connect with Kids (www.connectwithkids.com)]

The following activity is an example of part of an actual activity where we plan to use MAVE. (It's part of our "Developmental Video Tool Kit" series.)

It would be great to have a signal system that changed our taste preferences. A possible interior monologue would go something like this:

“Looks like there is plenty of food and there is going to be abundant food in the foreseeable future. The layer of fat in my body is at an optimal level. The regulatory system and the sensation/perception system regarding taste and smell will now change so that foods high in sweets and fats will be rejected as not very appealing foods.”

Unfortunately, it doesn’t work that way. Foods that are no longer beneficial are still appealing.

Play the video to listen to some older children and their parents discuss food choices and some problems that overweight children face.

Question

8wVQ0PCwitmFBBXXH+jyMMCDnX3cvzOC7vaZvxlfD6BQVDOwiRB2FR9kbNR57R8eegO0PaMOSCnQx1smOB6Hvw/Q+PFZ0Vnzwqci7YL+gHq9EKfaY62h9rAtF3wiajURkcKUp+7POJP/Vt/R0vn+kFzabfM7I+R5mFOZ4lY5CLwe9EtrmHfYna1F+Vz8PVQPKyVSOd5j459QFPhzypJjnNCKwr+J3ix9hvMTZMbAJasI9SF8keIfr4ChessO7cZhPgFhVcyaCTBWvymW140nBqo9ZXRSN7PeyUW/JaMyR4dLlYdvua1J3qSWbobIhun0fZHwVZZU7S9FcD0SZd9IuxD0DgtyhAotfwTNAaO9sIKRDgQr75qfOHBP6p+8C8hU
Children and adults sometimes eat unhealthy foods as a reward after a stressful day or a special event. Foods may be associated with certain activities, such as popcorn or candy with watching a movie. Adults often feel time pressures and may settle for unhealthy food choices because these foods can be purchased and eaten quickly. Because adults have more control over food choices, they can decide to make a commitment to increase healthy foods and decrease fast foods. Unfortunately, a 10-year-old cannot do this without parental support!

1.4 Survey Questions

Question 1.1 About your courses

YmUtev9cmO7jzq+mDP2UoAtbQwP/hD6Uv4m5v3rRB0nYmzP3/V3kt9R+d5V+Q9dpldskkFSRMnSe79H4ZA1dEf77nO1GcIVXyjWYszKb+le1mSmtF7ZzYQMZTGcAGbsa1hPxYPwhNWdEYu+TQSBb28eCZ29UM68Kzd+84pmDkp7CTl/JHtVuq2q8qpqIeirL zIfjZdmJ5RyTRal5gOEJB1uH8nKy/SqnAKJJAhVvPfDV2AK+Oiw2fQ2OyDdMFcIAhRRIIVaSOt+9sdplRXoMWsXAbgLZPWq9

Question 1.2 Your Experience

Gnoct14ojExIfyLjw3qT1LYaF4j3JUrWH55JonpRlSnP7knPqIEMCnImoLI2HcLEdRYVRoH7MWRzEWFiQ7z+Mrac4gpkCZ+m5lIKJIBi2f8Znl3F1g3/EE3X4uZC50N0zhRcTwXwr2+rVx2l6/OGh100P73GUhpaYPVcvkAB+Wn2AThkKBab9BLLDA1M2iBP1IfrnGtQDS0=

Question 1.3 Using MAVE

DrQxgaDNahLnH/y69WbulHGbqPNYmooAjHrqR8P/4MJkFGzEdJhsJfZVAgsKxLqYUQwfeumzMcMK28JzTi0eLZn69zAO155wfSSyZlzavKag6ENIxJ584iSOygjTn0wk+ndsdtbKq9GabU5CtPn06W3+nD8PIAOANlGNU/sGajyU9lPlFkIF0cJHpDUtziwEGPRKkDDo51OnOUJsKAJzdFktyjbx5LVmkf/bWaYBi/w+3eYC3RMXN9Sl33U9N3yhiNiE69aAzwaDq9tvL04dFTn1NJRJ1Atf59ZFui7dkfHfxwqoYTTVxoWrBcq1AFLQ3TJz9evYLMA8qYlX+0bpcmTNsyk5eZMi//HjfhJCPYEhHqg4I/VLHA== r7vPzvfrUBRlsL/Utln6q/XjvH8U8TUrHw7OeD8MM1ag5gurXfewm52/kJ2MsOGFjixYt/oa5FgWyOE0m9c9hn4OJeucZ+P/6wXPBNOLAieFvX3g/+tKmbhNBjmotj65P2syMJtXLd/ut33jq48s5q0GeZ73HWaDMsKfhwF+mre+eimuRPEiSfy0PR0CxttUQzTSW8N94OzPnhtwFyjCA6nLDL/Czzraoepus5AX03Iun8rlZMMggXFkaBPn9vh7mO0qy7SIxzdGGEGN9GFWk76YIwf6+hXkoXr4qGq9Ki3KbEL2chi2BbOFJNI3EvEaqH8U4/ZkYVg1NV9RqIV19odLEuseF/EC1LL4eEnDiRH9RZXOxMwXYprGI0CO3xsdueaB0WL+9XwYeKeoPwe+DEme1vvIO0BS4BLvVmQlG33hFWOg7KbQi/E/L0q1SYm2vao4QDXd+rtkQgch8DnIgvGUKmInshoMRHlhMtNg181dIOEGzFmbTRPDm50lpvBGkf0F/v02yvkwmvveN3p7aiCj2MepNo/EcH0Y6dw2NqFaKQp24NN2Y6GSML+93ktVQM8YZ1FdDv1L9s4H53hyxzqS3tR+aX3A5Os6alPhhx9N7ruIfekITPEiRf4e+0MSsgvbgsRZDCUNzcUdbGhJuxzjkNoLb39fuPkMo4KL56Nnw4/DwKn5xre8u6bUKiPHuWdOpUe0PI9SS1gM0DgiICH+aCXFNQmrSDXPu+s0BAgjm1WslUSTMYibFo2ElGL07xJOieKJtLbAOl4nr60wjy4OkKUKZzIED36pvVDwMeso3JU1NNjED2usL/Gx7dXSiZAjVSqLUH25Hmkcn/rMflnQc4E= 2eU/oPxuDyfM5uI0KvYxstW4ElO8l+7CB3UbclIA0/apPOyLLbx6ZHoNGm57FK09UZegw2bwTfaHewfezO7T7sLodFM0gcL6FelI8vK7R5Oa0/w9BqJ9fdlFS4vYLy6w0Z+eqLNrVoIBnlKa+LAm7MKEJf/KfFtHb/PFywciDe0pVZ/GV+wC6Wz1j6YmtQvFoeG9E03tQTum+6lDsJ7X8GsGUD23p6z/td++PPwTxRYIFpUav8sVcxGnLDUfVM5QtC2MGBAwuhx9IShZ+xyY7U7FjVU+SmgsKo9qrwxmMqpcjQzbGsUZsasciTW99ndBn1pKvzSosvI8r2vl6l11kdcAjBL3yN8S4VtAToFdf1OpX97pv2EPKp9aK/Smw1cxDgcjt7/VM43Mg+4PSaAjChZbtMcuvgA2XuNqhhhvyzQxdO1Akm4ZHXWHfGf5mknHb+dH5vuzMuJE5A7PnnSULyLLKd2wnM4vOr41o0HziZZ9ZV1RmnaYliX1n1P6puc3G7lCydjnlBtjaP0ZNTPwAA==

Question 1.4 Other Comments or Feedback

h9XRYmlzFjvPcyc78WP9XaU+B+8TneHEfMXmCw/QEHPUQf1ukpo942d/JDJ7+FnyGgwvj6CYjHddt4xjQunZNwvwxOJWNpitX189Js/3/HUN+C5kMHgtk+A647x17JlvQpr7KG+jZtBAP5/A8VRx3IuaSp0= haAqwnA2OeWGULHZWwhSRVBqqfMRiayhDfkJB8CzH6ir/UQvQzhXBtJ1YRFA7g46fsbg0sdvNwR6/hYVqTsouqJEJQvorxBkEiutODRQXmQmQAs6XkPnj+fYsP9H9OzG6+EscZutr6VpsOBFqrQ+QMxlXayRKX3FQmJ+5izRbIlOame3cFsKrw==