CSE6040_SP20_Midterm1_Solutions_Courseware_edX.

Problem 0: Caption Contest Version 1.3b This problem is a data mining task that exercises basic data structure manipulation, strings (and maybe regex!), and translation of simple math to code. It has 5 exercises, numbered 0-4. These depend on one another as follows. Exercises 0 (1 point), 1 (3 points), 2 (2 points), and 3 (2 points) are all independent from one another. Therefore, you can do any of them for partial credit. Exercise 4 (2 points) depends on successful completion of Exercise 3. Pro-tips. If your program behavior seem strange, try resetting the kernel and rerunning everything. If you mess up this notebook or just want to start from scratch, save copies of all your partial responses and use Actions → Reset Assignment to get a fresh, original copy of this notebook. (Resetting will wipe out any answers you've written so far, so be sure to stash those somewhere safe if you intend to keep or reuse them!) If you generate excessive output (e.g., from an ill-placed print statement) that causes the notebook to load slowly or not at all, use Actions → Clear Notebook Output to get a clean copy. The clean copy will retain your code but remove any generated output. However, it will also rename the notebook to . Since the autograder expects a notebook file with the original name, you'll need to rename the clean notebook accordingly. Good luck! Background Every week, the New Yorker magazine runs a cartoon caption contest. It presents readers with a cartoon having no caption, and then invites readers to submit their ideas. For example, run the following code cell to see a cartoon from a couple weeks ago: In [1]: Out[1]: You should see a picture of a piece of bread, wearing a bowtie, standing in front of a host stand at a restaurant or other event. from problem_utils import get_path, display_image display_image(get_path("toast/")) # Number 695 at 3/1/2020 Midterm 1: Solutions | Midterm 1: Solutions | CSE6040x Courseware | edX import json with open(get_path('toast/'), 'rt', encoding='utf-8') as fp: captions_json = (fp) print(f"== The dataset contains {len(captions_json)} captions. The first four are:") captions_json[:4] When the exam is over, you should see if you can come up with even better schemes than what is developed here! The data. The New Yorker allows readers to vote on the submitted captions ( to get these submissions, which will serve as your dataset. (These data are just the captions, not the votes.) The data are stored in a JSON file, which Python can easily read using the json module ( load the submitted caption data and inspect the first four captions: In [2]: == The dataset contains 2458 captions. The first four are: Out[2]: [{'target_id': 0, 'primary_type': 'text', 'primary_description': 'I told you not to pick the one from the pilot experiment...'}, {'target_id': 1, 'primary_type': 'text', 'primary_description': 'Well that explains the gas station'}, {'target_id': 2, 'primary_type': 'text', 'primary_description': "The dairy-free vegan soy cheese doesn't seem to be having the same effect..."}, {'target_id': 3, 'primary_type': 'text', 'primary_description': 'Repeatedly, cheese demonstrated characteristics of a performance enhancing drug'}] Observe that the variable holding this caption data, captions_json, is a list of dictionaries. Each list element is the data for a single caption, where the caption text itself is a string value associated with the primary_description key. Your task: Data mining the captions for "comedic gold" There are a lot of submissions (even more than the 2,400+ in the data above), but the first four captions above are kind of strange, and do not seem to match the image. Can we mine the submissions to find more relevant, and even funny, ones, automatically? That is your task in the exercises below. Extraction and basic cleaning Exercise 0 (1 point). Complete the function get_captions(captions_json), below, subject to these requirements: 1. The input, captions_json, is an object just like the one loaded above (a list of dictonaries with the given keys and values). 2. The function returns a list of just the text (string) captions. For example, after running captions_orig = get_captions(captions_json) the returned value, captions_orig, should look like captions_orig == ['I told you not to pick the one from the pilot experiment...',